diff --git a/AGENTS.md b/AGENTS.md index 13a66cd..60c5ddd 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,7 +4,7 @@ Stable goals for this repository: -1. **Embeddable generator** — **`pip install docgen`** (or editable install from source), a `docgen.yaml`, and shell/CI are enough to build and maintain narrated demos. **No IDE assistant is required**; optional **`docgen wizard`** is a local web app only. +1. **Embeddable generator** — **`pip install docgen`** (git URL or editable install from this repo), a consumer **bundle** (`docgen.yaml` + hints/narration), and shell/CI are enough to build and maintain narrated demos. **Do not vendor** this library into a product repo’s `src/`; pin via `requirements-docgen.txt` / `pipx` / `uv tool`. **No IDE assistant is required**; optional **`docgen wizard`** is a local web app only. 2. **Hybrid config and prose** — **`docgen.yaml`** should stay maintainable: deterministic merges (**`yaml-generate`**, gap checks) plus **optional OpenAI** where it adds value (narration hints, declarative scene YAML). Prefer **Git-reviewed** changes over opaque single-shot generation. 3. **Video stack** — Long-form demos pair **Markdown narration**, **OpenAI TTS**, **Whisper-style timestamps**, **Manim** visuals, **`compose`** (ffmpeg), **`concat`**, and **`validate`** (sync and narration lint). The CLI also supports **`pages`** for static preview sites. 4. **Stable contracts** — CLI, exit codes, and reusable workflows should stay predictable for downstream repos and automation. @@ -41,7 +41,7 @@ The Playwright/VHS/demo-function/per-function/discover-tests/catalog surface are Commands registered on the **`docgen`** CLI include: - **`init`** — scaffold bundle layout and `docgen.yaml`. -- **`wizard`** — local web UI for narration/bootstrap workflows (focus files, **in-place narration revise**, per-segment **asset freshness** + **rebuild-from-here**). +- **`wizard`** — local web UI for narration/bootstrap workflows (focus files, **in-place narration revise**, per-segment **asset freshness** + **rebuild-from-here**, **Tool** tab to pip-upgrade docgen and pin `requirements-docgen.txt`). - **`tts`** — text-to-speech for segment files. - **`timestamps`** — word/segment timing (`timing.json`). Default engine **`local`** aligns the known narration text against the mp3 offline (ffmpeg silencedetect, no API); **`--engine whisper`** keeps OpenAI whisper-1 transcription. Both emit the same Whisper-shaped blocks. - **`image-generate`** — render scene-spec **image elements** (`image:` + `prompt:` boxes) via the OpenAI Images API into the bundle (also runs for missing assets inside `generate-all`). diff --git a/README.md b/README.md index 0312f9a..83c63e3 100644 --- a/README.md +++ b/README.md @@ -66,13 +66,31 @@ If you still need the legacy behaviour, pin a pre-removal commit OpenAI where a command calls the API). The wizard is a local Flask app, not a plugin tied to one editor. -## Install +## Install (external tool — do not vendor into project `src/`) + +`docgen` is a **pip-installable CLI/library**. Consumer projects keep only a +**bundle** (`docs/demos/docgen.yaml`, hints, narration, …). They should **not** +copy or submodule this repository into application source. ```bash -pip install docgen @ git+https://github.com/jmjava/documentation-generator.git +# Project / CI venv +pip install 'docgen @ git+https://github.com/jmjava/documentation-generator.git' +# Prefer a SHA pin in CI: +# pip install 'docgen @ git+https://github.com/jmjava/documentation-generator.git@' + +# Optional Manim extra +pip install 'docgen[manim] @ git+https://github.com/jmjava/documentation-generator.git' + +# Isolated global CLI (no project venv) +pipx install 'docgen @ git+https://github.com/jmjava/documentation-generator.git' +# or: uv tool install 'docgen @ git+https://github.com/jmjava/documentation-generator.git' + +docgen --version ``` -## Development setup +`docgen init` writes `requirements-docgen.txt` + a bundle `README.md` with the same guidance. + +## Development setup (this library) ```bash git clone https://github.com/jmjava/documentation-generator.git @@ -87,10 +105,11 @@ CI installs `ffmpeg` and `tesseract` via apt — see `.github/workflows/ci.yml`. **Roadmap:** [milestones/README.md](milestones/README.md). -## Quick start +## Quick start (in a consumer repo) ```bash -cd your-project/docs/demos +cd your-project/docs/demos # bundle only — library is on PATH via pip +pip install -r requirements-docgen.txt # after docgen init, or use the pip line above docgen wizard # optional: bootstrap narration from project docs docgen generate-all # TTS → timestamps → scene retime → Manim → compose → validate docgen validate --pre-push @@ -100,8 +119,9 @@ docgen validate --pre-push | Command | Description | |---------|-------------| -| `docgen init [TARGET_DIR] [--defaults] [--segments-file FILE]` | Scaffold a new project: `docgen.yaml`, wrapper scripts, directories | -| `docgen wizard [--port 8501]` | Local web GUI: focus files, **revise narration in place**, asset freshness chips, **rebuild-from-here** (default cascade: TTS → timestamps → scene-retime → Manim → compose → validate; LLM scene-spec is explicit) | +| `docgen --version` | Show installed version + recommended `pip install` line (external tool) | +| `docgen init [TARGET_DIR] [--defaults] [--segments-file FILE]` | Scaffold a bundle: `docgen.yaml`, `requirements-docgen.txt`, wrapper scripts, directories | +| `docgen wizard [--port 8501]` | Local web GUI: focus files, **revise narration**, asset freshness / rebuild-from-here, and a **Tool** tab to upgrade the installed `docgen` package (pip) + rewrite `requirements-docgen.txt` | | `docgen tts [--segment 01] [--dry-run]` | Generate TTS audio | | `docgen timestamps [--engine local\|whisper]` | Extract word/segment timestamps from TTS audio → `timing.json` (default `local`: offline narration-text alignment; `whisper`: OpenAI transcription) | | `docgen image-generate [--segment 01 \| --all \| --spec PATH] [--force] [--dry-run] [--model …] [--size …]` | Generate scene-spec image assets (`image:` + `prompt:` boxes) via the OpenAI Images API into the bundle | diff --git a/src/docgen/cli.py b/src/docgen/cli.py index 301f723..05df2c9 100644 --- a/src/docgen/cli.py +++ b/src/docgen/cli.py @@ -76,6 +76,16 @@ def _load_env(cfg: Config | None) -> None: os.environ.setdefault(k, v) +def _cli_version_string(ctx: click.Context, param: click.Parameter, value: bool) -> None: + if not value or ctx.resilient_parsing: + return + from docgen.install_spec import DOCGEN_PIP_SPEC, package_version + + click.echo(f"docgen {package_version()}") + click.echo(f"install: pip install '{DOCGEN_PIP_SPEC}'") + ctx.exit() + + @click.group() @click.option( "--config", @@ -84,9 +94,17 @@ def _load_env(cfg: Config | None) -> None: type=click.Path(exists=False), help="Path to docgen.yaml (parents of cwd are searched when omitted).", ) +@click.option( + "--version", + is_flag=True, + callback=_cli_version_string, + expose_value=False, + is_eager=True, + help="Show installed docgen version and the recommended pip install line.", +) @click.pass_context def main(ctx: click.Context, config_path: str | None) -> None: - """docgen — demo generation pipeline. + """docgen — demo generation pipeline (install as an external tool; keep only the bundle in-repo). Environment: keys already set in the shell are not replaced by ``env_file`` (see ``DOCGEN_ENV_OVERRIDES``). If no docgen.yaml is found, pass ``--config``. diff --git a/src/docgen/init.py b/src/docgen/init.py index 95f9bc4..da6a423 100644 --- a/src/docgen/init.py +++ b/src/docgen/init.py @@ -268,6 +268,8 @@ def generate_files(plan: InitPlan) -> list[str]: (plan.demo_dir / subdir).mkdir(parents=True, exist_ok=True) created.append(_write_config(plan)) + created.append(_write_requirements_docgen(plan)) + created.append(_write_bundle_readme(plan)) for path in _write_wrapper_scripts(plan): created.append(path) @@ -446,6 +448,62 @@ def _write_wrapper_scripts(plan: InitPlan) -> list[str]: return created +def _write_requirements_docgen(plan: InitPlan) -> str: + """Write ``requirements-docgen.txt`` so consumers install docgen externally.""" + from docgen.install_spec import requirements_docgen_txt + + path = plan.demo_dir / "requirements-docgen.txt" + path.write_text(requirements_docgen_txt(), encoding="utf-8") + return str(path) + + +def _write_bundle_readme(plan: InitPlan) -> str: + """Short bundle README: install docgen as a tool, keep only this bundle in-repo.""" + content = textwrap.dedent(f"""\ + # {plan.project_name} demos (docgen bundle) + + This directory is a **docgen bundle** (`docgen.yaml` + narration/hints/assets). + The ``docgen`` CLI itself is an **external** package — do **not** vendor the + ``documentation-generator`` source tree into this repository. + + ## Install the tool (once per machine / CI job) + + ```bash + python3 -m venv ../../.venv # or any project venv + source ../../.venv/bin/activate + pip install -r requirements-docgen.txt + # optional Manim: + # pip install 'docgen[manim] @ git+https://github.com/jmjava/documentation-generator.git' + ``` + + Isolated global install (no project venv): + + ```bash + pipx install 'docgen @ git+https://github.com/jmjava/documentation-generator.git' + # or: uv tool install 'docgen @ git+https://github.com/jmjava/documentation-generator.git' + ``` + + Verify: ``docgen --version`` + + ## Run from this bundle + + ```bash + cd {plan.demo_dir.name if plan.demo_dir.name else "."} + docgen wizard + docgen generate-all + docgen validate --pre-push + ``` + + Wrapper scripts (``generate-all.sh``, ``compose.sh``, …) activate a nearby + ``.venv`` when present, then call the ``docgen`` console script on ``PATH``. + """) + path = plan.demo_dir / "README.md" + if path.exists(): + return str(path) + path.write_text(content, encoding="utf-8") + return str(path) + + def _write_narration_readme(plan: InitPlan) -> str: content = textwrap.dedent("""\ # Narration scripts (TTS source) @@ -544,10 +602,13 @@ def print_summary(plan: InitPlan, created: list[str]) -> None: click.echo() click.secho(" Next steps:", fg="cyan") click.echo(f" cd {plan.demo_dir}") + click.echo(" pip install -r requirements-docgen.txt # external tool — not vendored into src") + click.echo(" docgen --version") click.echo(" docgen wizard # launch GUI to draft narrations") click.echo(" docgen tts --dry-run # preview TTS text stripping") click.echo(" docgen validate # check recordings") click.echo(" docgen generate-all # full pipeline (see docgen generate-all --help)") click.echo() click.echo(" Run docgen yaml-generate next, then edit docgen.yaml for segments, visuals, and TTS as needed.") + click.echo(" Keep only this bundle in your repo — install docgen via pip/pipx, do not copy the library source.") click.echo() diff --git a/src/docgen/install_spec.py b/src/docgen/install_spec.py new file mode 100644 index 0000000..d29666a --- /dev/null +++ b/src/docgen/install_spec.py @@ -0,0 +1,302 @@ +"""Canonical install strings for using docgen as an external tool (not vendored). + +Consumers should ``pip install`` (or ``pipx`` / ``uv tool``) this package and keep +only a demo **bundle** (``docgen.yaml``, hints, narration, …) in their repo — +never a copy of the ``documentation-generator`` source tree. + +The wizard can call :func:`update_docgen_install` to upgrade the running +environment and optionally rewrite ``requirements-docgen.txt``. +""" + +from __future__ import annotations + +import re +import subprocess +import sys +from dataclasses import asdict, dataclass +from importlib import metadata +from pathlib import Path +from typing import Any + +# Default Git install URL (main tip). Prefer pinning ``@`` in CI. +DOCGEN_GIT_URL = "git+https://github.com/jmjava/documentation-generator.git" +DOCGEN_PIP_SPEC = f"docgen @ {DOCGEN_GIT_URL}" +DOCGEN_PIP_MANIM_SPEC = f"docgen[manim] @ {DOCGEN_GIT_URL}" + +# Safe git refs only (SHA, tag, branch). Blocks shell metacharacters. +_REF_RE = re.compile(r"^[A-Za-z0-9._/-]{1,128}$") +_PIN_LINE_RE = re.compile( + r"^\s*docgen(?:\[[^\]]+\])?\s*@\s*" + r"git\+https://github\.com/jmjava/documentation-generator\.git" + r"(?:@(?P[A-Za-z0-9._/-]+))?\s*$", + re.MULTILINE, +) + + +@dataclass(frozen=True) +class ToolInfo: + version: str + location: str | None + editable: bool + pip_spec: str + requirements_path: str | None + requirements_pin: str | None + python: str + + def to_dict(self) -> dict[str, Any]: + return asdict(self) + + +@dataclass(frozen=True) +class UpdateResult: + ok: bool + ref: str + pip_spec: str + version_before: str + version_after: str + requirements_updated: bool + requirements_path: str | None + log: str + restart_required: bool = True + error: str | None = None + + def to_dict(self) -> dict[str, Any]: + return asdict(self) + + +def package_version() -> str: + """Installed distribution version, or ``unknown`` when not installed as a package.""" + try: + return metadata.version("docgen") + except metadata.PackageNotFoundError: + return "unknown" + + +def validate_git_ref(ref: str) -> str: + """Return a sanitized git ref or raise ``ValueError``.""" + cleaned = (ref or "").strip() or "main" + if cleaned.startswith("@"): + cleaned = cleaned[1:] + if not _REF_RE.match(cleaned): + raise ValueError( + f"invalid git ref {ref!r}: use a branch, tag, or commit SHA " + "(letters, digits, . _ / - only)" + ) + if ".." in cleaned or cleaned.startswith("-"): + raise ValueError(f"invalid git ref {ref!r}") + return cleaned + + +def pip_spec_for_ref(ref: str | None = None, *, with_manim: bool = False) -> str: + """Build a ``docgen @ git+…`` (or ``docgen[manim] @ …``) requirement string.""" + if ref is None or not str(ref).strip(): + return DOCGEN_PIP_MANIM_SPEC if with_manim else DOCGEN_PIP_SPEC + safe = validate_git_ref(ref) + base = f"{DOCGEN_GIT_URL}@{safe}" + return f"docgen[manim] @ {base}" if with_manim else f"docgen @ {base}" + + +def requirements_docgen_txt(*, pin_ref: str | None = None) -> str: + """Contents of a consumer ``requirements-docgen.txt``. + + ``pin_ref`` is an optional git ref (commit SHA, tag, or branch). When omitted, + the file installs from the default remote tip and comments show how to pin. + """ + ref = (pin_ref or "").strip() + if ref: + safe = validate_git_ref(ref) + pip_line = pip_spec_for_ref(safe, with_manim=False) + manim_line = pip_spec_for_ref(safe, with_manim=True) + pin_note = f"# Pinned ref: {safe}\n" + else: + pip_line = DOCGEN_PIP_SPEC + manim_line = DOCGEN_PIP_MANIM_SPEC + pin_note = ( + "# Tip: pin a commit SHA for reproducible CI, e.g.\n" + f"# docgen @ {DOCGEN_GIT_URL}@\n" + ) + + return ( + "# docgen — external CLI/library (do NOT vendor documentation-generator into this repo).\n" + "#\n" + "# Install into a project venv:\n" + "# python3 -m venv .venv && source .venv/bin/activate\n" + "# pip install -r requirements-docgen.txt\n" + "#\n" + "# Or isolate with pipx / uv (global `docgen` on PATH, no project src copy):\n" + f"# pipx install '{pip_line}'\n" + f"# uv tool install '{pip_line}'\n" + "#\n" + "# Manim visuals (optional extra):\n" + f"# pip install '{manim_line}'\n" + "#\n" + "# Wizard: Production → Tool → Update docgen (rewrites this file when pinned).\n" + "#\n" + f"{pin_note}" + f"{pip_line}\n" + ) + + +def find_requirements_docgen(bundle_dir: Path | None) -> Path | None: + if bundle_dir is None: + return None + path = Path(bundle_dir) / "requirements-docgen.txt" + return path if path.is_file() else None + + +def read_requirements_pin(path: Path | None) -> str | None: + """Return the git ref pinned in ``requirements-docgen.txt``, if any.""" + if path is None or not path.is_file(): + return None + try: + text = path.read_text(encoding="utf-8") + except OSError: + return None + m = _PIN_LINE_RE.search(text) + if not m: + return None + return m.group("ref") + + +def write_requirements_docgen(bundle_dir: Path, *, pin_ref: str | None = None) -> Path: + """Write/overwrite ``requirements-docgen.txt`` under the bundle.""" + path = Path(bundle_dir) / "requirements-docgen.txt" + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(requirements_docgen_txt(pin_ref=pin_ref), encoding="utf-8") + return path + + +def _distribution_location() -> tuple[str | None, bool]: + """Return (install location string, editable).""" + try: + dist = metadata.distribution("docgen") + except metadata.PackageNotFoundError: + # Running from a source checkout on PYTHONPATH + here = Path(__file__).resolve().parent + return str(here), True + # Prefer direct_url for editable / VCS installs + try: + direct = dist.read_text("direct_url.json") + except Exception: + direct = None + if direct and '"editable"' in direct: + return str(dist.locate_file("")), True + try: + loc = str(dist.locate_file("")) + except Exception: + loc = None + return loc, False + + +def tool_info(bundle_dir: Path | None = None) -> ToolInfo: + """Snapshot of the installed docgen tool + optional bundle pin.""" + req = find_requirements_docgen(bundle_dir) + loc, editable = _distribution_location() + return ToolInfo( + version=package_version(), + location=loc, + editable=editable, + pip_spec=DOCGEN_PIP_SPEC, + requirements_path=str(req) if req else None, + requirements_pin=read_requirements_pin(req), + python=sys.executable, + ) + + +def update_docgen_install( + *, + ref: str = "main", + with_manim: bool = False, + bundle_dir: Path | None = None, + update_requirements: bool = True, + timeout_sec: int = 300, +) -> UpdateResult: + """Upgrade docgen in the current interpreter via ``python -m pip install``. + + Only the canonical ``jmjava/documentation-generator`` git URL is allowed. + After a successful upgrade the running wizard process still serves the old + code until restarted (``restart_required=True``). + """ + safe_ref = validate_git_ref(ref) + spec = pip_spec_for_ref(safe_ref, with_manim=with_manim) + before = package_version() + cmd = [ + sys.executable, + "-m", + "pip", + "install", + "--upgrade", + spec, + ] + try: + proc = subprocess.run( + cmd, + capture_output=True, + text=True, + timeout=timeout_sec, + check=False, + ) + except subprocess.TimeoutExpired: + return UpdateResult( + ok=False, + ref=safe_ref, + pip_spec=spec, + version_before=before, + version_after=before, + requirements_updated=False, + requirements_path=None, + log="pip install timed out", + error="timeout", + ) + except OSError as exc: + return UpdateResult( + ok=False, + ref=safe_ref, + pip_spec=spec, + version_before=before, + version_after=before, + requirements_updated=False, + requirements_path=None, + log=str(exc), + error=str(exc), + ) + + log = ((proc.stdout or "") + (proc.stderr or "")).strip() + if proc.returncode != 0: + return UpdateResult( + ok=False, + ref=safe_ref, + pip_spec=spec, + version_before=before, + version_after=before, + requirements_updated=False, + requirements_path=None, + log=log or f"pip exited {proc.returncode}", + error=f"pip exited {proc.returncode}", + ) + + # Invalidate importlib metadata caches so version_after is fresh. + try: + metadata.packages_distributions.cache_clear() # type: ignore[attr-defined] + except Exception: + pass + after = package_version() + + req_path: str | None = None + req_updated = False + if update_requirements and bundle_dir is not None: + written = write_requirements_docgen(Path(bundle_dir), pin_ref=safe_ref) + req_path = str(written) + req_updated = True + + return UpdateResult( + ok=True, + ref=safe_ref, + pip_spec=spec, + version_before=before, + version_after=after, + requirements_updated=req_updated, + requirements_path=req_path, + log=log, + restart_required=True, + ) diff --git a/src/docgen/static/wizard.css b/src/docgen/static/wizard.css index 98c8da4..8194331 100644 --- a/src/docgen/static/wizard.css +++ b/src/docgen/static/wizard.css @@ -6,6 +6,17 @@ body{font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,san .brand{font-weight:700;font-size:1.1rem;margin-right:auto} .nav-btn{background:transparent;border:1px solid rgba(255,255,255,.3);color:#fff;padding:.4rem 1rem;border-radius:6px;cursor:pointer;font-size:.85rem;transition:all .2s} .nav-btn:hover,.nav-btn.active{background:rgba(255,255,255,.15);border-color:rgba(255,255,255,.6)} +.tool-version-badge{font-size:.75rem;opacity:.85;margin-left:.25rem;padding:.2rem .55rem;border:1px solid rgba(255,255,255,.25);border-radius:999px} +.tool-layout{display:grid;grid-template-columns:1fr 1.1fr;gap:1.5rem;align-items:start;max-width:1100px} +@media(max-width:900px){.tool-layout{grid-template-columns:1fr}} +.tool-info{display:grid;gap:.65rem;margin:1rem 0 0} +.tool-info>div{display:grid;grid-template-columns:7.5rem 1fr;gap:.5rem;align-items:start;font-size:.88rem} +.tool-info dt{color:#666;font-weight:600} +.tool-info dd{margin:0;word-break:break-all} +.tool-info .mono,.mono{font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:.8rem} +.tool-update-form{display:flex;flex-direction:column;gap:.65rem;margin-top:.75rem} +.tool-update-form label{font-size:.85rem;color:#444} +.tool-log{margin-top:1rem;max-height:280px;overflow:auto;background:#f8f9fc;border:1px solid #e8ecf4;border-radius:8px;padding:.75rem;font-size:.75rem;white-space:pre-wrap} /* Views */ .view{padding:1.5rem}.hidden{display:none!important} diff --git a/src/docgen/static/wizard.js b/src/docgen/static/wizard.js index 8d48ae7..7a434ab 100644 --- a/src/docgen/static/wizard.js +++ b/src/docgen/static/wizard.js @@ -24,9 +24,112 @@ document.querySelectorAll(".view").forEach((v) => v.classList.add("hidden")); document.getElementById("view-" + btn.dataset.view).classList.remove("hidden"); if (btn.dataset.view === "production") loadProductionView(); + if (btn.dataset.view === "tool") loadToolView(); }); }); + // ---- Tool (external install / upgrade) ---- + async function loadToolView() { + const badge = document.getElementById("tool-version-badge"); + const status = document.getElementById("tool-update-status"); + try { + const res = await fetch("/api/tool"); + const data = await res.json(); + if (badge) badge.textContent = "v" + (data.version || "?"); + const set = (id, val) => { + const el = document.getElementById(id); + if (el) el.textContent = val == null || val === "" ? "—" : String(val); + }; + set("tool-info-version", data.version); + set("tool-info-python", data.python); + set("tool-info-location", data.location); + set("tool-info-editable", data.editable ? "yes (source / editable install)" : "no (package install)"); + set( + "tool-info-pin", + data.requirements_pin + ? data.requirements_pin + (data.requirements_path ? " (" + data.requirements_path + ")" : "") + : data.requirements_path + ? "unpinned (" + data.requirements_path + ")" + : "no requirements-docgen.txt in bundle" + ); + if (data.requirements_pin && document.getElementById("tool-ref-input")) { + const inp = document.getElementById("tool-ref-input"); + if (!inp.dataset.touched) inp.value = data.requirements_pin; + } + } catch (err) { + if (status) status.textContent = "Could not load tool info"; + if (badge) badge.textContent = "v?"; + } + } + + document.getElementById("tool-ref-input")?.addEventListener("input", (e) => { + e.target.dataset.touched = "1"; + }); + + document.getElementById("btn-tool-refresh")?.addEventListener("click", () => loadToolView()); + + document.getElementById("btn-tool-update")?.addEventListener("click", async () => { + const btn = document.getElementById("btn-tool-update"); + const status = document.getElementById("tool-update-status"); + const logEl = document.getElementById("tool-update-log"); + const ref = (document.getElementById("tool-ref-input")?.value || "main").trim(); + const withManim = !!document.getElementById("tool-with-manim")?.checked; + const updateReq = !!document.getElementById("tool-update-requirements")?.checked; + if (!ref) { + alert("Enter a git ref (main, tag, or commit SHA)."); + return; + } + if (!confirm( + "Update docgen to ref “" + ref + "” in this Python environment?\n\n" + + "You must restart the wizard afterward for the new code to load." + )) { + return; + } + btn.disabled = true; + if (status) status.textContent = "Updating… (pip install)"; + if (logEl) { + logEl.classList.remove("hidden"); + logEl.textContent = "Running pip install…\n"; + } + try { + const res = await fetch("/api/tool/update", { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ + ref, + with_manim: withManim, + update_requirements: updateReq, + }), + }); + const data = await res.json(); + if (logEl) logEl.textContent = data.log || JSON.stringify(data, null, 2); + if (!res.ok || data.ok === false) { + if (status) status.textContent = "Update failed"; + alert("Update failed: " + (data.error || res.status)); + } else { + if (status) { + status.textContent = + "Updated " + (data.version_before || "?") + " → " + (data.version_after || "?") + + " — restart the wizard"; + } + await loadToolView(); + alert( + "docgen updated to ref “" + data.ref + "”.\n" + + "Version: " + (data.version_before || "?") + " → " + (data.version_after || "?") + "\n\n" + + (data.requirements_updated ? "requirements-docgen.txt rewritten.\n\n" : "") + + "Restart the wizard (Ctrl+C, then `docgen wizard`) to load the new code." + ); + } + } catch (err) { + if (status) status.textContent = "Error"; + alert("Update error: " + err.message); + } + btn.disabled = false; + }); + + // Prefetch version badge on load + loadToolView(); + // ---- Tab switching ---- document.addEventListener("click", (e) => { if (!e.target.classList.contains("tab-btn")) return; diff --git a/src/docgen/templates/wizard.html b/src/docgen/templates/wizard.html index 887dec0..14474d4 100644 --- a/src/docgen/templates/wizard.html +++ b/src/docgen/templates/wizard.html @@ -12,6 +12,8 @@ docgen wizard + + … @@ -57,6 +59,51 @@

Generated drafts

+ + + + + diff --git a/src/docgen/wizard.py b/src/docgen/wizard.py index 7aeea1c..12279e2 100644 --- a/src/docgen/wizard.py +++ b/src/docgen/wizard.py @@ -329,6 +329,47 @@ def _cfg(): def index(): return render_template("wizard.html") + # -- API: tool version / upgrade (external install) ------------------------ + + @app.route("/api/tool") + def api_tool_info(): + """Return installed docgen version and bundle requirements pin.""" + from docgen.install_spec import tool_info + + cfg = _cfg() + bundle = cfg.base_dir if cfg else None + info = tool_info(bundle) + return jsonify(info.to_dict()) + + @app.route("/api/tool/update", methods=["POST"]) + def api_tool_update(): + """Upgrade docgen in this interpreter via pip; optionally rewrite requirements pin. + + Body: ``{"ref": "main", "with_manim": false, "update_requirements": true}`` + + Only the canonical GitHub repo URL is allowed. Restart the wizard after + a successful update so the new code is loaded. + """ + from docgen.install_spec import update_docgen_install + + cfg = _cfg() + data = request.json or {} + ref = str(data.get("ref") or "main") + with_manim = bool(data.get("with_manim", False)) + update_req = bool(data.get("update_requirements", True)) + bundle = cfg.base_dir if cfg else None + try: + result = update_docgen_install( + ref=ref, + with_manim=with_manim, + bundle_dir=bundle, + update_requirements=update_req, + ) + except ValueError as exc: + return jsonify({"ok": False, "error": str(exc)}), 400 + status = 200 if result.ok else 500 + return jsonify(result.to_dict()), status + # -- API: scan files ------------------------------------------------------- @app.route("/api/scan") diff --git a/tests/test_init.py b/tests/test_init.py index f13fd78..9d61bed 100644 --- a/tests/test_init.py +++ b/tests/test_init.py @@ -84,6 +84,8 @@ def test_generate_files_minimal(tmp_path: Path) -> None: created = generate_files(plan) assert (tmp_path / "demos" / "docgen.yaml").exists() + assert (tmp_path / "demos" / "requirements-docgen.txt").exists() + assert (tmp_path / "demos" / "README.md").exists() assert (tmp_path / "demos" / "generate-all.sh").exists() assert (tmp_path / "demos" / "compose.sh").exists() assert (tmp_path / "demos" / "rebuild-after-audio.sh").exists() @@ -92,6 +94,13 @@ def test_generate_files_minimal(tmp_path: Path) -> None: assert (tmp_path / "demos" / "narration" / "01-intro.md").exists() assert (tmp_path / "demos" / "narration" / "02-setup.md").exists() + req = (tmp_path / "demos" / "requirements-docgen.txt").read_text(encoding="utf-8") + assert "docgen @ git+https://github.com/jmjava/documentation-generator.git" in req + assert "do NOT vendor" in req + bundle_readme = (tmp_path / "demos" / "README.md").read_text(encoding="utf-8") + assert "external" in bundle_readme.lower() + assert "pip install -r requirements-docgen.txt" in bundle_readme + cfg_text = (tmp_path / "demos" / "docgen.yaml").read_text() cfg = yaml.safe_load(cfg_text.split("\n\n", 1)[-1]) assert cfg["segments"]["all"] == ["01", "02"] diff --git a/tests/test_install_spec.py b/tests/test_install_spec.py new file mode 100644 index 0000000..2aba6b3 --- /dev/null +++ b/tests/test_install_spec.py @@ -0,0 +1,144 @@ +"""Tests for external-install helpers (no vendored library in consumer src).""" + +from __future__ import annotations + +from pathlib import Path +from unittest.mock import patch + +import pytest +import yaml +from click.testing import CliRunner + +from docgen.cli import main +from docgen.config import Config +from docgen.install_spec import ( + DOCGEN_PIP_SPEC, + package_version, + pip_spec_for_ref, + read_requirements_pin, + requirements_docgen_txt, + tool_info, + update_docgen_install, + validate_git_ref, + write_requirements_docgen, +) +from docgen.wizard import create_app + + +def test_requirements_docgen_txt_default() -> None: + text = requirements_docgen_txt() + assert DOCGEN_PIP_SPEC in text + assert "pipx install" in text + assert "uv tool install" in text + assert "do NOT vendor" in text + assert "Wizard:" in text + + +def test_requirements_docgen_txt_pin() -> None: + text = requirements_docgen_txt(pin_ref="abc1234") + assert "@abc1234" in text + assert "Pinned ref: abc1234" in text + + +def test_validate_git_ref_rejects_shell_meta() -> None: + with pytest.raises(ValueError): + validate_git_ref("main; rm -rf /") + with pytest.raises(ValueError): + validate_git_ref("../etc/passwd") + assert validate_git_ref("main") == "main" + assert validate_git_ref("@deadbeef") == "deadbeef" + + +def test_pip_spec_for_ref_manim() -> None: + assert "docgen[manim]" in pip_spec_for_ref("main", with_manim=True) + assert "@main" in pip_spec_for_ref("main") + + +def test_write_and_read_requirements_pin(tmp_path: Path) -> None: + path = write_requirements_docgen(tmp_path, pin_ref="cafe0123") + assert path.is_file() + assert read_requirements_pin(path) == "cafe0123" + + +def test_update_docgen_install_mocked(tmp_path: Path) -> None: + class _Proc: + returncode = 0 + stdout = "Successfully installed docgen\n" + stderr = "" + + with patch("docgen.install_spec.subprocess.run", return_value=_Proc()) as run: + result = update_docgen_install( + ref="main", + bundle_dir=tmp_path, + update_requirements=True, + ) + assert result.ok + assert result.ref == "main" + assert result.requirements_updated + assert (tmp_path / "requirements-docgen.txt").is_file() + assert "documentation-generator.git@main" in run.call_args.args[0][-1] + assert result.restart_required is True + + +def test_update_docgen_install_rejects_bad_ref(tmp_path: Path) -> None: + with pytest.raises(ValueError): + update_docgen_install(ref="main && true", bundle_dir=tmp_path) + + +def test_cli_version() -> None: + runner = CliRunner() + r = runner.invoke(main, ["--version"]) + assert r.exit_code == 0, r.output + assert "docgen" in r.output + assert DOCGEN_PIP_SPEC in r.output or "documentation-generator.git" in r.output + assert package_version() # non-empty + + +def test_wizard_tool_api(tmp_path: Path) -> None: + write_requirements_docgen(tmp_path, pin_ref="abcd1234") + (tmp_path / "docgen.yaml").write_text( + yaml.dump( + { + "dirs": {"narration": "narration"}, + "segments": {"all": ["01"]}, + "segment_names": {"01": "01-x"}, + } + ), + encoding="utf-8", + ) + (tmp_path / "narration").mkdir() + cfg = Config.from_yaml(tmp_path / "docgen.yaml") + app = create_app(cfg) + client = app.test_client() + + info = client.get("/api/tool") + assert info.status_code == 200 + body = info.get_json() + assert body["requirements_pin"] == "abcd1234" + assert "version" in body + + class _Proc: + returncode = 0 + stdout = "ok\n" + stderr = "" + + with patch("docgen.install_spec.subprocess.run", return_value=_Proc()): + upd = client.post( + "/api/tool/update", + json={"ref": "main", "update_requirements": True, "with_manim": False}, + ) + assert upd.status_code == 200, upd.get_json() + data = upd.get_json() + assert data["ok"] is True + assert data["restart_required"] is True + assert read_requirements_pin(tmp_path / "requirements-docgen.txt") == "main" + + bad = client.post("/api/tool/update", json={"ref": "main;id"}) + assert bad.status_code == 400 + + +def test_tool_info_without_requirements(tmp_path: Path) -> None: + info = tool_info(tmp_path) + assert info.requirements_path is None + assert info.requirements_pin is None + assert info.python