diff --git a/AGENTS.md b/AGENTS.md index 60c5ddd..52e949e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -56,7 +56,7 @@ Commands registered on the **`docgen`** CLI include: - **`clean-bundle`** — remove regenerable outputs per policy. - **`concat`** — stitch segment videos. - **`pages`** — emit static HTML for demo assets. -- **`generate-all`** — orchestrated pipeline: TTS → timestamps → **scene retime** (compile existing `*.scene.yaml` against fresh timing) → images → Manim → compose → validate → concat → pages. Optional `--regen-scene-specs` for LLM scene YAML first. +- **`generate-all`** — orchestrated pipeline: TTS → timestamps → **scene specs** (auto `scene-spec-generate` when `animations/specs/` is empty; otherwise offline retime) → images → Manim → compose → validate → concat → pages. `--regen-scene-specs` forces LLM rewrite; `--skip-scene-retime` keeps legacy hand `scenes.py` only. - **`rebuild-after-audio`** — same as generate-all with TTS skipped (still retimes scenes after timestamps). ## Implications for changes here diff --git a/README.md b/README.md index 83c63e3..c2bde75 100644 --- a/README.md +++ b/README.md @@ -38,13 +38,15 @@ If you still need the legacy behaviour, pin a pre-removal commit `whisper-1` remains available via `timestamps.engine: whisper` / `docgen timestamps --engine whisper`. Both engines write the same `timing.json` shape. -- **Manim animations** — primary visual surface. Use **`docgen scene-spec-generate`** - + **`scene-compile`** (or hand-maintained **`animations/specs/*.scene.yaml`**) - for deterministic diagram layout: rows are auto-paginated when they exceed the - frame stack budget, specs that overflow safe width / budget are rejected, and - (when `timing.json` carries Whisper words) each row’s first label is mapped to - a **`wait_word`** index. Hand-maintained custom Manim classes still live in - `animations/scenes.py` outside the `BEGIN/END GENERATED SCENE` markers. +- **Manim animations (default: declarative scene specs)** — primary visual surface. + Prefer **`animations/specs/*.scene.yaml`** via **`docgen scene-spec-generate`** + + **`scene-compile`**. On **`generate-all`**, if no specs exist yet, the pipeline + **auto-runs** scene-spec-generate; later runs **retime-compile** specs against fresh + `timing.json` (no OpenAI) unless you pass **`--regen-scene-specs`**. Specs auto-paginate + when they exceed the frame stack budget, reject unsafe width / subject-beat gaps, and + map each paced label to a **`wait_word`** index. Hand-maintained custom Manim classes + may still live in `animations/scenes.py` outside `BEGIN/END GENERATED SCENE` markers + (use **`--skip-scene-retime`** to bypass the declarative stage). - **OpenAI image assets in Manim scenes** — a scene-spec box may be an **image element** (`image: images/.png` + `prompt:`); `docgen image-generate` renders the prompt via the OpenAI Images API (default `gpt-image-1`) and the @@ -131,7 +133,7 @@ docgen validate --pre-push | `docgen lint [--segment 01]` | Narration lint only | | `docgen concat [--config full-demo]` | Concatenate full demo files | | `docgen pages [--force]` | Generate `index.html`, `pages.yml`, `.gitattributes`, `.gitignore` | -| `docgen generate-all [--skip-tts] [--skip-manim] [--retry-manim] [--regen-scene-specs]` | Full pipeline: TTS → timestamps → **scene retime** (existing specs) → Manim → compose → validate. `--regen-scene-specs` also runs OpenAI scene-spec-generate | +| `docgen generate-all [--skip-tts] [--skip-manim] [--retry-manim] [--regen-scene-specs] [--skip-scene-retime]` | Full pipeline: TTS → timestamps → **scene specs** → Manim → compose → validate. Missing `animations/specs/*.scene.yaml` auto-generates; existing specs retime offline; `--regen-scene-specs` forces LLM rewrite | | `docgen rebuild-after-audio [--regen-scene-specs]` | Timestamps → scene retime → Manim → compose → validate (skips TTS) | | `docgen clean-bundle [-y] [--delete-config] [--keep-narration]` | Remove regenerable outputs under the bundle | | `docgen narration-generate --segment 01 [--extra-path REL] [--hint TEXT] [--dry-run] [--force] [--revise --revision-notes TEXT]` | Generate narration `.md` from repo sources + owner hints (OpenAI); `--revise` edits the existing script in place | diff --git a/docs/demos/README.md b/docs/demos/README.md index bf49680..8467d05 100644 --- a/docs/demos/README.md +++ b/docs/demos/README.md @@ -40,15 +40,20 @@ Full **`docgen generate-all`** for this bundle needs: Historic **Playwright** / **demo-function** / **per-function** capture paths were **removed** from the library (see root **`AGENTS.md`**); do not expect browser-test recording commands in current `docgen`. -## Declarative Manim (`animations/specs/*.scene.yaml`) +## Declarative Manim (`animations/specs/*.scene.yaml`) — **default** -For **`manim`** segments built from labeled `_box` diagrams: +This is the **default** visual path for all consumers (not optional decoration): -1. **`docgen scene-spec-generate --segment [--compile] [--hint "…"]`** — optional OpenAI emits **YAML**; **`--compile`** injects into **`animations/scenes.py`**. Use **`--all --compile`** for every declarative segment. -2. **`docgen scene-compile path/to/spec.scene.yaml`** — compile a spec without another API call. -3. Then **`docgen timestamps`**, **`docgen manim`**, **`docgen compose`**, and **`docgen concat full-demo`** when refreshing recordings. +1. First **`docgen generate-all`** (or **`scene-spec-generate --all --compile`**) writes + **`animations/specs/.scene.yaml`** and injects generated classes into + **`animations/scenes.py`** between marker comments. +2. Later **`generate-all`** / **`rebuild-after-audio`** **retime-compiles** those specs + against fresh **`timing.json`** (no OpenAI) so beat sync stays correct. +3. Force a layout rewrite with **`--regen-scene-specs`**. Skip the stage only for + legacy hand scenes: **`--skip-scene-retime`**. -Run **`docgen timestamps`** after changing narration audio so **`timing.json`** gains **`words`** / segments required by **`wait_segment`** / **`wait_word`** in specs. +Label boxes with **spoken phrases from the narration** (must match `timing.json` +words), keep ~3 rows/page, and follow **`hints/manim-scene-specs.md`**. ## Full reset (total nuke + regen) diff --git a/docs/demos/hints/manim-scene-specs.md b/docs/demos/hints/manim-scene-specs.md index 8c78189..5ad56b7 100644 --- a/docs/demos/hints/manim-scene-specs.md +++ b/docs/demos/hints/manim-scene-specs.md @@ -7,6 +7,7 @@ Use these constraints when generating **`animations/specs/*.scene.yaml`** (via ` - **Frame budget:** dogfood scenes use a **14.22×8** Manim frame (`scenes.py` header). Content sits under the title — tall stacks (**many rows × box `height` + `row_gap`**) scroll past the bottom. Prefer **extra pages** or **shorter boxes** (`height` ~0.72–0.9, tighter `row_gap`) over piling 5+ full-height rows on one page. - **~3 rows per page** is a safe default (~6 when rows use compact height); match beats in **`narration/.md`** and optional **`wait_segment`** / **`wait_at`** when `timing.json` has Whisper data. - **Subject-beat coverage (library contract):** hold the board while consecutive sentences elaborate the **same** topic; when the topic shifts, add a spoken-phrase label for that beat. `scene-spec-generate` and `validate` run `layout_density_violations` — **coverage of subject beats + no invented unspoken labels**, not a blind label count. Disable with `validation.subject_beat_coverage.enabled: false` if needed. +- **Hyphenated / script labels:** Whisper often emits `version-controlled` or `setup-agent-prompts.sh` as **one** word. Prefer those exact compounds in box labels (or set `pace: none`). Retiming glues split label tokens to match those spoken compounds. - **Palette tokens only:** `C_BG`, `C_ACCENT`, `C_GREEN`, `C_ORANGE`, `C_BLUE`, `C_RED`, `C_TEAL`, `C_PURPLE`, `C_WHITE`. - **Readable sizes:** `font_size` ≥ 14; widths ~3–6, heights ~0.7–1.3; tune `row_gap` / `column_gap` in `layout` if labels feel cramped. - **Layout gate:** `docgen scene-spec-generate` rejects specs that exceed a computed vertical stack budget or safe row width (`layout_budget_violations` in `scene_spec.py`). `docgen scene-compile` does **not** enforce that (hand fixes allowed). diff --git a/src/docgen/cli.py b/src/docgen/cli.py index 05df2c9..2d11e8d 100644 --- a/src/docgen/cli.py +++ b/src/docgen/cli.py @@ -1103,11 +1103,14 @@ def generate_all( regen_scene_specs: bool, skip_scene_retime: bool, ) -> None: - """Run full pipeline: TTS → timestamps → scene retime → Manim → compose → validate. + """Run full pipeline: TTS → timestamps → scene specs → Manim → compose → validate. - Order matters for beat sync: timestamps must land before scene compile so - ``wait_word`` indices match the current mp3. Existing declarative specs are - retime-compiled offline; pass ``--regen-scene-specs`` to call OpenAI first. + Default Manim path is declarative ``animations/specs/*.scene.yaml``: + + - Specs exist → offline retime-compile against fresh ``timing.json``. + - No specs yet → auto ``scene-spec-generate`` (OpenAI) then compile. + - ``--regen-scene-specs`` forces LLM regenerate even when specs exist. + - ``--skip-scene-retime`` skips the whole scene-spec stage (legacy hand scenes). """ from docgen.pipeline import Pipeline diff --git a/src/docgen/init.py b/src/docgen/init.py index da6a423..91fb145 100644 --- a/src/docgen/init.py +++ b/src/docgen/init.py @@ -367,7 +367,8 @@ def _write_config(plan: InitPlan) -> str: "# See: https://github.com/jmjava/documentation-generator\n" "#\n" "# Edit this file to match your project structure, then run:\n" - "# docgen generate-all # full pipeline\n" + "# docgen generate-all # TTS→timestamps→scene-spec→Manim→compose\n" + "# # (creates animations/specs/*.scene.yaml if missing)\n" "# docgen wizard # interactive GUI\n" "# docgen validate # check recordings\n\n" ) @@ -392,7 +393,9 @@ def _write_wrapper_scripts(plan: InitPlan) -> list[str]: scripts = { "generate-all.sh": "\n".join([ "#!/usr/bin/env bash", - "# Full pipeline (TTS, segment visuals, compose, validate, concat). Wraps: docgen generate-all", + "# Full pipeline. Default Manim path: animations/specs/*.scene.yaml", + "# (auto scene-spec-generate on first run; retime thereafter).", + "# Wraps: docgen generate-all", "set -euo pipefail", _bash_dir, _venv_activate, diff --git a/src/docgen/manim_scene_support.py b/src/docgen/manim_scene_support.py index fd54227..74da9cc 100644 --- a/src/docgen/manim_scene_support.py +++ b/src/docgen/manim_scene_support.py @@ -162,7 +162,7 @@ def _load_timing_words(segment_key: str) -> list[dict]: def _box(label, color, w=2.2, h=0.75, fs=18, subtitle=""): - """Labeled rounded box — slightly stronger fill/stroke for readable diagram boards. + """Labeled rounded box - slightly stronger fill/stroke for readable diagram boards. Optional ``subtitle`` is a second, smaller line under the primary label (decorative; not used for wait_word beat matching). diff --git a/src/docgen/pipeline.py b/src/docgen/pipeline.py index 68833b8..c71788f 100644 --- a/src/docgen/pipeline.py +++ b/src/docgen/pipeline.py @@ -4,9 +4,16 @@ entries (see :meth:`docgen.config.Config.pipeline_manim_scene_names`). Segments whose visuals are pre-recorded (``recordings/*.mp4``) do not run through Manim capture here. -After timestamps, existing ``animations/specs/*.scene.yaml`` files are **retime-compiled** -against fresh ``timing.json`` (no OpenAI) so ``wait_word`` indices stay aligned. Optional -``regen_scene_specs`` runs LLM ``scene-spec-generate`` for manim segments before that compile. +After timestamps, declarative ``animations/specs/*.scene.yaml`` files are the **default** +Manim authoring path: + +- **Specs present** → offline **retime-compile** against fresh ``timing.json`` (no OpenAI) + so ``wait_word`` indices stay aligned. +- **No specs yet** (and the bundle has manim ``visual_map`` rows) → automatically run LLM + ``scene-spec-generate`` + compile (same as ``--regen-scene-specs`` for a first run). +- **``regen_scene_specs=True``** → force LLM regenerate even when specs already exist. +- Hand-authored classes outside ``BEGIN/END GENERATED SCENE`` markers remain supported, + but new consumer bundles should prefer specs. """ from __future__ import annotations @@ -40,7 +47,19 @@ def run( TimestampExtractor(self.config).extract_all() if not skip_manim and not skip_scene_retime: - self._run_scene_stages(regen_scene_specs=regen_scene_specs) + from docgen.scene_retime import list_scene_spec_paths + + auto_regen = regen_scene_specs + if not auto_regen and self._manim_segment_ids(): + if not list_scene_spec_paths(self.config): + print( + "\n[pipeline] No animations/specs/*.scene.yaml yet — " + "auto-running scene-spec-generate (declarative Manim default). " + "Pass --skip-scene-retime to keep hand-authored scenes.py only, " + "or --regen-scene-specs to force regenerate later." + ) + auto_regen = True + self._run_scene_stages(regen_scene_specs=auto_regen) if not skip_manim: from docgen.image_generate import generate_missing_images_for_bundle diff --git a/src/docgen/scene_spec.py b/src/docgen/scene_spec.py index e1e1e7c..19e82df 100644 --- a/src/docgen/scene_spec.py +++ b/src/docgen/scene_spec.py @@ -378,6 +378,56 @@ def _tokens_match(label_token: str, word_token: str) -> bool: return _stem(label_token) == _stem(word_token) +def _consume_label_tokens_at( + tokens: list[str], + word_norms: list[str], + start: int, +) -> int | None: + """Advance through ``word_norms`` from ``start`` until all ``tokens`` are consumed. + + Whisper often emits hyphenated compounds as **one** token (``version-controlled`` → + ``versioncontrolled``, ``setup-agent-prompts.sh`` → ``setupagentpromptssh``) while + labels split on hyphens into multiple tokens. Accept either: + + - one label token ↔ one spoken word (exact / stem), or + - two or more consecutive label tokens glued ↔ one spoken word. + + Returns the index of the last consumed spoken word, or ``None`` if the label + cannot be matched starting at ``start``. + """ + if not tokens or start < 0 or start >= len(word_norms): + return None + ti = 0 + wi = start + m = len(tokens) + n = len(word_norms) + while ti < m: + if wi >= n: + return None + spoken = word_norms[wi] + if _tokens_match(tokens[ti], spoken): + ti += 1 + wi += 1 + continue + # Glue 2+ label tokens into one Whisper word (hyphenated / dotted compounds). + glued = tokens[ti] + matched_k: int | None = None + for k in range(2, m - ti + 1): + glued += tokens[ti + k - 1] + if glued == spoken: + matched_k = k + break + # Spoken word is longer only when TTS/Whisper glued extra chars we already + # normalized away (rare); require exact equality for compounds. + if len(glued) > len(spoken): + break + if matched_k is None: + return None + ti += matched_k + wi += 1 + return wi - 1 + + def segment_index_for_whisper_time( segments: list[dict[str, Any]], wall_time: float ) -> int: @@ -442,10 +492,12 @@ def sync_row_labels_to_whisper_words( ) -> dict[str, Any]: """Set ``wait_word`` on each **box** from its ``label`` → first spoken match (in order). - Matching is **fail-closed**: exact/stem token equality only (plus hyphen splits). - No fuzzy containment and no leftover LLM ``wait_word`` when the label is absent - from the transcript. Each matched box waits at word ``start``. Row-level - ``wait_word`` / ``wait_segment`` are cleared when ``overwrite=True``. + Matching is **fail-closed**: exact/stem token equality, hyphen/underscore splits, + and glued Whisper compounds (label ``version-controlled`` ↔ spoken + ``versioncontrolled``). No fuzzy containment and no leftover LLM ``wait_word`` + when the label is absent from the transcript. Each matched box waits at word + ``start``. Row-level ``wait_word`` / ``wait_segment`` are cleared when + ``overwrite=True``. """ if not isinstance(words, list) or not words: return spec @@ -473,17 +525,13 @@ def _find_label(label: str, from_idx: int) -> tuple[int, int] | None: tokens = _label_tokens(label) if not tokens: return None + norms = [w[0] for w in word_stream] n = len(word_stream) - m = len(tokens) i = from_idx - while i <= n - m: - ok = True - for k in range(m): - if not _tokens_match(tokens[k], word_stream[i + k][0]): - ok = False - break - if ok: - return (i + m - 1, word_stream[i][2]) + while i < n: + last = _consume_label_tokens_at(tokens, norms, i) + if last is not None: + return (last, word_stream[i][2]) i += 1 return None diff --git a/tests/test_pipeline.py b/tests/test_pipeline.py index d368287..ad1e2ef 100644 --- a/tests/test_pipeline.py +++ b/tests/test_pipeline.py @@ -101,7 +101,9 @@ def compose_segments(self, _segments) -> int: pipeline_manim_scene_names=lambda: ["Scene01"], ) - Pipeline(cfg).run(skip_tts=True, retry_manim_on_freeze=True) + Pipeline(cfg).run( + skip_tts=True, retry_manim_on_freeze=True, skip_scene_retime=True + ) assert FlakyComposer.attempts == 2 assert calls.count("manim") == 2, "Manim should run once initially and once on retry" @@ -186,7 +188,70 @@ def compose_segments(self, _segments) -> int: ) with pytest.raises(ComposeError, match="FREEZE GUARD"): - Pipeline(cfg).run(skip_tts=True, retry_manim_on_freeze=False) + Pipeline(cfg).run( + skip_tts=True, retry_manim_on_freeze=False, skip_scene_retime=True + ) assert calls.count("manim") == 1 assert media_dir.exists(), "Without retry flag, Manim cache should be untouched" + + +def test_pipeline_auto_regens_scene_specs_when_missing(tmp_path, monkeypatch) -> None: + """First run with manim segments and no specs → scene-spec-generate.""" + calls: list[str] = [] + + class OkComposer: + def __init__(self, _config) -> None: + pass + + def compose_segments(self, _segments) -> int: + calls.append("compose") + return 1 + + _patch_pipeline_stages(monkeypatch, OkComposer, calls) + + import docgen.scene_retime as retime_module + import docgen.scene_spec_generate as ssg + + monkeypatch.setattr( + retime_module, "list_scene_spec_paths", lambda cfg, segment_id=None: [] + ) + + def fake_generate(cfg, sid, **_kwargs): + calls.append(f"scene-spec:{sid}") + specs = cfg.animations_dir / "specs" + specs.mkdir(parents=True, exist_ok=True) + path = specs / f"{sid}.scene.yaml" + path.write_text("segment_id: '01'\nclass_name: Scene01\nrows: []\n", encoding="utf-8") + return SimpleNamespace( + seg_name=sid, + yaml_text=path.read_text(encoding="utf-8"), + spec={"segment_id": sid, "class_name": "Scene01", "rows": []}, + ) + + monkeypatch.setattr(ssg, "generate_scene_spec", fake_generate) + monkeypatch.setattr( + ssg, + "linted_class_block_from_spec", + lambda cfg, spec, timing_key=None: ("class Scene01: pass", spec), + ) + monkeypatch.setattr( + ssg, "inject_class_block_into_scenes_py", lambda *a, **k: calls.append("inject") + ) + + animations_dir = tmp_path / "animations" + animations_dir.mkdir(parents=True) + + cfg = SimpleNamespace( + animations_dir=animations_dir, + segments_all=["01"], + visual_map={"01": {"type": "manim", "class": "Scene01"}}, + pipeline_manim_scene_names=lambda: ["Scene01"], + ) + + Pipeline(cfg).run(skip_tts=True) + + assert "scene-spec:01" in calls + assert "inject" in calls + assert calls.index("timestamps") < calls.index("scene-spec:01") + assert calls.index("scene-spec:01") < calls.index("manim") diff --git a/tests/test_scene_spec.py b/tests/test_scene_spec.py index 899fe0d..12b6006 100644 --- a/tests/test_scene_spec.py +++ b/tests/test_scene_spec.py @@ -1002,6 +1002,33 @@ def test_sync_row_labels_hyphenated_label_matches_spoken_parts() -> None: assert out["rows"][0]["boxes"][0]["wait_word"] == 1 +def test_sync_row_labels_hyphenated_label_matches_glued_whisper_word() -> None: + """Whisper often emits hyphenated compounds as one token; labels still split.""" + spec = { + "segment_id": "1", + "class_name": "X", + "title": {"text": "T", "font_size": 36, "color": "C_WHITE"}, + "rows": [ + _row("version-controlled files"), + _row("setup-agent-prompts.sh"), + _row("SDLC-SPDD Orchestrator"), + ], + } + words = [ + {"word": "into", "start": 0.0, "end": 0.2}, + {"word": "version-controlled", "start": 1.0, "end": 1.4}, + {"word": "files", "start": 1.5, "end": 1.8}, + {"word": "Run", "start": 2.0, "end": 2.2}, + {"word": "setup-agent-prompts.sh", "start": 2.3, "end": 3.0}, + {"word": "SDLC-SPDD", "start": 3.5, "end": 4.0}, + {"word": "Orchestrator", "start": 4.1, "end": 4.6}, + ] + out = sync_row_labels_to_whisper_words(spec, words) + assert out["rows"][0]["boxes"][0]["wait_word"] == 1 + assert out["rows"][1]["boxes"][0]["wait_word"] == 4 + assert out["rows"][2]["boxes"][0]["wait_word"] == 5 + + def test_compile_title_and_box_subtitles() -> None: spec = { "segment_id": "01",