Fix teething problems found while triaging the SICPy launch

[martin-henz]

Summary

Four fixes found while triaging the newly-launched interactive Python edition. The first three follow directly from #1316 (SICP_PUBLISHED_CHAPTERS gate); the fourth is an unrelated pre-existing bug surfaced during the same triage.

• README.md: documents that the frontend fetches toc.json/toc-navigation.json at runtime (companion to the frontend fix in source-academy/frontend#4273 — the frontend previously snapshotted these into its own repo, which is how the TOC went stale after the gate merged).
• Literal & in the interactive editions: <AMP/> in the XML source was translated to the JSON output as the HTML entity string "&" instead of the character &. Correct for an HTML sink, but the frontend renders JSON #text nodes as plain React text, which doesn't decode HTML entities. Affects both SICP JS and SICPy (shared parseXmlJson.js) — e.g. "Centrum Wiskunde & Informatica" and the References section's "John Wiley & Sons" / "Harper & Row". The PDF was unaffected (separate LaTeX converter).
• Forward references into not-yet-published chapters: chapter/section/exercise/figure/footnote references from a published chapter into a later, still-gated one (e.g. "as we will see in Chapter 4") showed as "Chapter" with no number in the interactive site and "??" in the PDF, since labels are only ever registered for chapters the build actually visits.
  • json: label/exercise/figure/footnote registration now runs over the entire book unconditionally, so a forward reference resolves to a real number and href — only that resolution is kept; unpublished chapters' actual page content is still never written to json_<lang>/. The href just 404s into the frontend's "this section is in preparation" message until that chapter is published.
  • pdf: chapters are now brought in with \include instead of \input, with every chapter's .tex fragment generated unconditionally and gating moved entirely to \includeonly. scripts/do.sh does a two-pass compile when SICP_PUBLISHED_CHAPTERS is set: a throwaway shadow pass typesets every chapter once to populate each one's .aux with real label numbers, then the real publish pass restricts \includeonly to the published subset, resolving forward \ref/\pageref against the .aux the shadow pass left behind.
• SICPy code snippets opening the evaluator on the wrong chapter (unrelated to Add SICP_PUBLISHED_CHAPTERS gate for staged chapter publishing #1316 — a pre-existing SICPy-only bug): clicking any code snippet always opened the embedded evaluator on "Python §1", regardless of the snippet's actual chapter. The snippet's embedded-evaluator hash was built with the Source (JavaScript) chap=/variant= scheme for every edition; that's correct for SICP JS (part of the js-slang Chapter enum), but Python is a Conductor-based language directory entry (python1..python4) addressed via language=/variant= instead, so chap= was never meaningful to it and the evaluator just stayed on whatever chapter it already had open.

Test plan

• SICP_EDITION=py SICP_PUBLISHED_CHAPTERS=2 yarn json — verified chap:state → "3", chap:meta → "4", plus exercise/figure/footnote refs (e.g. 4.1.2#ex-4.7, 4.1.3#footnote-1) all resolve to real hrefs; zero unpublished content in json_py//toc.json; zero literal &; snippet hashes now read language=python&variant=N&prgrm=...
• yarn json (JS edition) — zero REF not found warnings, zero literal &, snippet hashes unchanged (chap=N&variant=...&prgrm=...)
• SICP_EDITION=py SICP_PUBLISHED_CHAPTERS=2 yarn pdf — compiled sicpy.pdf (229 pages) shows "Chapter 3"/"Chapter 4", zero ?? anywhere
• SICP_EDITION=js yarn pdf (unaffected edition, never gated) — builds cleanly, 640 pages, zero undefined refs, confirming the \include switch doesn't regress it
• Snippet-chapter fix verified against a local frontend build (patching a fetched section's snippet hash in-browser): opening a chapter-2 snippet now shows "Python §2" and the banner reads "You have chosen Python §2, matching SICPy §2" instead of §1

Found but not fixed here (flagged for editorial follow-up, not a pipeline issue): two pre-existing broken labels in the still-draft chapter 3/4 XML — foot:function-decl-vs-lambda (referenced twice from chapter 3, but its <LABEL> doesn't exist anywhere) and foot:value_producing vs. the actually-defined foot:value_producing_2 (naming mismatch). These only surfaced now because the shadow pass fully typesets those chapters; they don't affect the shipped PDF since chapters 3/4 aren't printed yet.

🤖 Generated with Claude Code

https://claude.ai/code/session_01RsEaJ8SQRH4QkXir4JSs4S