Skip to content

test(docs): nothing compiles JSDoc @example blocks, so a published JSR reference can import a module that does not exist #1533

Description

@rickylabs

Summary

packages/** JSDoc @example blocks render on JSR — they are the reference documentation a
consumer or agent reads for every published package. Nothing compiles them, and nothing checks
their import specifiers against the shipped module layout.
#1425 is the instance; this is the
class.

Evidence

Verified while closing #1425 at 01aa12b67:

  1. deno doc --lint does not compile examples. deno task doc:lint --root packages/sdk --pretty
    exits 0 on a @example that imports a module which does not exist. It reports missing JSDoc,
    private type refs, and similar surface diagnostics — never the contents of an example block.
  2. The scoped type-check wrappers check source, not comments.
    .llm/tools/run-deno-check.ts --root packages/sdk selects 78 files, 0 diagnostics, while the
    example inside one of them referenced a module removed months earlier.
  3. That is how fix(sdk): JSDoc example still imports the removed ./api-clients.ts module, rendering on JSR #1425 survived. create-service-query-utils.ts:39 imported './api-clients.ts'
    for the entire life of the golden-path rewrite. fix(docs/sdk): the golden path names three different client modules and two incompatible query APIs, and the file the quickstart points at is a CSS entry #1373/PR fix(docs/sdk): teach one client module and query dialect #1424 swept every docs/site page (10
    occurrences → 0) and every gate stayed green, because no gate looks inside JSDoc.
  4. A second instance existed at the same time, undetected. packages/sdk/src/desktop/mod.ts:12
    imported './contracts/orders.ts' — a relative app-level path that does not resolve in a
    scaffolded project, on the published @netscript/sdk/desktop subpath. It was found by human/agent
    review of PR fix(sdk): update stale API-client JSDoc example #1526, not by any check, and fixed there as a recorded scope extension. Two defects
    of the same class in one package, both invisible to CI.
  5. test(docs): docs:accuracy is a fixed-string needle checker — no docs/site code block is ever compiled, and a CLI change never revalidates the site #1374 does not close this. It builds a compile gate for fenced ts/tsx blocks under
    docs/site/**. Its own boundaries state it covers prose code blocks; JSDoc is explicitly not in
    scope. After test(docs): docs:accuracy is a fixed-string needle checker — no docs/site code block is ever compiled, and a CLI change never revalidates the site #1374 lands, docs/site snippets are falsifiable and JSR reference examples still
    are not.
  6. The acceptance language cannot currently be satisfied. fix(sdk): JSDoc example still imports the removed ./api-clients.ts module, rendering on JSR #1425's acceptance row reads "the
    example compiles against the shipped module layout". PR fix(sdk): update stale API-client JSDoc example #1526 ticks it on construction and
    inspection with the mechanism disclosed, because no gate can establish the word. Note also that
    under the repo's own JSDoc convention an @example never imports the symbol it documents, so a
    naive "extract and deno check" would fail every example in the workspace — the gate must inject
    the documented symbol into scope.

Why it matters

JSR reference pages are the surface where a wrong import costs the most: the reader is already
outside the guided path, and an agent reading the reference has no prose context to correct it. We
have now fixed this class twice by hand in one package. The third instance will also be found by
hand, or not at all.

Scope

  • A checked-in gate that extracts @example blocks from published packages/**/plugins/** JSDoc
    and type-checks them against the published entrypoints, with the documented symbol injected into
    scope so the repo's existing convention stays valid.
  • Explicit opt-out with a stated reason for examples that legitimately cannot compile (partial
    fragments, illustrative pseudo-code), and a census of exempt blocks reported on every run — the
    same shape test(docs): docs:accuracy is a fixed-string needle checker — no docs/site code block is ever compiled, and a CLI change never revalidates the site #1374 establishes for docs/site, reused rather than reinvented.
  • An import-specifier check: an example on a published surface must not import a relative app-level
    path, and app aliases used (@app/, @<project>/contracts) must be ones the scaffold actually
    generates (generate-app-deno-json.ts:126-131).

Boundaries

Acceptance criteria

  • A checked-in tool extracts JSDoc @example blocks from the publish set and type-checks them.
  • The documented symbol is in scope, so existing convention-conforming examples pass unchanged.
  • Examples resolve @netscript/* and scaffold-generated app aliases; a relative app-level import
    on a published surface fails.
  • Non-compiling examples require an explicit marker with a stated reason; the exempt census is
    reported on every run.
  • The gate runs in CI on packages/** and plugins/** changes.
  • Negative test: reintroducing import { ordersClient } from './api-clients.ts' into
    create-service-query-utils.ts makes the gate exit non-zero.
  • Negative test: reintroducing a relative app-level contract import into
    packages/sdk/src/desktop/mod.ts makes the gate exit non-zero.

Both negative tests are drawn from real defects that shipped, so the predicate is known to be
satisfiable — the guard cannot be one whose condition never fires.

Provenance

Filed by the 0.0.6 documentation-lane orchestrator while running the pre-merge gate for PR #1526
(#1425). The trigger was that #1425's "compiles" acceptance row could not be truthfully ticked on
gate evidence, and the investigation of why produced the second undetected instance.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions