Skip to content

docs(site): auto-regen hooks, auto sidebars, e2e route check#181

Open
ottobolyos wants to merge 25 commits into
TrakHound:masterfrom
ottobolyos:chore/docs-route-crawler
Open

docs(site): auto-regen hooks, auto sidebars, e2e route check#181
ottobolyos wants to merge 25 commits into
TrakHound:masterfrom
ottobolyos:chore/docs-route-crawler

Conversation

@ottobolyos
Copy link
Copy Markdown
Contributor

@ottobolyos ottobolyos commented Jun 1, 2026

Summary

Three docs-site changes that together let a contributor clone the repo and run npm install && npm run dev to get a fully rendered, fully browsable site—no manual generation steps, no hand-edited sidebars, no router gaps—plus a Playwright e2e test that fails the build if any route 404s.

Auto-regen on dev and build

predev and prebuild npm hooks now run the docfx + Roslyn reference generators automatically. The contributor's first npm install && npm run dev cycle regenerates docs/api/ and docs/reference/ before the dev server starts, so the local site mirrors what CI builds on every push.

  • docs/package.json—adds regen, predev, prebuild scripts.
  • .github/workflows/docs.yml—drops the now-redundant explicit generate-api-ref.sh and generate-reference.sh steps (the prebuild hook runs them both before npm run build). The --check drift gate moves ahead of the build so it compares against the committed reference pages rather than the freshly regenerated ones the prebuild hook would otherwise overwrite them with.

Auto-generated sidebars

docs/.vitepress/sidebar.ts derives the /api/ and /reference/ sidebars at config-load time from the generated trees on disk, replacing the hand-maintained entries in config.ts.

  • /api/—parses docfx's toc.yml and projects the namespace hierarchy into nested collapsible groups. MTConnect.Devices.DataItems.SampleDataItem lands at MTConnect > Devices > DataItems > SampleDataItem; all ~1800 types are one click from a section landing. Collapsed by default so the viewport stays usable.
  • /reference/—lists every .md file under docs/reference/ except index.md (wired in as the Overview entry). Flat alphabetical list—a future generator output appears automatically once generate-reference.sh emits it.

The hand-curated narrative sidebars (/compliance/, /configure/, /concepts/, /cookbook/, /modules/, /wire-formats/, /troubleshooting/, /examples/, /migration/, /development/, /cli/) remain hand-edited; only the two generated trees swap to auto-derivation.

Playwright e2e test for VitePress routes

tests/MTConnect.NET-Docs-Tests/RouteCheckTests.cs builds the docs site, spawns vitepress preview against the built dist/ tree, walks every route the markdown source tree implies in a headless Chromium browser, and asserts no 404s. The test runs as part of the existing dotnet test matrix on the ubuntu-latest leg (windows-latest already excludes Category=E2E). Catches the failure modes a filesystem-level link checker misses—VitePress router misregistration (the trailing-slash-vs-cleanUrls bug from #157 would have failed this test), missing static assets, JS errors in custom theme components.

  • tests/MTConnect.NET-Docs-Tests/RouteCheckTests.cs—new NUnit fixture. [OneTimeSetUp] runs npm ci && npm run build when dist/ is missing, installs the Playwright chromium binary, spawns vitepress preview on a free port, then waits for the listener to bind. The route walk allocates one BrowserContext per worker (cap = Math.Min(8, Environment.ProcessorCount)) and reuses it across that worker's share of routes; a companion negative test pins the detector itself by asserting a synthetic unmapped route surfaces as a 404.
  • tests/MTConnect.NET-Docs-Tests/MTConnect.NET-Docs-Tests.csproj—adds Microsoft.Playwright 1.60.0.
  • .github/workflows/dotnet.yml—adds actions/setup-node (pinned to v6.4.0) on the ubuntu-latest leg so the test fixture's npm ci && npm run build bootstrap can run. Caches docs/package-lock.json, and the Playwright cache key hashes the installed playwright-core package.json so a chromium-revision bump auto-invalidates.

@ottobolyos ottobolyos changed the title ci: add Playwright route-check job for the docs site docs(site): self-rendering local dev (regen hooks, auto sidebars, route check) Jun 1, 2026
@ottobolyos ottobolyos changed the title docs(site): self-rendering local dev (regen hooks, auto sidebars, route check) docs(site): auto-regen hooks, auto sidebars, Playwright route check Jun 1, 2026
@ottobolyos ottobolyos changed the title docs(site): auto-regen hooks, auto sidebars, Playwright route check docs(site): auto-regen hooks, auto sidebars, e2e route check Jun 1, 2026
@ottobolyos ottobolyos force-pushed the chore/docs-route-crawler branch from b1cdb78 to ec2928c Compare June 1, 2026 18:41
ottobolyos added a commit to ottobolyos/mtconnect.net that referenced this pull request Jun 1, 2026
ottobolyos added a commit to ottobolyos/mtconnect.net that referenced this pull request Jun 1, 2026
ottobolyos added a commit to ottobolyos/mtconnect.net that referenced this pull request Jun 1, 2026
ottobolyos added a commit to ottobolyos/mtconnect.net that referenced this pull request Jun 1, 2026
ottobolyos added a commit to ottobolyos/mtconnect.net that referenced this pull request Jun 1, 2026
ottobolyos added 10 commits June 1, 2026 23:54
Adds docs/scripts/check-routes.mjs, which spawns `vitepress preview`
against the pre-built dist/ tree, walks every route derived from the
markdown source tree in a headless Chromium browser (via Playwright),
detects VitePress's .NotFound 404 marker per page, and exits non-zero
on any 404. Concurrency-capped at 8 via p-limit; progress line every
25 routes keeps CI logs live during the walk.

Updates docs/package.json with a `check-routes` npm script and three
new devDependencies: playwright, p-limit, tree-kill.
Adds two sequential steps to the `build` job in docs.yml, immediately
after `Build site`:

1. `Install Playwright chromium` — runs `npx playwright install
   --with-deps chromium` so the OS-level shared libraries are present
   on ubuntu-latest without a separate apt step.
2. `Check VitePress routes` — runs `npm run check-routes`, which
   spawns `vitepress preview` against the built dist/ tree, walks
   every route in a headless Chromium browser, and exits non-zero on
   any client-side 404.

Both steps run on every push and pull_request trigger (not gated on
the deploy path) so broken routes surface in PR review, not after a
deploy.
Wire `generate-api-ref.sh` + `generate-reference.sh` into npm `predev`
and `prebuild` hooks via a `regen` umbrella script. A contributor who
clones the repo and runs `npm run dev` (or `npm run build`) now gets
a fully populated reference tree without remembering to run the two
generators by hand.

Adds a `check` umbrella script for local convenience (currently fans
out to `check-routes`; future link checkers can be added without
touching the CI workflow).

Trims the now-redundant explicit `generate-api-ref.sh` and
`generate-reference.sh` workflow steps; `npm run build`'s `prebuild`
hook handles both. The `--check` drift gate moves ahead of the build
so it runs against the committed reference pages, not the freshly
regenerated ones the prebuild hook would overwrite them with.
Replaces the hand-curated `/api/` and `/reference/` sidebar blocks in
`docs/.vitepress/config.ts` with auto-derivation from the generated
trees on disk. Two functions in the new `docs/.vitepress/sidebar.ts`:

  - apiSidebar()       parses docfx's `toc.yml` and projects the
                       namespace hierarchy into nested collapsible
                       groups. `MTConnect.Devices.DataItems.SampleDataItem`
                       lands at `MTConnect > Devices > DataItems >
                       SampleDataItem`. All ~1800 types are one click
                       from a section landing; the tree is collapsed
                       by default so the viewport stays usable.
  - referenceSidebar() lists every `.md` file under `docs/reference/`
                       except `index.md` (wired in as Overview). Flat
                       alphabetical list — a future page emitted by
                       `MTConnect.NET-DocsGen` surfaces automatically
                       once the regen scripts have produced it.

Hand-curated narrative sections (`/compliance/`, `/configure/`,
`/concepts/`, `/cookbook/`, `/modules/`, `/wire-formats/`,
`/troubleshooting/`, `/examples/`, `/migration/`, `/development/`,
`/cli/`) remain hand-edited; only the two generated trees swap to
auto-derivation. Gitignores the VitePress `.temp/` cache directory
that the build emits alongside `dist/`.
The route check belongs in the dotnet test matrix, not as a
docs-workflow Node script. Drop the .mjs crawler, its three
devDependencies (playwright, p-limit, tree-kill), the
check-routes / check npm scripts, and the two docs-workflow steps
that installed chromium and ran the crawler. The .NET e2e form
that supersedes this lands in the next commit.
Walks every markdown-backed VitePress route against the built
dist/ tree in a headless Chromium browser and fails on any
client-side 404. Categorised E2E so the existing dotnet matrix
runs it on the ubuntu-latest leg (the windows-latest leg already
excludes Category=E2E).

The fixture's one-time setup builds the docs site via
`npm ci && npm run build` if dist/ is missing, installs the
Playwright chromium binary, spawns `vitepress preview` on a free
port, then waits for the listener to bind. The test class lives
alongside DocsReferenceGenerationTests; both belong to the same
docs-coverage surface but cover different failure modes (drift vs
router gaps).
MTConnect.NET-Docs-Tests now ships a Category=E2E route walk whose
[OneTimeSetUp] bootstraps the docs site via `npm ci && npm run
build` when dist/ is missing, then drives `vitepress preview`
through Microsoft.Playwright. Only the ubuntu-latest leg runs
Category=E2E (windows-latest filters it out), so pin Node 20
there with the npm cache keyed on docs/package-lock.json. The
existing `dotnet test MTConnect.NET.sln` invocation already
discovers the test project; no separate test step is needed.
VitePress's SPA keeps background work alive indefinitely (analytics,
web-vitals, hot-reload polling), so NetworkIdle never settles within
the 15 s window. Switch to WaitUntilState.Load, which fires once all
sub-resources finish — well before background JS quiets — and is
sufficient for the .NotFound / title / body-text 404 detection.
Raise PageNavigationTimeoutMs to 30 000 ms to give complex pages
(e.g. multiple Mermaid diagrams) enough headroom under Load.
@ottobolyos ottobolyos force-pushed the chore/docs-route-crawler branch from ec2928c to d8721ce Compare June 1, 2026 21:55
ottobolyos added a commit to ottobolyos/mtconnect.net that referenced this pull request Jun 1, 2026
ottobolyos added a commit to ottobolyos/mtconnect.net that referenced this pull request Jun 1, 2026
ottobolyos added 10 commits June 2, 2026 15:15
The prepare-docs job stopped generating the docfx /api/ tree and the
Roslyn /reference/ tree, banking on the downstream Build VitePress site
job's `npm run build` to trigger the `prebuild` npm hook instead. The
Build job has neither the .NET SDK nor docfx installed, so the hook
fails on `docfx not found on PATH`. The Check internal links job
downloads the same sparse artefact (only the committed `docs/api/index.md`
landing page) and reports ~150 `/api/MTConnect.*` broken links.

Restore the two regen steps in prepare-docs so the artefact carries the
full generated tree, and invoke VitePress directly in the Build job to
skip the now-redundant prebuild hook. The hook stays in package.json
for local dev convenience.
ottobolyos added a commit to ottobolyos/mtconnect.net that referenced this pull request Jun 2, 2026
ottobolyos added a commit to ottobolyos/mtconnect.net that referenced this pull request Jun 2, 2026
ottobolyos added a commit to ottobolyos/mtconnect.net that referenced this pull request Jun 2, 2026
@ottobolyos ottobolyos marked this pull request as ready for review June 2, 2026 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

2 participants