Replace internal-source line selectors with string markers; retire Scribe/DPM remote extraction#1059
Draft
danielporterda wants to merge 3 commits into
Draft
Replace internal-source line selectors with string markers; retire Scribe/DPM remote extraction#1059danielporterda wants to merge 3 commits into
danielporterda wants to merge 3 commits into
Conversation
…traction extractByStringMarker() used .trim() on the extracted region, which stripped leading whitespace from the first content line only. For indented sources (RST literal blocks), that misaligned the first line against the rest of the snippet. Trim blank edge lines instead, preserving every line's indentation — matching the Sphinx literalinclude :start-after:/:end-before: semantics this function already documents. This is behavior-neutral for every existing stringMarker snippet (daml, splice): their marked content starts at column 0, where .trim() and blank-edge trimming are equivalent, confirmed by scanning all 11 checked-in stringMarker outputs. Also fail extraction when a marker string does not appear exactly once in its source file, so a stray duplicate can't silently select the wrong region. Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Replace the line-number selectors for the internal source repositories with the stringMarker tokens now present on the source branches: - canton-snippet-list-remote.json: 10 selectors -> CF_DOCS_SNIPPET_* comment markers in community/app documentation-snippets .conf files (DACH-NY/canton#34155). - daml-shell-snippet-list-remote.json: all 20 selectors -> cf-docs-snippet-daml-shell-* :name:/comment markers in the two RST docs (DACH-NY/daml-shell#193). Regenerate the checked-in snippet outputs with focused extraction against those source branches. Daml Shell content is byte-for-byte identical to regenerating with the old line selectors on current upstream main (the only checked-in diff is a stale trailing blank line that today's wrapCode trimming already removes). Six Canton snippets change content intentionally, because the old line ranges had drifted: three configuration-reference snippets regain their closing braces, the remote-participant snippet regains its block header, and the two performance-optimization snippets become the complete storage-pool and pruning-batch blocks instead of fragments that split the storage block. Retire the max-burst-factor snippet entirely: DACH-NY/canton removed that configuration in d4ed3d74e485 when BaseCantonConfig started failing on unknown keys. Drop the selector, the generated snippet, its import, and the unsupported 'Batch sizes' example from the global-synchronizer performance-optimization page. Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…cf-docs Both upstream repositories deleted their docs after transferring documentation ownership to cf-docs: - DACH-NY/scribe 75ac557896d7 removed the docs and states PQS documentation is now hosted by canton-network/cf-docs. - digital-asset/dpm 6c85930ac6e1 deleted its external docs because they now live in canton-network/cf-docs. Extracting from those repositories can never succeed again, so remote extraction is retired rather than recreating upstream docs to host markers: - Rehome the 22 still-used PQS snippets from the misleading snippets/external/scribe/main path to cf-docs-owned snippets/pqs/ and update the three importing pages. - Rehome the 3 used DPM snippets to snippets/dpm/, delete the 3 unused checked-in DPM snippet files, and update the two importing pages. (The seventh DPM config entry, dpm-components-shell-42, had no checked-in output at all.) - Remove both configs from remote-snippet-lists.json, delete scribe-/dpm-snippet-list-remote.json, and drop the scribe/dpm generator targets from generate_external_snippets.py. Add regression tests that fail if the canton or daml-shell external configs reintroduce location.type == "lines", if the retired scribe/dpm configs come back, or if marker tokens collide within a source file, plus unit tests for the indentation-preserving and exactly-once stringMarker extraction semantics. Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Contributor
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
Blocked on the two source PRs merging: DACH-NY/canton#34155 and DACH-NY/daml-shell#193. The marker selectors here reference tokens added on those branches. Generation was validated against the source PR commits (canton
982b60d56381ba58013c231c65357dbc9f73dfc8, daml-shell3f5e4315e10c992eb3ff7fdb42290e5d8bee4907); re-validate against the merged commits before landing.What
Eliminates every
location.type == "lines"selector associated with internal source repositories and retires stale remote-source metadata where upstream explicitly transferred docs ownership to cf-docs.Selector counts (before → after)
linesbeforecanton-snippet-list-remote.jsonstringMarker, 1 retired)daml-shell-snippet-list-remote.jsonstringMarker)scribe-snippet-list-remote.jsondpm-snippet-list-remote.jsonThe remaining 164 line selectors in public
digital-asset/cn-quickstartare a separate migration and are not claimed as fixed by this internal-repo work.Commits
extractByStringMarker()now trims blank edge lines instead of.trim(), preserving first-line indentation for indented sources (RST literal blocks), and fails if a marker doesn't appear exactly once. Behavior-neutral for all 11 existing daml/splice stringMarker snippets (their content starts at column 0; verified by scanning the checked-in outputs).max-burst-factorsnippet (Canton d4ed3d74e485 removed that config whenBaseCantonConfigbegan failing on unknown keys), including its import and the unsupported Batch sizes section on the GS performance-optimization page.75ac557896d7) and DPM (digital-asset/dpm6c85930ac6e1) both deleted their docs after transferring ownership to cf-docs, so remote extraction can never succeed again. The 22 used PQS snippets move tosnippets/pqs/, the 3 used DPM snippets tosnippets/dpm/, 3 unused DPM snippet files are deleted (a 7th config entry had no checked-in output), configs and generator targets are removed, and regression tests lock all of this in. Note: the live DPM repository is publicdigital-asset/dpm;DACH-NY/dpmdoes not exist.Generated-output diff summary
wrapCodetrimming already removes. The marker switch itself is content-neutral (verified by diffing both generations).}canton.remote-participants.myparticipant {headerValidation
npm run test:external-snippets— 9 passed (includes new regression tests: nolinesselectors for these repos, retired configs stay retired, marker-token uniqueness, indentation-preserving + exactly-once extraction semantics)jqvalidation of every changed JSON file — OKgit diff --check— cleanbroken-linksfromdocs-main— output identical tomain(only pre-existing|gsf_scan_url|placeholder warnings)/snippets/...import resolves to an existing file — 0 missingSource ownership notes
75ac557896d794f5f7e74b07da4f9a3f7787b3dcstates PQS documentation is now hosted by canton-network/cf-docs); this PR does not recreate upstream docs to host markers.6c85930ac6e14ee51e744a752ee6bdb8be53d96a) for the same reason.