Skip to content

fix: Normalize terminology in sample updates - #1397

Merged
IEvangelist merged 6 commits into
mainfrom
dapine/fix-sample-terminology
Jul 29, 2026
Merged

fix: Normalize terminology in sample updates#1397
IEvangelist merged 6 commits into
mainfrom
dapine/fix-sample-terminology

Conversation

@IEvangelist

Copy link
Copy Markdown
Member

Summary

  • centralize Aspire terminology normalization for generated data
  • normalize sample titles, descriptions, README content, and AppHost code before writing samples.json
  • prevent future generated catalogs from reintroducing .NET Aspire or app host

This fixes the sample-update regression visible in #1393.

Tests

  • pnpm --dir ./src/frontend exec vitest run --config vitest.config.ts tests/unit/update-samples.vitest.test.ts tests/unit/update-integrations.vitest.test.ts
  • replayed the samples.json content from chore: Update integration data and GitHub stats (7/24/26) #1393 through the normalizer (8 deprecated occurrences reduced to 0)

Copilot AI review requested due to automatic review settings July 23, 2026 01:56

@aspire-repo-bot aspire-repo-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated wording suggestions from the forbidden-words check. Apply the suggestions to resolve them.

Comment thread src/frontend/scripts/aspire-terminology.ts Outdated
Comment thread src/frontend/scripts/aspire-terminology.ts Outdated
Comment thread src/frontend/tests/unit/update-samples.vitest.test.ts Outdated
Comment thread src/frontend/tests/unit/update-samples.vitest.test.ts Outdated
Comment thread src/frontend/tests/unit/update-samples.vitest.test.ts Outdated
Comment thread src/frontend/tests/unit/update-samples.vitest.test.ts Outdated
Comment thread src/frontend/tests/unit/update-samples.vitest.test.ts Outdated
Comment thread src/frontend/tests/unit/update-samples.vitest.test.ts Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR centralizes terminology normalization (e.g., replacing “.NET Aspire” and “app host”) into a shared helper and applies it during data generation to keep samples.json and integration metadata consistent going forward, addressing the regression noted in #1393.

Changes:

  • Introduces normalizeAspireTerminology() and reuses it across generators.
  • Normalizes sample titles/descriptions/README text/AppHost code during update-samples generation and makes the script safe to import in tests by guarding the main() entrypoint.
  • Updates integration description normalization to use the shared helper and adds unit coverage (including asserting existing samples.json has no deprecated terms).

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/frontend/tests/unit/update-samples.vitest.test.ts Adds unit tests for sample terminology normalization and validates generated samples.json is clean.
src/frontend/src/utils/samples.ts Updates documentation comment to reflect that readmeRaw is terminology-normalized upstream content.
src/frontend/scripts/update-samples.ts Centralizes sample field normalization and guards main() so the module can be imported safely.
src/frontend/scripts/update-integrations.ts Replaces ad-hoc regex normalization with the shared terminology helper.
src/frontend/scripts/aspire-terminology.ts Adds a shared terminology normalizer used by generator scripts.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@aspire-repo-bot

Copy link
Copy Markdown
Contributor

Frontend HTML artifact ready

The latest frontend build uploaded the frontend-dist artifact for PR #1397. Use the VS Code button below to open this PR with GitHub Artifacts Explorer and browse the built HTML locally.

VS Code: Open PR #1397 artifacts

This comment updates automatically when a new frontend build artifact is uploaded.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: febf329c-97d8-41aa-bd42-43273bc57fe7
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: febf329c-97d8-41aa-bd42-43273bc57fe7
@IEvangelist
IEvangelist force-pushed the dapine/fix-sample-terminology branch from 9bbb307 to ec1ec43 Compare July 23, 2026 13:32
@IEvangelist IEvangelist added the ci:skip-forbidden-words Allows maintainers to bypass false positives in the forbidden-word check. label Jul 23, 2026

@JamesNK JamesNK left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

Phase A source checkpoint: microsoft/aspire main at e6388016c6c4b90f47589c15e9714975dc4aebc4 (2026-07-24). Claims: 0 product API/config/CLI claims were introduced, so there are no source verdicts to report.

Phase B: No documentation route or rendered example changed; 0 critical issues and 0 warnings. The docs server was not started because there was no affected page to exercise and the requested review excluded a website build.

The generation-code review found two blocking normalization cases, documented inline.

Phase A - Claim verification

No changed file introduces a factual claim about Aspire APIs, runtime behavior, configuration, CLI surface, or package versions.

Verified and verified-with-nuance claims

None; the changed prose is implementation-local commentary rather than Aspire product documentation.

Phase B - Doc-tester results

Documentation Test Report

Focus Area: PR #1397 changed-file route mapping
Date: 2026-07-24
Tester: doc-tester agent
PR ref: ec1ec43481ddfce284d53cddf6c65859beb613c5
Local server: Not started - no documentation routes were affected.

Summary

Category Passed Failed Warnings
Content Accuracy 0 0 0
Code Examples 0 0 0
CLI Commands 0 0 0
Links 0 0 0

Critical Issues

None.

Warnings

None.

Passed Checks

  • Changed-file mapping confirmed that the PR only changes generator scripts, a sample utility comment, and unit tests; no rendered documentation page or route is modified.

Recommendations

None for rendered documentation.

Knowledge Gaps

None.

Comment thread src/frontend/scripts/aspire-terminology.ts Outdated
Comment thread src/frontend/scripts/aspire-terminology.ts Outdated
…alizer

Address review feedback on #1397:

- Require .NET to sit at a non-word boundary so tokens like ASP.NET Aspire and Microsoft.NET Aspire are left intact instead of corrupted into ASPAspire / MicrosoftAspire.

- Consume Markdown emphasis/link openers (**, [) between the article and the term so 'a **.NET Aspire**' and 'a [.NET Aspire](url)' correct the article to 'an'.

- Add regression tests for bold/link article correction and word-boundary cases.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 020561bf-2df6-4f8b-a730-1c72d0f6cc5e
Replace the hardcoded replace-chain with a small TerminologyRule table so a new deprecated term is a single entry (pattern/replacement/optional article). Article correction, Markdown-wrapper tolerance, and word-boundary guarding are now applied generically per rule.

Also add the 'dotnet aspire' -> 'Aspire' rule that was present in .github/forbidden-words.json but missing from the normalizer, and cover it with tests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 020561bf-2df6-4f8b-a730-1c72d0f6cc5e
@IEvangelist
IEvangelist requested a review from JamesNK July 27, 2026 13:15

@JamesNK JamesNK left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static review of 5273ce9ae51636f9416d9bf976c0a1da7dd6d0ae against PR base 0df9b83ee31925baf6ab98f8e07579020be8bba7.

The three terminology rules match .github/forbidden-words.json, and the new call sites cover generated integration descriptions plus sample text fields. I found three remaining issues in the normalizer and its generated-data guard, noted inline.

No documentation route is changed, so there was no affected page to exercise. Per reviewer instruction, I did not run builds, tests, generators, or a development server.

Comment thread src/frontend/tests/unit/update-samples.vitest.test.ts Outdated
Comment thread src/frontend/scripts/aspire-terminology.ts Outdated
Comment thread src/frontend/scripts/aspire-terminology.ts Outdated
… wrappers

Replaces the per-rule mix of \\b\/\(?<!\w)\ boundaries with uniform alphanumeric edge guards applied centrally, so a rule core can never fuse into a longer token (e.g. \.NET AspireX\) and authors cannot forget a boundary. Underscore and inline-code wrappers now normalize correctly (\_.NET Aspire_\, \\.NET Aspire\\), and the generated-data invariant now covers the \dotnet aspire\ spelling.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 020561bf-2df6-4f8b-a730-1c72d0f6cc5e
@IEvangelist
IEvangelist requested a review from JamesNK July 27, 2026 15:28

@JamesNK JamesNK left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the terminology normalization logic, generated-field coverage, boundary handling, and tests. No actionable issues found.

@adamint adamint left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is safe to merge as-is. The normalizer is applied to readmeRaw and appHostCode (update-samples.ts:124,126), but it's a plain regex pass over raw Markdown and C#, so it rewrites inside code fences, inline code, and string literals.

I ported aspire-terminology.ts at this head (30923d83) and ran it. Real output:

IN : "```bash\ndotnet aspire run\n```"
OUT: "```bash\nAspire run\n```"

IN : "Run `dotnet aspire run` to start."
OUT: "Run `Aspire run` to start."

IN : "var cmd = \"dotnet aspire run --project ./AppHost\";"
OUT: "var cmd = \"Aspire run --project ./AppHost\";"

Aspire run isn't a command anyone can paste. The .NET AspireAspire rule substitutes a fixed capitalized string, so it can't produce the lowercase aspire run a shell would need. This matters because readmeRaw is what the Copy / View Markdown buttons serve (src/utils/samples.ts:195-196) and appHostCode renders through <Code lang="csharp"> on the sample page — so we'd be handing people broken commands.

Two more from the same pass:

IN : "# A .NET Aspire sample\n\nSee [overview](#a-net-aspire-sample)."
OUT: "# An Aspire sample\n\nSee [overview](#a-net-aspire-sample)."

The heading gets rewritten, the anchor doesn't, so in-page links break.

IN : "The app.host file and my-app host"      OUT: "The app.host file and my-AppHost"
IN : ".NET ASPIRE and DOTNET ASPIRE"          OUT: "Aspire and Aspire"

The boundaries are alphanumeric-only, so - isn't a boundary, and case isn't preserved.

To be fair about severity: I checked the committed samples.json at this head and there are currently zero matches for dotnet aspire, .NET Aspire, or app host, so nothing is corrupted today. But this script exists to run against future upstream READMEs, and dotnet aspire run in a fenced block is about the most likely thing for one to contain. It'd land silently in generated data.

What I'd do: normalize the prose fields only (title, description, readme if that's rendered prose) and leave appHostCode alone entirely — it's C#, there's no prose to fix. For the Markdown fields, run it through an AST pass that skips code/inlineCode nodes, link destinations, and headings that already have inbound anchors. The repo already has remark in the dependency graph, so that's not a big lift.

Smaller things:

  • normalizeAspireTerminology(null) throws (TypeError). PackageRecord.description is typed non-null and the call sites guard, but the previous optional-chaining behavior was more forgiving — worth a guard or an explicit contract.
  • Worth adding tests for the cases above. The current tests don't cover fenced commands, inline code, anchors, case, or idempotence. Idempotence does hold, I checked.

Credit where it's due: the boundary work is solid — ASP.NET Aspire and .NET AspireX both stay intact, which is the part I expected to be broken and wasn't.

Also worth asking: .github/forbidden-words.json + check-forbidden-words.sh already gate this terminology in CI. This silently rewrites third-party README content instead of failing, which is a different policy than what we apply to our own docs. Is silent rewriting what we want for content we don't own?

The normalizer ran a plain pass over raw Markdown and appHostCode, which could rewrite runnable sample commands (e.g. 'dotnet aspire run' -> 'Aspire run') inside fenced/inline code and corrupt compilable C#. Now fenced blocks and inline code are copied through verbatim, appHostCode is left untouched (it renders as C#, not prose), and the entry point is null-safe. Adds code-skip, mixed prose+code, idempotence, and null tests, and replaces the deprecated-term scan with an idempotence-based invariant.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 020561bf-2df6-4f8b-a730-1c72d0f6cc5e
@IEvangelist

Copy link
Copy Markdown
Member Author

Thanks for the careful review @adamint — especially porting the script and running it. The "un-pasteable command" case is exactly the failure mode I wanted to avoid, so I've made the normalizer code-safe in aba2cb5.

Fixed

  • Code fences, inline code, and C# are no longer rewritten. The normalizer now copies fenced blocks (``` / ~~~) and inline code spans (`) through verbatim, applying rules only to the prose gaps between them. Your three cases now round-trip unchanged:
    • ```bash\ndotnet aspire run\n``` → identical
    • Run `dotnet aspire run` to start. → identical
    • readmeRaw prose is still normalized, but any embedded command survives.
  • appHostCode is no longer normalized at all. It renders through <Code lang="csharp">, so there's no prose to fix and every reason not to touch identifiers, string literals, or comments. Removed from normalizeSampleTerminology (update-samples.ts) with a comment explaining why.
  • normalizeAspireTerminology(null) no longer throws. It's now null-safe via overloads — string → string, and null/undefined pass through unchanged.
  • Tests added for fenced commands, inline-code commands, a fenced legacy brand, mixed prose+code, idempotence (prose and mixed), and the null/undefined contract. The generated-data invariant is now idempotence-based (normalize(x) === x for title/description/readme/readmeRaw), which is both stronger than the old fixed-term scan and robust to code fences.

On the AST suggestion

I went with string-region masking (skip fenced + inline code) rather than a full remark round-trip. It covers the realistic corruption vectors you found without adding a parser dependency, and — importantly for readmeRaw, which the Copy/View Markdown buttons serve byte-for-byte — it leaves the raw Markdown exactly as authored outside prose. Happy to revisit an AST pass if we later need to skip link destinations too, but I wanted to keep this "simple and clean."

Intentional (with rationale)

  • Case: .NET ASPIREAspire is deliberate. Aspire is the canonical brand regardless of input casing, so normalizing to it (not preserving ASPIRE) is the intended outcome.
  • Hyphen boundary: treating punctuation (-, _, *, [) as a boundary is by design — JamesNK specifically asked for _/*/[ to count as boundaries. Making - a non-boundary on the start side would be asymmetric with the end side (.NET Aspire-basedAspire-based relies on - being a boundary there). my-app host fusing is rare enough that I'd rather not risk that regression.
  • Heading anchors: the on-page renderer regenerates heading slugs from the (normalized) text, so the auto-generated TOC stays consistent. The only breakage is a hand-authored [text](#old-slug) link pointing at the pre-normalization slug, which is narrow. A correct fix needs slug-aware in-doc link rewriting; I left it out to keep the change small, but can add it if you think it's worth it.

On the policy question

Fair to raise. The way I see it, these are two different layers: check-forbidden-words.sh gates first-party docs we author, where failing the build is the right call. This normalizer runs at generation time over third-party upstream READMEs we surface but don't own — we can't fail someone else's README, so normalizing terminology for display is the pragmatic path (which is why the ci:skip-forbidden-words label is on this PR). That said, if the team would rather surface a warning/report than silently rewrite, I'm open to it — happy to discuss.

I checked the same thing you did: zero matches in samples.json today, so this is future-proofing rather than a fix for current corruption. Re-requesting your review — thanks again.

@IEvangelist
IEvangelist requested a review from adamint July 29, 2026 01:32
@IEvangelist
IEvangelist merged commit 68f615a into main Jul 29, 2026
10 checks passed
@IEvangelist
IEvangelist deleted the dapine/fix-sample-terminology branch July 29, 2026 13:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:skip-forbidden-words Allows maintainers to bypass false positives in the forbidden-word check.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants