Skip to content

DOCS-1879 - Remove the unused tag taxonomy - #7133

Merged
kimsauce merged 11 commits into
mainfrom
DOCS-1879
Sep 16, 2026
Merged

kimsauce merged 11 commits into
mainfrom
DOCS-1879

Conversation

@kimsauce

Copy link
Copy Markdown
Collaborator

Purpose of this pull request

This pull request removes the unused Docusaurus tag taxonomy sitewide. GA4 shows the ~120 auto-generated /help/docs/tags/* pages get almost no organic traffic and no established reader benefit, while costing ongoing upkeep (drift against keywords:, another thin crawl-surface URL per tag). See DOCS-1879 for the full rationale, including verified GA4 numbers and technical findings.

Five commits, one per batch:

  1. 93 C2C source docs (docs/send-data/hosted-collectors/cloud-to-cloud-integration-framework/): tags: renamed to keywords:, values unchanged.
  2. zip-tools.md: tags: removed (no existing keywords: field, single value not worth preserving).
  3. Style guide + 2 scaffold templates: tags: guidance removed/deprecated so contributors don't reintroduce it; templates renamed to keywords: to keep new docs consistent with (1).
  4. Claude Code tooling (2 skills + /app-doc command): same treatment, so the doc-creation slash commands match the updated templates.
  5. robots.txt + sitemap config: removed the now-obsolete Disallow: /tags/* / Disallow: /*?tags= rules and the /tags/** sitemap-ignore entry.

Verified with a clean local yarn build: zero docs-tags-* route artifacts generated, confirming the docs plugin stops generating tag routes once no doc has tags: frontmatter (there's no config flag for this — route generation is driven purely by frontmatter presence).

Not in this repo, flagged separately: the Algolia Crawler Admin's exclusionPatterns has two now-obsolete tag-related entries (harmless no-ops, not blocking) to remove manually in the Algolia dashboard.

Select the type of change

  • Site and Tools - .clabot, version updates, maintenance, dependencies, new packages for the site (Docusaurus, Gatsby, React, etc.)
  • Minor Changes - Typos, formatting, slight revisions
  • Update Content - Revisions, updating sections
  • New Content - New features, sections, pages, tutorials

Ticket (if applicable)

https://sumologic.atlassian.net/browse/DOCS-1879

🤖 Generated with Claude Code

kimsauce and others added 5 commits September 10, 2026 23:13
Rename the tags: frontmatter to keywords: across all Cloud-to-Cloud
source docs, preserving the existing values. GA4 shows the generated
/help/docs/tags/* pages get ~99.7% bot traffic and 17 organic views
total over 90 days, no SEO/GEO value, so the tags feature itself is
being retired sitewide. The terms are still useful for search, so
they move into keywords: instead of being dropped.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Drop the unused tags: [cloud soar integrations] frontmatter. Unlike
the C2C source docs, this page had no pre-existing keywords: field
and the single tag value isn't worth introducing one for, so it's
just removed. Also includes an incidental trailing-whitespace fix on
the Password bullet.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Remove tags: from the style guide's example frontmatter and the two
scaffold templates (app-template-v2, c2c-source), renaming it to
keywords: in the templates so new docs keep the same search terms
without generating a tag page. Mark tags: as deprecated in the style
guide's frontmatter parameter table so it isn't reintroduced.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Remove tags: from the docusaurus and sumo-style skill frontmatter
examples, and rename tags to keywords in app-doc.md's required
substitutions, checklist, and worked example so the /app-doc command
matches the updated app-template-v2.md scaffold.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…p config

Drop the Disallow: /tags/* and Disallow: /*?tags= rules from
robots.txt, and /tags/** from the sitemap ignorePatterns, now that no
tag routes exist to disallow or ignore.

Note: this repo's robots.txt is served at /help/robots.txt, not the
production domain root (https://www.sumologic.com/robots.txt, a
separate WordPress-managed file that never included these rules), so
this was cleanup rather than a fix to something that was blocking
crawlers in production.

The Algolia Crawler Admin's exclusionPatterns (external, not in this
repo) also has two tags-related entries that are now no-ops; flagged
separately for manual removal there.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@kimsauce kimsauce self-assigned this Sep 11, 2026
@cla-bot cla-bot Bot added the cla-signed Contributor approved, listed in .clabot file label Sep 11, 2026
kimsauce and others added 2 commits September 14, 2026 16:41
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The prior redirects commit covered every tag that mapped to exactly
one doc, but missed the 3 tags that were shared across multiple docs:
cloud-to-cloud (82 docs), cloud-siem-enterprise (4 docs), and cyberark
(2 docs, the live one plus an unlisted beta variant). Verified using
Docusaurus's actual lodash.kebabCase tag-slug algorithm against every
distinct tag value that existed on main, pre-PR: 96 total, now all 96
have a redirect (was 93/96).

Shared tags redirect to the C2C source list index page, since no
single vendor doc is a fair target for a page that covered several.
cyberark redirects to the live doc, not the unlisted beta one.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

@amee-sumo amee-sumo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left inline comments on 6 broken redirect targets in cid-redirects.json. Full context in my top-level review comment. Everything else I checked (the tags->keywords renames, zero remaining tags: frontmatter repo-wide, robots.txt/sitemap cleanup, style guide/template/tooling updates) is clean.

Comment thread cid-redirects.json
Comment thread cid-redirects.json
Comment thread cid-redirects.json
Comment thread cid-redirects.json
Comment thread cid-redirects.json
Comment thread cid-redirects.json
Fix redirects that assumed every C2C doc's filename ends in -source,
which isn't true for bitwarden.md, crowdstrike-fdr-host-inventory.md,
crowdstrike-filevantage.md, and jfrog-xray.md, plus one hyphenation
mismatch (intel-471-threat-intel-source.md, not intel471-...). All 6
were anyone hitting the old tag URL landing on a 404 instead of the
real doc.

Found by amee-sumo in PR review. Verified by checking every one of
the 97 /docs/tags/* redirect targets against the actual filesystem,
not just the 6 flagged — no other mismatches exist.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@kimsauce
kimsauce requested a review from amee-sumo September 16, 2026 07:21
@kimsauce

Copy link
Copy Markdown
Collaborator Author

Fixed all 6 broken redirect targets in f5d17e2. Replied on each thread with what it now points to.

While fixing them I checked every one of the 97 /docs/tags/* redirects against the actual filesystem, not just the 6 you flagged. No other mismatches. Also re-ran the earlier slug-completeness check (every distinct tag value that ever existed on main, matched against Docusaurus's real lodash.kebabCase algorithm) and it's still at 0 missing.

amee-sumo and others added 2 commits September 16, 2026 12:56
My review comments on this PR flagged these 5 as broken by checking their target against the filesystem filename. That was the wrong check: this repo's docs derive their actual URL from the `id:` frontmatter field, not the filename, and sidebars.ts confirms these routes are id-based. The original targets (before my requested fix) were already correct; the fix commit broke them by matching the filename instead. The build failure this caused is what surfaced the mistake.

Reverts:
- bitwarden -> bitwarden-source
- crowdstrike-fdr-host-inventory -> crowdstrike-fdr-host-inventory-source
- crowdstrike-filevantage -> crowdstrike-filevantage-source
- intel-471-threat-intel-source -> intel471-threat-intel-source
- jfrog-xray -> jfrog-xray-source (both /docs/tags/jfrog and /docs/tags/xray)

Verified all 97 /docs/tags/* redirects by building a route map from every doc's actual id: frontmatter (or slug:, or category-index behavior), not just filesystem paths.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@amee-sumo

Copy link
Copy Markdown
Collaborator

I owe a correction on my earlier review comments. I flagged 6 redirect targets as broken by checking them against the filesystem filename, but that was the wrong check — this repo derives each doc's actual URL from its id: frontmatter field, not the filename, and sidebars.ts confirms these specific routes are id-based (e.g. bitwarden.md has id: bitwarden-source, and the site's own sidebar links to it via .../bitwarden-source). The original redirect targets in this PR were already correct. The fix I asked for changed them to match the filename instead, which broke the real routes — that's exactly what run-build-checks just caught ("You are trying to create client-side redirections to invalid paths").

I've pushed a commit reverting those 5 entries back to their original, correct values, and re-verified all 97 /docs/tags/* redirects this time by building a route map from every doc's actual id:/slug: frontmatter (plus category-index root behavior), not filesystem paths. All 97 now resolve.

Sorry for the back-and-forth this caused — the PR itself was correct on this point from the start.

@amee-sumo amee-sumo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

run-build-checks now passes. All 97 /docs/tags/* redirects are verified against the real routing (id: frontmatter, not filenames), the tags->keywords renames are clean, zero tags: frontmatter remains repo-wide, and robots.txt/sitemap/style-guide/tooling updates all check out.

Approving.

@kimsauce
kimsauce enabled auto-merge September 16, 2026 19:24
@kimsauce
kimsauce added this pull request to the merge queue Sep 16, 2026
Merged via the queue into main with commit cfd7f9a Sep 16, 2026
4 checks passed
@kimsauce
kimsauce deleted the DOCS-1879 branch September 16, 2026 19:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed Contributor approved, listed in .clabot file

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants