cmd: advisory check for un-overridden HTTP-framed descriptions (API-488)#170
Open
somanshreddy wants to merge 2 commits into
Open
cmd: advisory check for un-overridden HTTP-framed descriptions (API-488)#170somanshreddy wants to merge 2 commits into
somanshreddy wants to merge 2 commits into
Conversation
d3fd237 to
8a04bc6
Compare
d25524a to
d840978
Compare
Adds `make check-descriptions`, a standalone tool that scans every generated command's summary/description/flag-help/schema-field text for HTTP-framing markers (uppercase HTTP verbs, /vN/ paths, poll words) and reports any flagged text on a command that has no override in internal/clidesc. Advisory only: prints a report, always exits 0, never fails CI. It is how new HTTP-framed descriptions get caught and reframed on each codegen resync. Purely additive on top of the overrides overlay: imports internal/clidesc as the shared source of truth; no changes to existing behavior. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
8a04bc6 to
afea2e3
Compare
Address Codex review (F1/H1): the checker scanned overlay-applied text but then marked every hit on a command "informational" if the command had ANY override. A partial override (common) therefore hid newly HTTP-framed text in that command's uncovered summary/description/flags/fields, defeating the resync-coverage purpose. Now a hit is action-listed unless THAT specific location is overridden, which also keeps suppressing intentional cases (e.g. the asset override that legitimately mentions a manual S3 "PUT"). Per-location semantics resolves H1. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope
Surfaces: CLI | Module: Output/Builder (codegen tooling)
Description
Stacked on #169 (the description-override overlay). That overlay reframes the HTTP-framed descriptions we know about today; this PR adds the mechanism that catches new ones. Every codegen resync pulls in fresh OpenAPI descriptions, and some are written in raw HTTP terms (
poll via GET /v3/videos/{id},Pass to POST /v3/video-agents) that read badly for a CLI user. Without a check, a resync author has no signal that a new description needs reframing.make check-descriptionsruns a standalone scanner (./checkdescriptions) that iteratesgen.Groupsand inspects each command's summary, description, flag help, and request/response schema-field descriptions — after the overlay is applied — for divergence markers.Markers, validated against the real corpus:
\b(GET|POST|PUT|PATCH|DELETE)\b/v\d+/...\bpoll(ed|ing|s)?\bcursor/next_token/presignedare deliberately not markers: the corpus analysis showed those are overwhelmingly accurate, CLI-neutral text (e.g. "Opaque cursor token fromnext_token", "Presigned URL to download"), so flagging them would only add noise.Per-location suppression. A marker hit is action-listed unless that specific location (the summary, the description, that flag, or that field) is overridden in
internal/clidesc. Suppressing per-command would let a partial override hide newly HTTP-framed text elsewhere on the same command; per-location keeps the resync-coverage guarantee while still silencing intentional cases (e.g. an override that legitimately mentions a manual S3PUT). Because it scans the overlay-applied text, a location that #169 reframed is not re-flagged.Advisory, not a gate. It prints a report and always exits 0; it never fails CI. The signal is inherently fuzzy — some marker hits are accurate text (a webhook
urlthat legitimately describes an inboundPOST; avoice getdescription that correctly says "use this to poll"). The tool surfaces candidates for a human to judge rather than blocking the build on heuristic matches.Resync workflow (documented in CONTRIBUTING.md): after
make generate, runmake check-descriptions; for anything flagged that genuinely misleads a CLI user, add an override ininternal/clidesc. This is how new HTTP-framed descriptions get caught and reframed going forward.This PR is purely additive on top of #169: it imports the
internal/clidescoverlay table as the shared source of truth and adds the tool, themaketarget, and the resync doc — no changes to existing behavior. On the currentgen/, the checker reports 48 total marker hits, 43 at locations without an override (the expected advisory tail — KEEP-class text like webhook callback URLs, schema "data"-wrapper descriptions citing endpoints, and the correctly-CLI-framedvoice getpoll note).Testing
make buildandmake testpass (thecheckdescriptionsandinternal/clidescpackages have unit tests: marker scanning incl. the deliberately-excluded cursor/presigned cases, nested schema-field collection, the overlay resolvers, and a guard that every override key maps to a real generated command).make lintis broken on cleanmain(golangci-lint go1.23 vs go1.25 toolchain) unrelated to this change; rangofmt -l(clean for changed files) andgo vet ./...(clean) as substitutes. Reviewed with Codex (2 rounds; it caught the per-command vs per-location suppression gap, now fixed).