Skip to content

cmd: advisory check for un-overridden HTTP-framed descriptions (API-488)#170

Open
somanshreddy wants to merge 2 commits into
06-12-cli_description_overridesfrom
06-12-cli_description_resync_check
Open

cmd: advisory check for un-overridden HTTP-framed descriptions (API-488)#170
somanshreddy wants to merge 2 commits into
06-12-cli_description_overridesfrom
06-12-cli_description_resync_check

Conversation

@somanshreddy

@somanshreddy somanshreddy commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Scope

Surfaces: CLI | Module: Output/Builder (codegen tooling)

Description

Stacked on #169 (the description-override overlay). That overlay reframes the HTTP-framed descriptions we know about today; this PR adds the mechanism that catches new ones. Every codegen resync pulls in fresh OpenAPI descriptions, and some are written in raw HTTP terms (poll via GET /v3/videos/{id}, Pass to POST /v3/video-agents) that read badly for a CLI user. Without a check, a resync author has no signal that a new description needs reframing.

make check-descriptions runs a standalone scanner (./checkdescriptions) that iterates gen.Groups and inspects each command's summary, description, flag help, and request/response schema-field descriptions — after the overlay is applied — for divergence markers.

Markers, validated against the real corpus:

  • uppercase HTTP verbs — \b(GET|POST|PUT|PATCH|DELETE)\b
  • version paths — /v\d+/...
  • poll words — \bpoll(ed|ing|s)?\b

cursor / next_token / presigned are deliberately not markers: the corpus analysis showed those are overwhelmingly accurate, CLI-neutral text (e.g. "Opaque cursor token from next_token", "Presigned URL to download"), so flagging them would only add noise.

Per-location suppression. A marker hit is action-listed unless that specific location (the summary, the description, that flag, or that field) is overridden in internal/clidesc. Suppressing per-command would let a partial override hide newly HTTP-framed text elsewhere on the same command; per-location keeps the resync-coverage guarantee while still silencing intentional cases (e.g. an override that legitimately mentions a manual S3 PUT). Because it scans the overlay-applied text, a location that #169 reframed is not re-flagged.

Advisory, not a gate. It prints a report and always exits 0; it never fails CI. The signal is inherently fuzzy — some marker hits are accurate text (a webhook url that legitimately describes an inbound POST; a voice get description that correctly says "use this to poll"). The tool surfaces candidates for a human to judge rather than blocking the build on heuristic matches.

Resync workflow (documented in CONTRIBUTING.md): after make generate, run make check-descriptions; for anything flagged that genuinely misleads a CLI user, add an override in internal/clidesc. This is how new HTTP-framed descriptions get caught and reframed going forward.

This PR is purely additive on top of #169: it imports the internal/clidesc overlay table as the shared source of truth and adds the tool, the make target, and the resync doc — no changes to existing behavior. On the current gen/, the checker reports 48 total marker hits, 43 at locations without an override (the expected advisory tail — KEEP-class text like webhook callback URLs, schema "data"-wrapper descriptions citing endpoints, and the correctly-CLI-framed voice get poll note).

Testing

make build and make test pass (the checkdescriptions and internal/clidesc packages have unit tests: marker scanning incl. the deliberately-excluded cursor/presigned cases, nested schema-field collection, the overlay resolvers, and a guard that every override key maps to a real generated command). make lint is broken on clean main (golangci-lint go1.23 vs go1.25 toolchain) unrelated to this change; ran gofmt -l (clean for changed files) and go vet ./... (clean) as substitutes. Reviewed with Codex (2 rounds; it caught the per-command vs per-location suppression gap, now fixed).

@somanshreddy somanshreddy force-pushed the 06-12-cli_description_resync_check branch from d3fd237 to 8a04bc6 Compare June 12, 2026 11:45
@somanshreddy somanshreddy force-pushed the 06-12-cli_description_overrides branch 2 times, most recently from d25524a to d840978 Compare June 12, 2026 11:55
Adds `make check-descriptions`, a standalone tool that scans every generated
command's summary/description/flag-help/schema-field text for HTTP-framing
markers (uppercase HTTP verbs, /vN/ paths, poll words) and reports any flagged
text on a command that has no override in internal/clidesc. Advisory only:
prints a report, always exits 0, never fails CI. It is how new HTTP-framed
descriptions get caught and reframed on each codegen resync.

Purely additive on top of the overrides overlay: imports internal/clidesc as
the shared source of truth; no changes to existing behavior.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@somanshreddy somanshreddy force-pushed the 06-12-cli_description_resync_check branch from 8a04bc6 to afea2e3 Compare June 12, 2026 11:56
Address Codex review (F1/H1): the checker scanned overlay-applied text but
then marked every hit on a command "informational" if the command had ANY
override. A partial override (common) therefore hid newly HTTP-framed text in
that command's uncovered summary/description/flags/fields, defeating the
resync-coverage purpose. Now a hit is action-listed unless THAT specific
location is overridden, which also keeps suppressing intentional cases (e.g.
the asset override that legitimately mentions a manual S3 "PUT"). Per-location
semantics resolves H1.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant