Skip to content

docs(skills): align NyxID skill with CLI + deterministic AI flow rules#679

Draft
ctkm-aelf wants to merge 1 commit into
mainfrom
audit/skills-cli-alignment
Draft

docs(skills): align NyxID skill with CLI + deterministic AI flow rules#679
ctkm-aelf wants to merge 1 commit into
mainfrom
audit/skills-cli-alignment

Conversation

@ctkm-aelf

Copy link
Copy Markdown
Collaborator

Summary

The skills/nyxid/ bundle had drifted from the actual nyxid CLI in ways that would cause AI agents to fail silently or take wrong paths. This PR is a deliberate sweep to:

  1. Fix CLI drift — 7 HIGH-severity issues where the docs claimed flags or commands that don't exist (or use different syntax) in cli/src/cli.rs.
  2. Fix wizard-bypass bias — examples were leading agents to add --output json to wizard-capable create/rotate commands, which silently skips the browser wizard and dumps secrets to stdout.
  3. Fix edit-vs-create intent miscapture — agents kept reaching for nyxid api-key create when users asked to update an existing key. Partly because the skill never told them to list-first, partly because "API key" is overloaded (NyxID nyxid_ag_… agent key vs third-party downstream credential).
  4. Document missing CLI surfaceservice-account, developer-app, endpoint, external-key, node docker, info/repo/telemetry were not covered anywhere in the skill.
  5. Pin deterministic AI flows — added explicit decision trees so an agent reading the skill cold can pick the right command on the first try.

Highlights

  • SKILL.md — rewrote "Working Rules" into two subsections: "Capture user intent before acting" (list-first, edit-vs-create, api-key vs external-key disambiguation) and "CLI invocation defaults" (wizard is the default, --no-wait --output json for headless agents, scripted-flag exceptions narrowed).
  • references/services.md — added a 7-step "Quick reference for AI agents (deterministic flow)" at the top so agents don't have to derive the path from prose.
  • references/managing.md — new "Capturing user intent: edit vs. create" section with a 10-row "Edits agents commonly mistake for create" table mapping wrong invocations to right ones. Added new nyxid endpoint and nyxid external-key reference sections.
  • references/nodes.md — replaced ambiguous setup guidance with a 3-case decision matrix (same machine + catalog → credentials setup; same machine + custom → two-step; different machines → admin runs service add --via-node, operator runs credentials add).
  • references/accounts.md (new) — full nyxid service-account and nyxid developer-app reference plus a "which identity to pick" decision table.
  • HIGH drift fixeschannel-bot route create flag renames (--bot--bot-id, --agent--agent-key-id), phantom nyxid keys create removed, positional → flag fixes for service add-ssh --label and node credentials add --service, removed false claim that nyxid api-key update --rate-limit-per-second exists (rate limits are set via web UI / raw HTTP).
  • JSON sweep--output json added to read commands (list, show, discover, status); deliberately NOT added to wizard-capable creates/rotates as a default (only as labeled "scripted escape hatch" with warnings).

Verification

Three independent sub-agent reviews:

  1. CLI cross-check (round 1) — diffed every nyxid … invocation in the docs against cli/src/cli.rs clap definitions. Found 7 HIGH drifts; all fixed in this PR.
  2. Post-edit verification (round 2) — re-checked the new edits against the CLI surface. Status: clean across 7 audit items.
  3. Fresh-reader usability test (round 3) — a reviewer with no prior NyxID context traced 8 realistic user flows through the docs and reported pass/fail for each. All 8 PASS, including the previously-broken "edit my agent key's scopes" case and the api-key-vs-external-key disambiguation.

Net change: 542 insertions / 121 deletions across 9 modified files + 1 new file (references/accounts.md).

Test plan

  • nyxid channel-bot route create --bot-id … --agent-key-id … parses and creates a route
  • nyxid service add-ssh --label … --host … parses (was previously documented with positional label)
  • nyxid node credentials add --service … --header … parses (was previously documented as positional slug)
  • nyxid service-account create --name … --scopes "…" opens the wizard by default
  • nyxid api-key create --no-wait --output json returns the pairing handoff JSON; nyxid pairing resume <id> completes the flow
  • Agent flow check: "edit my agent key's scopes" should NOT spawn nyxid api-key create
  • Agent flow check: "rotate my OpenAI API key" should disambiguate to nyxid service rotate-credential (not nyxid api-key rotate)

🤖 Generated with Claude Code

…lows

Two reported failure modes drove this sweep:

1. Wizard-bypass bias — examples were leading agents to add `--output json`
   to wizard-capable create/rotate commands, which silently disables the
   browser wizard and dumps secrets to stdout.
2. Edit-vs-create intent miscapture — agents kept calling `nyxid api-key
   create` when users asked to update an existing key, partly because the
   skill never told them to list-first and partly because "API key" is
   overloaded (NyxID `nyxid_ag_…` agent key vs third-party downstream
   credential).

Also fixes 7 HIGH-severity drifts agents would hit verbatim: `channel-bot
route` flag renames (`--bot` → `--bot-id`, `--agent` → `--agent-key-id`),
phantom `nyxid keys create`, positional → flag fixes for `service add-ssh
--label` and `node credentials add --service`, and removes claim that
`nyxid api-key update --rate-limit-per-second` exists.

Adds new reference for previously-undocumented surfaces: `service-account`,
`developer-app`, `endpoint`, `external-key`, `node docker`,
`info`/`repo`/`telemetry`. Pins canonical node-setup paths via a
same-machine-vs-split × catalog-vs-custom decision matrix.

Verified across three independent reviews including a fresh-reader
usability test of 8 user flows (all PASS).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant