Merged
Conversation
PR #109 added assertPathAllowed checks on input path fields (provar_home, project_path, results_path) but existing test fixtures used '..' segments that trigger PATH_TRAVERSAL before the allowedPaths.length guard. - antTools.test.ts: changed config to allowedPaths:[] and updated minimalInput() to use tmpDir-based paths (68 passing, was 26+38 fail) - automationTools.test.ts: fixed traversal test to use string concat instead of path.join (which normalises away '..' segments) - pathPolicy.test.ts: extracted typed local var to fix TS compile error that caused mocha to print help instead of running tests Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds 5 new MCP tools so AI agents can discover, read, validate, generate, and patch NitroX (Hybrid Model) .po.json component page objects for LWC, Screen Flow, Industry Components, Experience Cloud, and HTML5. Tools added: - provar.nitrox.discover: scan for Provar projects and inventory nitroX/ - provar.nitrox.read: read .po.json files for context/training - provar.nitrox.validate: validate against NX001-NX010 rules (score 0-100) - provar.nitrox.generate: generate new .po.json from component description - provar.nitrox.patch: apply RFC 7396 merge-patch to existing .po.json Also adds: - 44 unit tests in nitroXTools.test.ts covering all 5 tools - 5 smoke test entries (TOTAL_EXPECTED 33->38) - .gitignore guards for FactComponent/FactPackage schema files - QualityHub failure detection unit tests (6 new, documenting PR #110 fix) - README and docs/mcp* updates for NitroX tooling and security model Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…sary type assertions - Extract root property checks (NX001/NX002/NX003/NX010) into validateRootProperties() helper to bring validateNitroXContent cyclomatic complexity from 26 to <=20 - Remove redundant 'as string' cast in nitroXTools.test.ts (TypeScript narrows after typeof === 'string' type guard) - Remove redundant 'as string' casts in qualityHubTools.test.ts (same) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
feat(mcp): NitroX Hybrid Model MCP tools (provar.nitrox.*)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
chore: bump version to 1.5.0-beta.2
Three issues observed in the field: 1. provar.automation.metadata.download had a sparse description with no mention of config.load as a prerequisite or how to use the -c flag. Updated description to call out PREREQUISITE, correct flags usage (["-c", "Name1,Name2"]), and that [DOWNLOAD_ERROR] means auth failure. 2. When [DOWNLOAD_ERROR] appears in the error output (95% of the time an expired/wrong credential for the connection), the response now includes a details.suggestion field with actionable steps: check .secrets credentials, connection name spelling, scratch org expiry, and that testprojectSecrets is an encryption key not a file path. 3. testprojectSecrets in propertiesTools.ts and secrets_password in antTools.ts had descriptions easily misread as file paths. Both now explicitly say "encryption key string, not a file path". Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
PR comment fixes:
- details: pass undefined (not {}) when no suggestion, keeping response
shape backward-compatible (Copilot comment on automationTools.ts:271)
- flags description: soften "required" wording to avoid misleading callers
since connections flag is not enforced server-side (automationTools.ts:257)
- testprojectSecrets: change "Leave empty" to "Omit this field" to prevent
accidental key removal via empty string (propertiesTools.ts:229)
Add CLAUDE.md with project-wide Claude Code guidelines:
- Documentation update checklist for every MCP tool change
(docs/mcp.md, docs/mcp-pilot-guide.md, README.md, external docs)
- Test coverage requirements (unit, smoke, compile gates)
- MCP tool authoring standards (description quality, field descriptions,
error responses, path safety)
- Branch/PR conventions and version bump rules
- ESLint common gotchas
Also removes CLAUDE.md from .gitignore (project-level instructions
should be tracked; .claude/ dir is still ignored for session files).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix(mcp): improve metadata download UX and secrets field descriptions
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
chore: bump version to 1.5.0-beta.3
Adds src/services/auth/credentials.ts as the single source of truth for Provar API key storage (~/.provar/credentials.json) and resolution. Priority: PROVAR_API_KEY env var > stored file > null. Empty/whitespace env var treated as unset. Phase 2 fields (username, tier, expires_at) defined as optional from the start to avoid schema migration later. Full unit test coverage in test/unit/services/auth/credentials.test.ts. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds three new commands under sf provar auth: - set-key --key <pv_k_...>: stores key to ~/.provar/credentials.json - status: reports key source (env var / file / not configured) - clear: removes stored credentials with local-fallback warning Registers auth subtopic in package.json oclif config and creates accompanying messages/*.md files for all three commands. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… key-based routing Phase 1 sections 1.3 + 1.4 (and lint fixes for previously staged Phase 1 files): - src/services/qualityHub/client.ts: stub validateTestCaseViaApi, normaliseApiResponse mapping AWS API response to internal format (per AWS memo 2026-04-10): valid->is_valid, errors[]/warnings[]->issues[], quality_metrics.quality_score->quality_score. Added getInfraKey() for PROVAR_INFRA_KEY env var (separate from user pv_k_ key). - src/mcp/tools/testCaseValidate.ts: async handler, resolveApiKey routing, quality_hub/local/local_fallback validation_source, onboarding/fallback warnings - test/unit/services/qualityHub/client.test.ts: 11 tests for normaliseApiResponse, 2 for getInfraKey - test/unit/mcp/testCaseValidate.test.ts: 6 handler-level tests (no-key, success, 3 fallback paths) - docs/auth-cli-plan.md: updated header contract (x-provar-key + x-api-key) and response shape table - Lint fixes across all Phase 1 src/commands and test/unit/commands/provar/auth/* files Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Restore original os.homedir function reference in all 5 test files instead of replacing it with a new closure (prevents cross-suite stub leakage) - Add resolveApiKey() test for invalid pv_k_ prefix filtering - Add resolveApiKey() test for ignored env var without pv_k_ prefix in status tests - credentials.ts: ignore PROVAR_API_KEY env vars that lack pv_k_ prefix - set-key.ts: trim whitespace from --key flag before validation/storage - status.ts: detect and report invalid env key prefix as misconfiguration - testCaseValidate.ts: extract local metadata (id, name, step_count) from XML and merge into Quality Hub API response so consumers get consistent fields - Update Quality Hub handler test to assert merged metadata fields - Add NUT tests for sf provar auth set-key, status, and clear commands - Extend test:nuts glob patterns to discover new auth NUT files Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
substring(0,12) on 'pv_k_nuttest...' yields 'pv_k_nuttest' (12 chars), not 'pv_k_nuttest12' (14). Same off-by-two for status test key prefix. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- status.nut.ts: assert on 'Prefix:' label rather than a hardcoded prefix string (avoids off-by-one errors in substring arithmetic) - CI_Execution.yml: add 'if: always()' to the artifact upload step so mochawesome report is published even when NUT tests fail Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Auth flow confirmed as PKCE / Hosted UI (Option B) - Token strategy: exchange immediately, store only pv_k_, discard Cognito tokens - Document three registered callback ports (1717, 7890, 8080) and port-selection logic - Add full PKCE implementation sketch (code verifier, challenge, localhost listener) - Note Cognito endpoint config env vars (PROVAR_COGNITO_DOMAIN, PROVAR_COGNITO_CLIENT_ID) - Phase 1 CLI infrastructure unchanged — credentials.ts/set-key/resolveApiKey unaffected - Update Phase 2 Done criteria to include token-not-on-disk assertion Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Document three confirmed endpoints on shared base URL: POST /auth/exchange, GET /auth/status, POST /auth/revoke - Add client.ts stubs for exchangeTokenForKey, fetchKeyStatus, revokeKey - Plan sf provar auth status live check via /auth/status (graceful offline fallback) - Plan sf provar auth clear revoke via /auth/revoke (best-effort, deletes locally regardless) - Renumber Phase 2 sections to accommodate new 2.2/2.3/2.4 client/status/clear updates Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ase 2) - Add loginFlow service: PKCE pair generation, port selection from registered callbacks (1717/7890/8080), browser open, localhost callback server, and HTTPS code-for-tokens exchange - Add login command: full OAuth 2.0 Authorization Code + PKCE flow against Cognito Hosted UI, with Quality Hub token exchange at the end - Extend qualityHubClient: exchangeTokenForKey, fetchKeyStatus, revokeKey using node:https (no DOM fetch dependency) - Update status command: prefix validation for env var keys, live key check via fetchKeyStatus with silent offline fallback - Update clear command: best-effort server-side revoke before clearing local credentials - Add unit tests for loginFlow (generatePkce, listenForCallback, credential writing, exchangeTokenForKey stubs) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…p beta.4
POST /auth/exchange expects { "access_token": "..." } in the JSON body with
x-api-key infra header — not an Authorization: Bearer header. Corrected based
on AWS team handoff (2026-04-11).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…p beta.4
/auth/exchange, /auth/status, /auth/revoke no longer require the API Gateway
infra key — Cognito JWT and pv_k_ keys are sufficient. POST /auth/exchange now
sends the Cognito access token as { "access_token": "..." } in the request body.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…all docs - README: add sf provar auth login/set-key/status/clear command entries; update MCP section to describe local vs Quality Hub validation modes - docs/mcp.md: add Authentication section (validation modes, key setup, env vars, CI/CD); add validation_source/validation_warning to testcase.validate output table - docs/mcp-pilot-guide.md: add Scenario 8 (Quality Hub API validation); update Scenario 2 tip; expand Credential handling to cover pv_k_ key - docs/provar-mcp-public-docs.md: add Step 3 auth setup; update validate-a-test-case section with validation_source and mode note - docs/university-of-provar-mcp-course.md: add Lab 2.5 (auth login); expand Module 4 with validation modes table; add knowledge check Q4 - docs/auth-cli-plan.md: removed (internal planning doc) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
us-east-1qpfw was a misread — correct domain is us-east-1xpfwzwmop. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Cognito requires a nonce when using the openid scope (OIDC spec replay prevention). Also drops the profile scope which was not configured in the App Client, and corrects the scope to openid email only. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…aged Login Cognito Managed Login requires state (CSRF protection) and behaves more reliably at /login than /oauth2/authorize. Both state and nonce are now generated per-request. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
/login is just the UI page. /oauth2/authorize with state + nonce + PKCE is the correct OAuth endpoint and confirmed working in browser testing. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
cmd.exe interprets '&' in URLs as a command separator, so only the first query parameter was reaching the browser. PowerShell Start-Process passes the full URL as a single uninterpreted argument. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…saging Cognito's GetUser API requires the access token to carry the aws.cognito.signin.user.admin scope — without it the Lambda receives a valid JWT but GetUser returns NotAuthorizedException. Added to the scope parameter in the authorize URL. Also updated auth clear output to suggest sf provar auth login as the primary reconfiguration path and mention PROVAR_API_KEY for CI/CD. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The docs said "set PROVAR_API_KEY for CI/CD" but never explained how to get the value or that the key expires. Added the full workflow: run sf provar auth login locally, copy api_key from credentials.json, store as pipeline secret, rotate every ~90 days. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces the stub with a real POST /validate call using x-provar-key for user auth. /validate has no infra-gate x-api-key requirement. Removed getInfraKey() — dead code in the CLI. The batch validator that requires the infra key is in the managed package, not here. 401 always uses our own message (never the API's) to avoid surfacing sf provar auth set-key which does not exist. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
No AWS route backs this command and keys can only be obtained via sf provar auth login. set-key was clutter in --help. Removed: src command, messages file, unit tests, NUT file. Updated: status.ts, testCaseValidate.ts, README, mcp.md, mcp-pilot-guide.md all point to sf provar auth login instead. clear/status NUTs now seed credentials.json directly rather than depending on set-key as a test fixture. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements /auth/rotate endpoint: atomically replaces the stored pv_k_ key with a new one without going through the browser login flow. Old key is invalidated immediately on success. - src/commands/provar/auth/rotate.ts — new SfProvarAuthRotate command - messages/sf.provar.auth.rotate.md — summary, description, examples - src/services/qualityHub/client.ts — rotateKey() function + indirection entry - test/unit/commands/provar/auth/rotate.test.ts — 5 unit tests (599 total) - README.md, docs/mcp.md — rotate command documentation Root cause of test ERROR: null debugged and fixed — ts-node/esm surfaces noUnusedLocals TS6133 as a null-prototype error when a module-level sinon stub variable is declared but never read (sinon.restore() cleans up without referencing it). Fixed by inlining stubs inside each it() block. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Validate OAuth state parameter in listenForCallback (CSRF protection) - Fix PowerShell URL injection in openBrowser by passing URL via $args[0] - Fix status command to fall through to stored credentials on invalid env var - Apply username from live fetchKeyStatus response in status command - Persist username/tier/expires_at from login and rotate exchange responses - Fix httpsRequest to respect URL port and add 30s request timeout - Fix docs: QH URL default and validation_warning scope Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add REQUEST_ACCESS_URL constant and display it at every auth dead-end: - sf provar auth login: catch 401 from exchange and show request URL - sf provar auth status: "no key configured" block includes request URL - MCP ONBOARDING_MESSAGE and AUTH_WARNING include request URL - QualityHubAuthError from /auth/exchange includes request URL - docs/mcp.md: "Don't have an account?" in Authentication section - README.md: "Get Access" badge + inline link in MCP section - messages/sf.provar.auth.login.md and status.md updated Note: provar-mcp-public-docs.md and university-of-provar-mcp-course.md are maintained separately — flag for manual update. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Change install command to sf plugins install @provartesting/provardx-cli@beta across README.md and docs/mcp-pilot-guide.md - Add 5 NitroX (Hybrid Model) tools to the TOOLS EXPOSED list in README: provar.nitrox.discover, read, validate, generate, patch (present since beta.2, missing from docs) Note: provar-mcp-public-docs.md and university-of-provar-mcp-course.md are maintained separately — flag for manual update of install tag. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Node 25 removed SlowBuffer from the buffer module, crashing the transitive dependency buffer-equal-constant-time (via jsonwebtoken). This breaks sf provar auth *, lint, and tests. - package.json engines: cap at <25.0.0 - README: Node version note in Installation section - docs/mcp.md: add Prerequisites section with Node requirement - docs/mcp-pilot-guide.md: update Node row to 18-24 with warning Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add Quick start sections to both README and docs/mcp.md with numbered steps and a provardx.ping verify step - Fix Claude Code section: replace non-existent /mcp add slash command and .claude/mcp.json path with correct `claude mcp add -s user|project|local` commands and real config file locations (.mcp.json, settings.local.json) - Move license requirement before client configuration in docs/mcp.md since it is a startup blocker - Add Windows note: use sf.cmd in Claude Desktop when sf is not found Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ub-api Feature/auth and quality hub api
Contributor
There was a problem hiding this comment.
Pull request overview
This PR expands the Provar DX CLI’s MCP capabilities by adding Quality Hub API integration (including auth flows and credential persistence) and introducing a new NitroX (Hybrid Model) toolset, with accompanying tests and documentation updates.
Changes:
- Add Quality Hub API client + local/remote validation routing, plus new
sf provar authcommands and credential storage. - Introduce NitroX MCP tools (
discover/read/validate/generate/patch) and extend smoke tests/docs accordingly. - Improve tool UX and test coverage (e.g., metadata download suggestions, failure detection tests, path-policy test adjustments).
Reviewed changes
Copilot reviewed 40 out of 43 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
yarn.lock |
Dependency lock updates for new/updated packages. |
package.json |
Version bump, Node engine constraint, new auth topic, and NUT glob updates. |
src/services/qualityHub/client.ts |
Adds typed Quality Hub HTTPS client, normalization, and auth endpoints. |
src/services/auth/loginFlow.ts |
Implements PKCE login flow helpers and localhost callback listener. |
src/services/auth/credentials.ts |
Adds credential read/write/clear and API key resolution logic. |
src/mcp/tools/testCaseValidate.ts |
Routes validation to Quality Hub API when key exists; adds validation source/warnings. |
src/mcp/tools/nitroXTools.ts |
Adds NitroX discover/read/validate/generate/patch MCP tools and validation rules. |
src/mcp/tools/automationTools.ts |
Enhances metadata download tool description and adds [DOWNLOAD_ERROR] suggestion details. |
src/mcp/tools/propertiesTools.ts |
Clarifies testprojectSecrets as a key string (not a path). |
src/mcp/tools/antTools.ts |
Clarifies secrets password semantics in schema description. |
src/mcp/server.ts |
Registers NitroX tool suite in the MCP server. |
src/commands/provar/auth/login.ts |
New browser-based auth login command using PKCE + token exchange. |
src/commands/provar/auth/status.ts |
New status command reporting key source and best-effort server validity check. |
src/commands/provar/auth/rotate.ts |
New rotate command to replace stored API key via server-side rotation. |
src/commands/provar/auth/clear.ts |
New clear command to revoke (best-effort) and remove local credentials. |
messages/sf.provar.auth.status.md |
User-facing help text for auth status. |
messages/sf.provar.auth.rotate.md |
User-facing help text for auth rotate. |
messages/sf.provar.auth.login.md |
User-facing help text for auth login. |
messages/sf.provar.auth.clear.md |
User-facing help text for auth clear. |
docs/mcp.md |
Adds prerequisites/auth guidance and full NitroX tool documentation. |
docs/mcp-pilot-guide.md |
Updates pilot scenarios and security notes; includes NitroX + validation_source guidance. |
docs/provar-mcp-public-docs.md |
Adds public-facing MCP documentation for installation and use cases. |
docs/university-of-provar-mcp-course.md |
Adds a full course document covering MCP usage end-to-end. |
README.md |
Updates install instructions, adds auth command docs, NitroX tool list, and access link badge. |
CLAUDE.md |
Adds repository contribution/testing/documentation requirements for Claude Code usage. |
.gitignore |
Updates ignore rules for Claude artifacts and schema placeholders. |
.github/workflows/CI_Execution.yml |
Ensures NUT report artifacts upload even on failure. |
scripts/mcp-smoke.cjs |
Extends smoke suite with NitroX tools and updates expected call count. |
test/unit/services/qualityHub/client.test.ts |
Adds unit tests for normaliseApiResponse mapping behavior. |
test/unit/services/auth/credentials.test.ts |
Adds unit tests for credential persistence and env/file resolution. |
test/unit/mcp/testCaseValidate.test.ts |
Extends validation tests to cover handler behavior and API fallback paths. |
test/unit/mcp/qualityHubTools.test.ts |
Adds failure-detection tests for testrun report suggestion behavior. |
test/unit/mcp/pathPolicy.test.ts |
Minor refactor for readability in allowed-dir variable usage. |
test/unit/mcp/automationTools.test.ts |
Adds tests for [DOWNLOAD_ERROR] suggestion details and traversal-string behavior. |
test/unit/mcp/antTools.test.ts |
Adjusts path-policy test setup and adds stricter path-policy coverage. |
test/unit/mcp/nitroXTools.test.ts |
Adds comprehensive NitroX tool tests (discover/read/validate/generate/patch). |
test/unit/commands/provar/auth/status.test.ts |
Adds unit tests for status source detection logic. |
test/unit/commands/provar/auth/rotate.test.ts |
Adds rotate-related tests around stored credentials behavior. |
test/unit/commands/provar/auth/login.test.ts |
Adds unit tests for PKCE and callback handling plus credential write assertions. |
test/unit/commands/provar/auth/clear.test.ts |
Adds unit tests for local credential clearing behavior. |
test/commands/provar/auth/status.nut.ts |
Adds NUT coverage for auth status output and key-source behaviors. |
test/commands/provar/auth/clear.nut.ts |
Adds NUT coverage for auth clear idempotency and interaction with status. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.