Skip to content

[AzSDK Agent] E2E Validation: JS/TS — Full workflow with new shared skills #15057

@ronniegeraghty

Description

@ronniegeraghty

Objective

End-to-end validation of all new shared Copilot skills in the azure-sdk-for-js repo. Test the complete inner loop workflow from TypeSpec to release.

Skills Under Test

Skill Stage Key Action
generate-sdk-locally Generate → Build → Test Generate JS/TS SDK from TypeSpec, build, run tests
apiview-feedback-resolution APIView Feedback Resolve APIView comments on a JS/TS SDK PR
pipeline-troubleshooting Pipeline/CI Diagnose a JS/TS CI pipeline failure
prepare-release-plan Release Planning Create and link a release plan for JS/TS package
sdk-release Release Check readiness and trigger JS/TS SDK release

E2E Flow to Test

  1. Start with a TypeSpec spec for a service
  2. Ask the agent to generate the JS/TS SDK → generate-sdk-locally
  3. Build and test → generate-sdk-locally (build/test stages)
  4. Submit for API review → check APIView feedback → apiview-feedback-resolution
  5. If pipeline fails → pipeline-troubleshooting
  6. Create release plan → prepare-release-plan
  7. Check release readiness → sdk-release

Key Validation: "What's Next?" Flow

Critical test: At each stage transition, ask "What's next?" and verify the agent properly leads to the next logical step in the workflow.

  • Does the agent connect the skills into a coherent workflow?
  • Does it know which skill/stage comes next?
  • Does it provide JS/TS-specific guidance (npm, rush, vitest, etc.)?

Environment

  • Repo: Azure/azure-sdk-for-js
  • Prerequisites: Node.js 18+, npm/rush, azure-sdk-mcp server, local clone
  • Language-specific tools: npm run build, npm test, rush build, npm registry

Metadata

Metadata

Assignees

No one assigned

    Labels

    Scenario 2Used for tracking work related to the Dev Inner Loop group's Scenario 2 spec.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions