Skip to content

Point skill-evals at skillscope's first-class commands - #4

Merged
danielholanda merged 1 commit into
mainfrom
cursor/skillscope-first-class-commands-3678
Aug 29, 2026
Merged

Point skill-evals at skillscope's first-class commands#4
danielholanda merged 1 commit into
mainfrom
cursor/skillscope-first-class-commands-3678

Conversation

@danielholanda

@danielholanda danielholanda commented Aug 29, 2026

Copy link
Copy Markdown
Owner

Pull Request Template

Note to AMDers:
This is a public repository. Please do not upload any confidential or customer data. Make sure all such data has been anonymized or removed before making this PR. If you need to attach any private files or links, please insert a Internal OneDrive Link or a Jira Ticket Link instead.

Updates .github/workflows/skill-evals.yml for the recent danielholanda/skillscope changes (#2#4).

The three graders are commands now, not modes of run. The workflow stays the same three jobs, one checkout-plus-action each:

Job Was Now
structural command: structural unchanged
routing command: run + --mode routing --routing-skills analysis-orchestrator command: routing
behavioral command: run + --mode behavior command: behavioral

--routing-skills is gone because this repo has one skill: that skill is the room. The behavior job is renamed behavioral to match the command.

No dataset or skill-body change. evals.json already matches the current schema.

Verification

Locally, against danielholanda/skillscope@main:

$ skillscope run
error: argument command: invalid choice: 'run'
        (choose from 'structural', 'routing', 'behavioral', ...)

$ skillscope structural --skills TraceLens/Agent/Analysis/skills/analysis-orchestrator
[evals] OK: 1 skill folder(s) in shape.
[evals] OK: 12 case(s) across 1 skill(s) plus 5 shared negative(s).
[evals] OK: 56 internal reference(s) across 17 markdown file(s).

$ skillscope select --all --skills TraceLens/Agent/Analysis/skills/analysis-orchestrator
{"routing": true, "default": [{"skill": "analysis-orchestrator", ...}]}

CI on this PR:

  • structural passed
  • routing and behavioral launched the new commands (skillscope routing / skillscope behavioral) and then failed at the Claude preflight with Not logged in. Same failure as Run the analysis-orchestrator evals with skillscope #3: ANTHROPIC_API_KEY is not a repository secret, so those two jobs cannot grade yet.
Open in Web Open in Cursor 

Routing and behavioral are commands now, not modes of `run`. A single-skill
repo also no longer has to name its routing set, so the args lines go away.

Co-authored-by: Daniel Holanda <holand.daniel@gmail.com>
@danielholanda
danielholanda marked this pull request as ready for review August 29, 2026 22:36
@danielholanda
danielholanda merged commit 3bf24f1 into main Aug 29, 2026
3 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants