Skip to content

feat: publish Judgeval JQL skill - #6

Merged
JJwilkin merged 5 commits into
mainfrom
joshua/publish-judgeval-jql-skill
Jul 24, 2026
Merged

feat: publish Judgeval JQL skill#6
JJwilkin merged 5 commits into
mainfrom
joshua/publish-judgeval-jql-skill

Conversation

@JJwilkin

@JJwilkin JJwilkin commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Summary

Publish the judgeval-jql skill for coding agents, with progressive Python and TypeScript references plus a single-fetch full.md bundle.

Source and provenance

  • JudgmentLabs/jql#74 is merged.
  • The published artifacts come from its merge commit: JudgmentLabs/jql@c92a144f7440814458c85c31573e2572cd95c888.
  • skills/judgeval-jql/source.json records that immutable commit and the SHA-256 hash of every copied artifact.
  • CI verifies all four published files against the recorded hashes.

Changes

  • add the generated skills/judgeval-jql/** artifact
  • publish runtime-neutral guidance with focused Python and TypeScript references
  • publish full.md with one document hierarchy and in-document runtime links
  • document installation and stable docs-hosted reference URLs
  • bump the Claude and Cursor plugin releases to 1.1.0; the marketplace catalog version remains independent

Validation

  • confirmed all four artifacts are byte-identical to the JQL #74 merge commit
  • node scripts/check-jql-skill-provenance.mjs
  • parsed all JSON manifests
  • installed with npx skills add ... --skill judgeval-jql --agent codex --copy -y
  • confirmed installed skill and provenance files remained byte-identical
  • upstream JQL generator, freshness, contract, and runtime tests pass

Remaining release dependency

Do not merge this PR until JudgmentLabs/judgeval#762 is merged and its production release workflow successfully publishes judgeval==1.3.0 to PyPI. That is the first public Python release containing judgeval.jql.

After 1.3.0 is available:

  1. Validate the installed Python examples against the released package.
  2. Get the required approval and merge this PR.
  3. Update JudgmentLabs/docs#456 to pin this PR's final publication commit and verify its preview endpoints.

@JJwilkin
JJwilkin marked this pull request as ready for review July 22, 2026 23:16

@SamGearou SamGearou left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The four skill files are byte-for-byte copies from JudgmentLabs/jql#74, so I left the content findings on that PR (notably: full.md opens with two H1s and instructs its reader to fetch the files it replaces; the TypeScript retry example doesn't compile to anything usable; the agg doc line drops half a validation rule). Comments here are about what's specific to this repo.

Comment thread skills/judgeval-jql/SKILL.md
Comment thread skills/judgeval-jql/full.md
Comment thread README.md Outdated
Comment thread .claude-plugin/marketplace.json Outdated
Comment thread README.md Outdated
@JJwilkin
JJwilkin merged commit 29f15c0 into main Jul 24, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants