Skip to content

ENG-3201 Fix eval auto-upload for metadata-linked environments#480

Open
d42me wants to merge 14 commits intomainfrom
fix/eval-auto-upload-metadata-linked-envs
Open

ENG-3201 Fix eval auto-upload for metadata-linked environments#480
d42me wants to merge 14 commits intomainfrom
fix/eval-auto-upload-metadata-linked-envs

Conversation

@d42me
Copy link
Copy Markdown
Contributor

@d42me d42me commented Mar 27, 2026

Summary

  • reuse the env push content hash implementation when comparing local envs against the platform
  • stop falsely marking metadata-linked local environments as ahead when their content matches the upstream
  • add regression coverage for hash parity and in-sync metadata-linked env resolution

Testing

  • pytest packages/prime/tests/test_verifiers_bridge.py packages/prime/tests/test_hosted_eval.py packages/prime/tests/test_eval_billing.py -q

Note

Medium Risk
Changes environment version/hash resolution and makes prime eval auto-upload fail fast when the local environment isn’t published or can’t be linked, which can alter user workflows. Risk is moderate due to reliance on API response shapes and new error/exit paths.

Overview
Fixes environment/version handling so metadata-linked local environments are no longer incorrectly marked as “ahead” when their content matches the hub, by reusing prime env push’s compute_content_hash and enriching @latest env details with content_hash from the /versions endpoint when missing.

Updates prime env status to display latest version/hash via a shared extract_env_version_summary helper, and changes prime eval passthrough to refuse pushing results when the local env diverges/unpublished (and improves messaging when no upstream is found). Adds regression tests for hash parity, remote detail enrichment, in-sync resolution, and new eval failure behavior, plus a retry loop in the sandbox command timeout test for transient 502s.

Written by Cursor Bugbot for commit 62e061e. This will update automatically on new commits. Configure here.

@d42me d42me changed the title Fix eval auto-upload for metadata-linked environments ENG-3201 Fix eval auto-upload for metadata-linked environments Mar 27, 2026
@d42me d42me requested a review from eligotts March 27, 2026 21:49
Comment thread packages/prime/src/prime_cli/verifiers_bridge.py Outdated
Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Comment thread packages/prime/src/prime_cli/verifiers_bridge.py
eligotts
eligotts previously approved these changes Mar 27, 2026
@d42me d42me requested review from JannikSt, burnpiro and eligotts March 28, 2026 01:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants