migrate: repair pr_diff tasks baked before the #145 oracle-isolation fix - #157
Open
KNambiarDJsc wants to merge 1 commit into
Open
KNambiarDJsc wants to merge 1 commit into
KNambiarDJsc wants to merge 1 commit into
Conversation
…-isolation fix huggingface#145 stopped baking the oracle/verifier/instruction into pr_diff's agent image, but already-published tasks (e.g. AdithyaSK/repo2rlenv-pr-diff, 181 tasks) keep the pre-fix Dockerfile until re-emitted — huggingface#155. repo2rlenv migrate pr-diff <dir> [--apply] detects the baked-oracle marker and rewrites environment/Dockerfile + tests/{test.sh,verifier.py, oracle.patch,instruction.md} via the SAME builder functions fresh generate calls today, so a migrated task is byte-identical to one emitted now, not a hand-maintained parallel implementation. instruction.md and solution/patch.diff (the oracle itself) are never touched, so a migrated task keeps its original content_hash. repo_url/base_commit are recovered from the existing Dockerfile's remote set-url line and task.toml's metadata.repo2env.ref, not re-derived or guessed. Refuses to touch a task whose instruction.md/solution/patch.diff don't hash to the content_hash task.toml already claims, rather than silently rewriting an inconsistent bundle. Defaults to a dry-run report; --apply writes.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes the tooling gap #145 explicitly left open: "Existing published datasets... need re-emission to pick up the fix. A migration/re-bake tool could be a follow-up." — this is that follow-up, for #155.
repo2rlenv migrate pr-diff <dir> [--apply]walks a dataset directory, detects the pre-#145 baked-oracle Dockerfile (base64 -d > /verifier/oracle.patch), and repairs it:environment/Dockerfile,tests/test.sh, and addstests/{verifier.py,oracle.patch,instruction.md}using the exact same builder functionsgenerateuses today (build_pr_diff_environment_dockerfile,build_pr_diff_eval_script,_pr_diff_aux_files) — a migrated task is byte-identical to one freshly emitted, not a hand-maintained parallel implementation that could drift from the real fix.instruction.mdandsolution/patch.diff(the oracle itself) are never touched, socontent_hash(which only ever covers those two files —emitter/harbor.py:_content_hash) is unchanged; a migrated task keeps its original identity.repo_url/base_commitare recovered from the existing Dockerfile'sremote set-url originline andtask.toml'smetadata.repo2env.ref— both untouched by the leak fix, so nothing is guessed.instruction.md/solution/patch.diffdon't hash to thecontent_hashitstask.tomlalready claims, rather than silently rewriting something inconsistent — surfaced asaction: "error".--applywrites. Idempotent — a task already on the fixed shape reportsalready_safeand is untouched.Scope note
Per #155's checklist, this PR covers detection + file repair only:
AdithyaSK/repo2rlenv-pr-diff. Those are exactly the steps this tool's output is meant to feed into.generateproduces today, whose end-to-end absence-of-oracle was already verified in pr_diff: ship oracle + verifier via tests/, not baked in image #145's own test plan, but I haven't independently re-verified it against a migrated (as opposed to freshly generated) task.pr_diffdatasets and found onlyAdithyaSK/repo2rlenv-pr-diff(181 tasks). If there are other copies/forks on the Hub, I don't have visibility into that from the repo alone.Test plan
uv run --all-extras pytest tests/test_pipeline_pr_diff.py -q— 36 passed (9 new: detects+repairs, preservescontent_hash, idempotent re-run, dry-run writes nothing, skips non-pr_diff tasks, already-safe task is a no-op, flagscontent_hashmismatch instead of rewriting, missing-task.tomlerror)uv run --all-extras pytest -q(full suite) — 2003 passed, 0 failed, 6 skipped (all live-network/Docker-gated)ruff check ./ruff format --check .— cleanmigrate pr-diffdry-run → apply → re-apply (idempotent) →validate --deepon the result, all as expected