Skip to content

Interrupted mirror loses a file when a repaired cache meets an unusable 206 resume (#581)#634

Merged
xroche merged 4 commits into
masterfrom
fix-581-win-crange-restart
Jul 18, 2026
Merged

Interrupted mirror loses a file when a repaired cache meets an unusable 206 resume (#581)#634
xroche merged 4 commits into
masterfrom
fix-581-win-crange-restart

Conversation

@xroche

@xroche xroche commented Jul 18, 2026

Copy link
Copy Markdown
Owner

#581 reports a Windows-only data loss: an interrupted mirror that resumes into an unusable Content-Range 206 drops its partial and never refetches, where the same sequence recovers the file whole on Linux. The trigger is the damaged cache a hard TerminateProcess leaves behind (MSYS can't signal a native exe), which pass 2 has to repair before resuming.

This does not fix the engine. I couldn't reproduce the failure on Linux, so an engine change would be guesswork. What it adds is a deterministic test that drives the damaged-cache regime, plus an analysis of where the two platforms part.

Test 71 leaves a partial and a temp-ref in pass 1, truncates new.zip past its last local entry so the central directory a hard kill never wrote is gone and the repair path runs, then resumes into the hostile 206. On Linux that fires the repair, takes the "unusable range -> restart whole" branch, and recovers the file whole every time, across every damage severity I tried, including a repair that recovers zero entries or fails outright. So the cache repair and the restart-whole logic are not themselves where Linux and Windows differ.

Root cause, as far as I can pin it from Linux: restart-whole doesn't refetch. It removes the partial and the temp-ref, flags STATUSCODE_NON_FATAL, and leans on the ordinary retry to requeue the URL. The requeued attempt rebuilds the request from the cache, then the temp-ref, then the on-disk partial. On Linux the removals leave none of those, so the retry is a clean whole-file GET and it succeeds. For the file to be lost, the retried attempt has to send a Range again: a second unusable 206, a second -5, and once the retry budget is spent the partial is already gone. That second Range can only come from a temp-ref or partial that outlived the restart-whole removal, which points at a Windows-specific removal or path effect I can't confirm from here.

For the maintainer: the fragile hinge is that restart-whole depends on a budget-consuming retry that re-derives its Range state from disk. A sturdier fix would make the retried attempt refuse to resume, via a per-link "refetch whole, no Range" flag the request builder honors, so a leftover temp-ref or partial can't re-enter the 206 loop whatever the removal quirk turns out to be. Checking that UNLINK and url_savename_refname_remove actually succeed on Windows would confirm the mechanism first.

Test 71 carries the same Windows skip as test 48. Lifting that skip should reproduce the failure, and turn the test into the fix's verification.

Refs #581.

xroche and others added 2 commits July 18, 2026 07:26
…-whole path (#581)

An interrupted mirror can leave a truncated new.zip that the next pass must
repair before resuming. When that repaired-cache resume then meets an unusable
206 (Content-Range not matching disk), the engine's restart-whole path must
still drop the partial and refetch the whole file. Test 48 covers the resume +
hostile 206 but always over a clean cache, so it never exercises the repair
interaction that #581 is about.

Test 71 damages new.zip between passes -- truncating past the last local entry
so the central directory a hard kill never wrote is gone, unzOpen fails, and the
repair path runs -- then asserts both that repair actually fired and that the
file comes back whole. On Linux this recovers regardless of the damage; it
carries the same Windows skip as test 48, and lifting that skip reproduces the
Windows-only failure once the engine fix lands.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
Resolve tests/Makefile.am: keep 66_engine-port80-strip (#632) alongside this
branch's test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
xroche and others added 2 commits July 18, 2026 09:59
Picks up the minizip READ_32 fix (#640) so the sanitize leg no longer aborts on
the repair path. The Windows offline runner pins its expected skip set exactly,
so a second Windows-skipped test (71_local-crange-repaircache) must be listed
there too, otherwise it fails with "unexpected skips".

Resolve tests/Makefile.am: keep 66/67/69 (merged) alongside this branch's 71.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
Resolve tests/Makefile.am: keep 66/67/68/69 (merged) alongside this branch's 71.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
@xroche
xroche merged commit d267aaf into master Jul 18, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant