Interrupted mirror loses a file when a repaired cache meets an unusable 206 resume (#581)#634
Merged
Merged
Conversation
…-whole path (#581) An interrupted mirror can leave a truncated new.zip that the next pass must repair before resuming. When that repaired-cache resume then meets an unusable 206 (Content-Range not matching disk), the engine's restart-whole path must still drop the partial and refetch the whole file. Test 48 covers the resume + hostile 206 but always over a clean cache, so it never exercises the repair interaction that #581 is about. Test 71 damages new.zip between passes -- truncating past the last local entry so the central directory a hard kill never wrote is gone, unzOpen fails, and the repair path runs -- then asserts both that repair actually fired and that the file comes back whole. On Linux this recovers regardless of the damage; it carries the same Windows skip as test 48, and lifting that skip reproduces the Windows-only failure once the engine fix lands. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
Resolve tests/Makefile.am: keep 66_engine-port80-strip (#632) alongside this branch's test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
Picks up the minizip READ_32 fix (#640) so the sanitize leg no longer aborts on the repair path. The Windows offline runner pins its expected skip set exactly, so a second Windows-skipped test (71_local-crange-repaircache) must be listed there too, otherwise it fails with "unexpected skips". Resolve tests/Makefile.am: keep 66/67/69 (merged) alongside this branch's 71. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
Resolve tests/Makefile.am: keep 66/67/68/69 (merged) alongside this branch's 71. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#581 reports a Windows-only data loss: an interrupted mirror that resumes into an unusable Content-Range 206 drops its partial and never refetches, where the same sequence recovers the file whole on Linux. The trigger is the damaged cache a hard TerminateProcess leaves behind (MSYS can't signal a native exe), which pass 2 has to repair before resuming.
This does not fix the engine. I couldn't reproduce the failure on Linux, so an engine change would be guesswork. What it adds is a deterministic test that drives the damaged-cache regime, plus an analysis of where the two platforms part.
Test 71 leaves a partial and a temp-ref in pass 1, truncates
new.zippast its last local entry so the central directory a hard kill never wrote is gone and the repair path runs, then resumes into the hostile 206. On Linux that fires the repair, takes the "unusable range -> restart whole" branch, and recovers the file whole every time, across every damage severity I tried, including a repair that recovers zero entries or fails outright. So the cache repair and the restart-whole logic are not themselves where Linux and Windows differ.Root cause, as far as I can pin it from Linux: restart-whole doesn't refetch. It removes the partial and the temp-ref, flags
STATUSCODE_NON_FATAL, and leans on the ordinary retry to requeue the URL. The requeued attempt rebuilds the request from the cache, then the temp-ref, then the on-disk partial. On Linux the removals leave none of those, so the retry is a clean whole-file GET and it succeeds. For the file to be lost, the retried attempt has to send a Range again: a second unusable 206, a second -5, and once the retry budget is spent the partial is already gone. That second Range can only come from a temp-ref or partial that outlived the restart-whole removal, which points at a Windows-specific removal or path effect I can't confirm from here.For the maintainer: the fragile hinge is that restart-whole depends on a budget-consuming retry that re-derives its Range state from disk. A sturdier fix would make the retried attempt refuse to resume, via a per-link "refetch whole, no Range" flag the request builder honors, so a leftover temp-ref or partial can't re-enter the 206 loop whatever the removal quirk turns out to be. Checking that
UNLINKandurl_savename_refname_removeactually succeed on Windows would confirm the mechanism first.Test 71 carries the same Windows skip as test 48. Lifting that skip should reproduce the failure, and turn the test into the fix's verification.
Refs #581.