Conversation
Every --write-midx rewrites the index whole: a cold read of every pack's .idx and a write of about the same size. On mirrors with millions of objects that costs seconds per job on the sticky disk, more than the roll-up itself, while lookups across the handful of packs the geometric repack leaves are fast without it. Remove any index an earlier version left behind instead of letting it go stale. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Contributor
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
piob-io
marked this pull request as ready for review
September 18, 2026 23:37
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Since v1.8.0 the post step runs
git repack -d -l -n --geometric=2 --write-midxon every job. On large mirrors (a few GiB, millions of objects) the fleet shows a bimodal post step: most runs finish maintenance in well under a second, but a recurring ~25% take 6–16 s inside thatrepackcommand with the same inputs (same two kept packs, nothing deferred, one small incoming pack). The roll-up itself is tiny in those runs; what scales with the mirror is--write-midx: git rewrites the multi-pack-index whole each time, reading every pack's.idx(tens of MiB per kept pack) and writing about as many bytes again, and on a freshly mounted sticky disk those reads are cold.Local measurement on a 2.8M-object / 1.3 GiB two-kept-pack fixture with one small incoming pack, page cache dropped before each run:
--geometric=2 --write-midx0.55–0.59 s,--geometric=2alone ~0.01 s; same ratio on a 1M-object fixture. The end-to-end confirmation on a real staging sticky disk is the steady-state legs added to thecheckout-mirror-maintenance-e2eworkflow in test-workflows (baselinemainvs this branch on the same seeded mirror).Change (
runMirrorMaintenance):Lookups across the handful of packs the geometric repack leaves (kept base packs + one or two roll-ups) are fast without a midx, and bitmaps were already off. The index a v1.8.0 run left on existing mirrors is removed up front rather than left to go stale;
git repack -dwould drop it itself on the first run that folds something, but not on a no-op run.Everything else is unchanged:
.keepselection and byte bound, the 14-day reclaim, the fail-open commit, stale-file cleanup.Tests: existing real-git tests now assert no midx is present after incremental and reclaim runs; new test seeds a midx (as an earlier version would have) and checks it is removed and every pack it listed is still rolled up, fsck clean.
cleanuptest asserts--write-midxis not passed.Link to Devin session: https://app.devin.ai/sessions/a3b42ef15226438faf3f9cdde9662918
Open in Devin Desktop: https://app.devin.ai/desktop/session/a3b42ef15226438faf3f9cdde9662918?variant=devin
Requested by: @piob-io
View with [code]smith Autofix with [code]smith
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled. (Staging)Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.