Conversation
With the mirror attached as an alternate every object a filter would omit is already local, so the filter saves nothing. It does make the workspace a partial clone, and since git 2.48 index-pack --promisor repacks every locally available object linked from the received pack into a promisor pack: with a full mirror behind the alternates that copies the mirror's reachable history into the workspace on every filtered fetch that receives objects, costing time proportional to the mirror rather than to the delta. Drop the filter (explicit or the blob:none that sparse-checkout implies) whenever the workspace verifiably shares the mirror object store. As a side effect fetch-depth 0 with a filter now takes the direct ref-copy path instead of a full network fetch of all heads and tags. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Contributor
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
… mount A later checkout step in the same job mounts its own copy of the sticky disk over the same path, so objects that only the earlier mount's sync added are no longer visible through the earlier workspace's alternates. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…directory Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A
filter:input (or theblob:nonethatsparse-checkoutimplies) is now dropped whenever the workspace verifiably shares the mirror's object store (sharesMirrorObjects, the same gate the shallow negotiation uses). Without a mirror the filter is passed through unchanged.Why. With the mirror attached as an alternate, every object a filter would omit is already local, so the filter cannot save any transfer — but it does make the workspace a partial clone, and that is expensive with an alternate behind it. Since git 2.48,
fetch-packrunsindex-pack --promisorfor a filtered fetch, andindex-pack --promisorrepacks every locally available, non-promisor object linked from the received pack into a new promisor pack (repack_local_links→pack-objects --exclude-promisor-objects-best-effort). The fetched commit's parent lives in the mirror, so that repack walks the mirror's history reachable from it and writes it into the workspace's.git/objects. There is no config to turn this off.Reproduced on an 8.5 GB mirror with git 2.51 (trace2),
--filter=blob:none --depth=2of a merge commit whose parent is the mirror tip:In production this shows up as a flat ~60 s per filtered fetch that receives any objects, independent of depth or ref count (proportional to mirror size), plus several GB written to the runner's root disk per job; unfiltered fetches of the same shape take ~1 s. A filtered fetch that receives no objects (ref already in the mirror) is unaffected, which is why the cost is bimodal.
Change (
git-source-provider.ts):Side effect:
fetch-depth: 0+filter:now qualifies for the direct mirror ref copy (fetchRefsFromMirror, previously gated on!fetchOptions.filter), so it no longer does a+refs/heads/*network fetch either. The resulting workspace is a regular clone (noremote.origin.promisor/partialclonefilter); the only behavioural difference for later steps is that objects the mirror lacks are not lazily fetched — with a full mirror behind the alternates there are none, anddissociate: trueis unaffected.Tests
__test__/mirror-filter-fetch-git.test.ts(real git): unfiltered full and depth-2 fetches into an alternates workspace store ≤ 6 objects, write no.promisor, leave no partial-clone config, and check out the whole tree; ablob:nonefetch into the same workspace does produce a promisor pack, and on git ≥ 2.48 its object count exceeds the mirror's history. Passes on git 2.34 and 2.51.test-blacksmith.yml:fetch-depth: 0+filter: blob:noneand asparse-checkoutcheckout on the real mirror; asserts no promisor packs, no partial-clone config, < 1000 local objects, fsck clean, sparse cone applied.action.ymlfilterdescription updated.Link to Devin session: https://app.devin.ai/sessions/a3b42ef15226438faf3f9cdde9662918
Open in Devin Desktop: https://app.devin.ai/desktop/session/a3b42ef15226438faf3f9cdde9662918?variant=devin
Requested by: @piob-io
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled. (Staging)