Skip to content

feat(update): add MergingSnapshotUpdate#1

Closed
gty404 wants to merge 10 commits into
mainfrom
merging-snapshot-update
Closed

feat(update): add MergingSnapshotUpdate#1
gty404 wants to merge 10 commits into
mainfrom
merging-snapshot-update

Conversation

@gty404
Copy link
Copy Markdown
Owner

@gty404 gty404 commented May 26, 2026

Summary

Implements MergingSnapshotUpdate — the abstract base class for merge-based snapshot operations (MergeAppend, OverwriteFiles, RowDelta, etc.), consistent with Java's MergingSnapshotProducer.

Core Implementation

  • Filter → write → merge pipeline in Apply()
  • Conflict detection helpers
  • AddDataFile, DeleteDataFile, AddDeleteFile, DeleteDeleteFile, AddManifest, AppendManifest primitive API

Supporting Changes

  • Add ReplacedManifestsCount() to ManifestFilterManager and ManifestMergeManager
  • Fix SnapshotSummaryBuilder: manifest count fields (manifests-created, manifests-kept, manifests-replaced) are now always emitted together — matching Java's SnapshotProducer.buildManifestCountSummary
  • Fix SnapshotUpdate::DeleteFile: removed static from capturing lambda (caused use-after-free heap corruption)

Tests

  • manifest_filter_manager_test.cc: 27 tests
  • merging_snapshot_update_test.cc: 34 tests aligned with Java's TestSnapshotSummary, TestMergeAppend, TestRowDelta

All 378 tests pass.

wgtmac and others added 10 commits May 12, 2026 15:18
Adds delete filter support for merge-on-read data batches, including
position deletes, equality deletes, dropped-field lookup through
historic schemas, optional delete counting, and required schema
expansion.
)

Bumps [github/codeql-action](https://github.com/github/codeql-action)
from 4.35.2 to 4.35.3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/github/codeql-action/releases">github/codeql-action's
releases</a>.</em></p>
<blockquote>
<h2>v4.35.3</h2>
<ul>
<li><em>Upcoming breaking change</em>: Add a deprecation warning for
customers using CodeQL version 2.19.3 and earlier. These versions of
CodeQL were discontinued on 9 April 2026 alongside GitHub Enterprise
Server 3.15, and will be unsupported by the next minor release of the
CodeQL Action. <a
href="https://redirect.github.com/github/codeql-action/pull/3837">#3837</a></li>
<li>Configurations for private registries that use Cloudsmith or GCP
OIDC are now accepted. <a
href="https://redirect.github.com/github/codeql-action/pull/3850">#3850</a></li>
<li>Best-effort connection tests for private registries now use
<code>GET</code> requests instead of <code>HEAD</code> for better
compatibility with various registry implementations. For NuGet feeds,
the test is now always performed against the service index. <a
href="https://redirect.github.com/github/codeql-action/pull/3853">#3853</a></li>
<li>Fixed a bug where two diagnostics produced within the same
millisecond could overwrite each other on disk, causing one of them to
be lost. <a
href="https://redirect.github.com/github/codeql-action/pull/3852">#3852</a></li>
<li>Update default CodeQL bundle version to <a
href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.3">2.25.3</a>.
<a
href="https://redirect.github.com/github/codeql-action/pull/3865">#3865</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/github/codeql-action/blob/main/CHANGELOG.md">github/codeql-action's
changelog</a>.</em></p>
<blockquote>
<h1>CodeQL Action Changelog</h1>
<p>See the <a
href="https://github.com/github/codeql-action/releases">releases
page</a> for the relevant changes to the CodeQL CLI and language
packs.</p>
<h2>[UNRELEASED]</h2>
<p>No user facing changes.</p>
<h2>4.35.4 - 07 May 2026</h2>
<ul>
<li>Update default CodeQL bundle version to <a
href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.4">2.25.4</a>.
<a
href="https://redirect.github.com/github/codeql-action/pull/3881">#3881</a></li>
</ul>
<h2>4.35.3 - 01 May 2026</h2>
<ul>
<li><em>Upcoming breaking change</em>: Add a deprecation warning for
customers using CodeQL version 2.19.3 and earlier. These versions of
CodeQL were discontinued on 9 April 2026 alongside GitHub Enterprise
Server 3.15, and will be unsupported by the next minor release of the
CodeQL Action. <a
href="https://redirect.github.com/github/codeql-action/pull/3837">#3837</a></li>
<li>Configurations for private registries that use Cloudsmith or GCP
OIDC are now accepted. <a
href="https://redirect.github.com/github/codeql-action/pull/3850">#3850</a></li>
<li>Best-effort connection tests for private registries now use
<code>GET</code> requests instead of <code>HEAD</code> for better
compatibility with various registry implementations. For NuGet feeds,
the test is now always performed against the service index. <a
href="https://redirect.github.com/github/codeql-action/pull/3853">#3853</a></li>
<li>Fixed a bug where two diagnostics produced within the same
millisecond could overwrite each other on disk, causing one of them to
be lost. <a
href="https://redirect.github.com/github/codeql-action/pull/3852">#3852</a></li>
<li>Update default CodeQL bundle version to <a
href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.3">2.25.3</a>.
<a
href="https://redirect.github.com/github/codeql-action/pull/3865">#3865</a></li>
</ul>
<h2>4.35.2 - 15 Apr 2026</h2>
<ul>
<li>The undocumented TRAP cache cleanup feature that could be enabled
using the <code>CODEQL_ACTION_CLEANUP_TRAP_CACHES</code> environment
variable is deprecated and will be removed in May 2026. If you are
affected by this, we recommend disabling TRAP caching by passing the
<code>trap-caching: false</code> input to the <code>init</code> Action.
<a
href="https://redirect.github.com/github/codeql-action/pull/3795">#3795</a></li>
<li>The Git version 2.36.0 requirement for improved incremental analysis
now only applies to repositories that contain submodules. <a
href="https://redirect.github.com/github/codeql-action/pull/3789">#3789</a></li>
<li>Python analysis on GHES no longer extracts the standard library,
relying instead on models of the standard library. This should result in
significantly faster extraction and analysis times, while the effect on
alerts should be minimal. <a
href="https://redirect.github.com/github/codeql-action/pull/3794">#3794</a></li>
<li>Fixed a bug in the validation of OIDC configurations for private
registries that was added in CodeQL Action 4.33.0 / 3.33.0. <a
href="https://redirect.github.com/github/codeql-action/pull/3807">#3807</a></li>
<li>Update default CodeQL bundle version to <a
href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.2">2.25.2</a>.
<a
href="https://redirect.github.com/github/codeql-action/pull/3823">#3823</a></li>
</ul>
<h2>4.35.1 - 27 Mar 2026</h2>
<ul>
<li>Fix incorrect minimum required Git version for <a
href="https://redirect.github.com/github/roadmap/issues/1158">improved
incremental analysis</a>: it should have been 2.36.0, not 2.11.0. <a
href="https://redirect.github.com/github/codeql-action/pull/3781">#3781</a></li>
</ul>
<h2>4.35.0 - 27 Mar 2026</h2>
<ul>
<li>Reduced the minimum Git version required for <a
href="https://redirect.github.com/github/roadmap/issues/1158">improved
incremental analysis</a> from 2.38.0 to 2.11.0. <a
href="https://redirect.github.com/github/codeql-action/pull/3767">#3767</a></li>
<li>Update default CodeQL bundle version to <a
href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.1">2.25.1</a>.
<a
href="https://redirect.github.com/github/codeql-action/pull/3773">#3773</a></li>
</ul>
<h2>4.34.1 - 20 Mar 2026</h2>
<ul>
<li>Downgrade default CodeQL bundle version to <a
href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.24.3">2.24.3</a>
due to issues with a small percentage of Actions and JavaScript
analyses. <a
href="https://redirect.github.com/github/codeql-action/pull/3762">#3762</a></li>
</ul>
<h2>4.34.0 - 20 Mar 2026</h2>
<ul>
<li>Added an experimental change which disables TRAP caching when <a
href="https://redirect.github.com/github/roadmap/issues/1158">improved
incremental analysis</a> is enabled, since improved incremental analysis
supersedes TRAP caching. This will improve performance and reduce
Actions cache usage. We expect to roll this change out to everyone in
March. <a
href="https://redirect.github.com/github/codeql-action/pull/3569">#3569</a></li>
<li>We are rolling out improved incremental analysis to C/C++ analyses
that use build mode <code>none</code>. We expect this rollout to be
complete by the end of April 2026. <a
href="https://redirect.github.com/github/codeql-action/pull/3584">#3584</a></li>
<li>Update default CodeQL bundle version to <a
href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.0">2.25.0</a>.
<a
href="https://redirect.github.com/github/codeql-action/pull/3585">#3585</a></li>
</ul>
<h2>4.33.0 - 16 Mar 2026</h2>
<ul>
<li>Upcoming change: Starting April 2026, the CodeQL Action will skip
collecting file coverage information on pull requests to improve
analysis performance. File coverage information will still be computed
on non-PR analyses. Pull request analyses will log a warning about this
upcoming change. <a
href="https://redirect.github.com/github/codeql-action/pull/3562">#3562</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/github/codeql-action/commit/e46ed2cbd01164d986452f91f178727624ae40d7"><code>e46ed2c</code></a>
Merge pull request <a
href="https://redirect.github.com/github/codeql-action/issues/3867">#3867</a>
from github/update-v4.35.3-8c6e48dbe</li>
<li><a
href="https://github.com/github/codeql-action/commit/b73d1d163446ca5e62b96698027210ab41df6a4a"><code>b73d1d1</code></a>
Add changelog entry for <a
href="https://redirect.github.com/github/codeql-action/issues/3853">#3853</a></li>
<li><a
href="https://github.com/github/codeql-action/commit/24e0bb00a931e2a5edb703ce3b22a70f3a3e800b"><code>24e0bb0</code></a>
Reorder changelog entries</li>
<li><a
href="https://github.com/github/codeql-action/commit/ec298daba71cf7592feacbd1c0887cddc0659f62"><code>ec298da</code></a>
Update changelog for v4.35.3</li>
<li><a
href="https://github.com/github/codeql-action/commit/8c6e48dbe051ceb3015c19554831af1b43275f46"><code>8c6e48d</code></a>
Merge pull request <a
href="https://redirect.github.com/github/codeql-action/issues/3865">#3865</a>
from github/update-bundle/codeql-bundle-v2.25.3</li>
<li><a
href="https://github.com/github/codeql-action/commit/719098349ea5beae8aa364bf9b71ff1c8d937df2"><code>7190983</code></a>
Add changelog note</li>
<li><a
href="https://github.com/github/codeql-action/commit/2bb209555a024d051f6271c8a846b402497f9445"><code>2bb2095</code></a>
Update default bundle to codeql-bundle-v2.25.3</li>
<li><a
href="https://github.com/github/codeql-action/commit/7851e55dc3be31ec4bcc3ef98453de2cb306e698"><code>7851e55</code></a>
Merge pull request <a
href="https://redirect.github.com/github/codeql-action/issues/3850">#3850</a>
from github/mbg/private-registry/cloudsmith-gcp</li>
<li><a
href="https://github.com/github/codeql-action/commit/262a15f6cf4c7a43d6a38ad76392e5e2d4977751"><code>262a15f</code></a>
Add generic non-printable chars test for OIDC configs</li>
<li><a
href="https://github.com/github/codeql-action/commit/a6109b1c07173a53ece3d179a925ff9644d1fabd"><code>a6109b1</code></a>
Merge pull request <a
href="https://redirect.github.com/github/codeql-action/issues/3853">#3853</a>
from github/mbg/start-proxy/improved-checks</li>
<li>Additional commits viewable in <a
href="https://github.com/github/codeql-action/compare/95e58e9a2cdfd71adc6e0353d5c52f41a045d225...e46ed2cbd01164d986452f91f178727624ae40d7">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github/codeql-action&package-manager=github_actions&previous-version=4.35.2&new-version=4.35.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
)

Bumps [github/codeql-action](https://github.com/github/codeql-action)
from 4.35.3 to 4.35.4.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/github/codeql-action/releases">github/codeql-action's
releases</a>.</em></p>
<blockquote>
<h2>v4.35.4</h2>
<ul>
<li>Update default CodeQL bundle version to <a
href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.4">2.25.4</a>.
<a
href="https://redirect.github.com/github/codeql-action/pull/3881">#3881</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/github/codeql-action/blob/main/CHANGELOG.md">github/codeql-action's
changelog</a>.</em></p>
<blockquote>
<h1>CodeQL Action Changelog</h1>
<p>See the <a
href="https://github.com/github/codeql-action/releases">releases
page</a> for the relevant changes to the CodeQL CLI and language
packs.</p>
<h2>[UNRELEASED]</h2>
<ul>
<li>Add support for SHA-256 Git object IDs. <a
href="https://redirect.github.com/github/codeql-action/pull/3893">#3893</a></li>
</ul>
<h2>4.35.5 - 15 May 2026</h2>
<ul>
<li>We have improved how the JavaScript bundles for the CodeQL Action
are generated to avoid duplication across bundles and reduce the size of
the repository by around 70%. This should have no effect on the runtime
behaviour of the CodeQL Action. <a
href="https://redirect.github.com/github/codeql-action/pull/3899">#3899</a></li>
<li>For performance and accuracy reasons, <a
href="https://redirect.github.com/github/roadmap/issues/1158">improved
incremental analysis</a> will now only be enabled on a pull request when
diff-informed analysis is also enabled for that run. If diff-informed
analysis is unavailable (for example, because the PR diff ranges could
not be computed), the action will fall back to a full analysis. <a
href="https://redirect.github.com/github/codeql-action/pull/3791">#3791</a></li>
<li>If multiple inputs are provided for the GitHub-internal
<code>analysis-kinds</code> input, only <code>code-scanning</code> will
be enabled. The <code>analysis-kinds</code> input is experimental, for
GitHub-internal use only, and may change without notice at any time. <a
href="https://redirect.github.com/github/codeql-action/pull/3892">#3892</a></li>
<li>Added an experimental change which, when running a Code Scanning
analysis for a PR with <a
href="https://redirect.github.com/github/roadmap/issues/1158">improved
incremental analysis</a> enabled, prefers CodeQL CLI versions that have
a cached overlay-base database for the configured languages. This speeds
up analysis for a repository when there is not yet a cached overlay-base
database for the latest CLI version. We expect to roll this change out
to everyone in May. <a
href="https://redirect.github.com/github/codeql-action/pull/3880">#3880</a></li>
</ul>
<h2>4.35.4 - 07 May 2026</h2>
<ul>
<li>Update default CodeQL bundle version to <a
href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.4">2.25.4</a>.
<a
href="https://redirect.github.com/github/codeql-action/pull/3881">#3881</a></li>
</ul>
<h2>4.35.3 - 01 May 2026</h2>
<ul>
<li><em>Upcoming breaking change</em>: Add a deprecation warning for
customers using CodeQL version 2.19.3 and earlier. These versions of
CodeQL were discontinued on 9 April 2026 alongside GitHub Enterprise
Server 3.15, and will be unsupported by the next minor release of the
CodeQL Action. <a
href="https://redirect.github.com/github/codeql-action/pull/3837">#3837</a></li>
<li>Configurations for private registries that use Cloudsmith or GCP
OIDC are now accepted. <a
href="https://redirect.github.com/github/codeql-action/pull/3850">#3850</a></li>
<li>Best-effort connection tests for private registries now use
<code>GET</code> requests instead of <code>HEAD</code> for better
compatibility with various registry implementations. For NuGet feeds,
the test is now always performed against the service index. <a
href="https://redirect.github.com/github/codeql-action/pull/3853">#3853</a></li>
<li>Fixed a bug where two diagnostics produced within the same
millisecond could overwrite each other on disk, causing one of them to
be lost. <a
href="https://redirect.github.com/github/codeql-action/pull/3852">#3852</a></li>
<li>Update default CodeQL bundle version to <a
href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.3">2.25.3</a>.
<a
href="https://redirect.github.com/github/codeql-action/pull/3865">#3865</a></li>
</ul>
<h2>4.35.2 - 15 Apr 2026</h2>
<ul>
<li>The undocumented TRAP cache cleanup feature that could be enabled
using the <code>CODEQL_ACTION_CLEANUP_TRAP_CACHES</code> environment
variable is deprecated and will be removed in May 2026. If you are
affected by this, we recommend disabling TRAP caching by passing the
<code>trap-caching: false</code> input to the <code>init</code> Action.
<a
href="https://redirect.github.com/github/codeql-action/pull/3795">#3795</a></li>
<li>The Git version 2.36.0 requirement for improved incremental analysis
now only applies to repositories that contain submodules. <a
href="https://redirect.github.com/github/codeql-action/pull/3789">#3789</a></li>
<li>Python analysis on GHES no longer extracts the standard library,
relying instead on models of the standard library. This should result in
significantly faster extraction and analysis times, while the effect on
alerts should be minimal. <a
href="https://redirect.github.com/github/codeql-action/pull/3794">#3794</a></li>
<li>Fixed a bug in the validation of OIDC configurations for private
registries that was added in CodeQL Action 4.33.0 / 3.33.0. <a
href="https://redirect.github.com/github/codeql-action/pull/3807">#3807</a></li>
<li>Update default CodeQL bundle version to <a
href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.2">2.25.2</a>.
<a
href="https://redirect.github.com/github/codeql-action/pull/3823">#3823</a></li>
</ul>
<h2>4.35.1 - 27 Mar 2026</h2>
<ul>
<li>Fix incorrect minimum required Git version for <a
href="https://redirect.github.com/github/roadmap/issues/1158">improved
incremental analysis</a>: it should have been 2.36.0, not 2.11.0. <a
href="https://redirect.github.com/github/codeql-action/pull/3781">#3781</a></li>
</ul>
<h2>4.35.0 - 27 Mar 2026</h2>
<ul>
<li>Reduced the minimum Git version required for <a
href="https://redirect.github.com/github/roadmap/issues/1158">improved
incremental analysis</a> from 2.38.0 to 2.11.0. <a
href="https://redirect.github.com/github/codeql-action/pull/3767">#3767</a></li>
<li>Update default CodeQL bundle version to <a
href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.1">2.25.1</a>.
<a
href="https://redirect.github.com/github/codeql-action/pull/3773">#3773</a></li>
</ul>
<h2>4.34.1 - 20 Mar 2026</h2>
<ul>
<li>Downgrade default CodeQL bundle version to <a
href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.24.3">2.24.3</a>
due to issues with a small percentage of Actions and JavaScript
analyses. <a
href="https://redirect.github.com/github/codeql-action/pull/3762">#3762</a></li>
</ul>
<h2>4.34.0 - 20 Mar 2026</h2>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/github/codeql-action/commit/68bde559dea0fdcac2102bfdf6230c5f70eb485e"><code>68bde55</code></a>
Merge pull request <a
href="https://redirect.github.com/github/codeql-action/issues/3885">#3885</a>
from github/update-v4.35.4-803d9e8c3</li>
<li><a
href="https://github.com/github/codeql-action/commit/9739ad2d182c072da0d01a6887f7f39620f71b1e"><code>9739ad2</code></a>
Update changelog for v4.35.4</li>
<li><a
href="https://github.com/github/codeql-action/commit/803d9e8c3ca8b0dd2029a1da3b541a18b6bfb076"><code>803d9e8</code></a>
Merge pull request <a
href="https://redirect.github.com/github/codeql-action/issues/3883">#3883</a>
from github/mbg/test/macro-wrapper</li>
<li><a
href="https://github.com/github/codeql-action/commit/0fd9c7d1358a7404e46ed8165f12262f56bd1434"><code>0fd9c7d</code></a>
Merge pull request <a
href="https://redirect.github.com/github/codeql-action/issues/3882">#3882</a>
from github/dependabot/github_actions/dot-github/wor...</li>
<li><a
href="https://github.com/github/codeql-action/commit/922d6fb888d665134eb982b150b8912dbd48e21a"><code>922d6fb</code></a>
Use <code>makeMacro</code> instead of <code>test.macro</code></li>
<li><a
href="https://github.com/github/codeql-action/commit/df77e87896689b5c736433984c5df14d86c63d56"><code>df77e87</code></a>
Update test macro snippet</li>
<li><a
href="https://github.com/github/codeql-action/commit/6e3f985e4fc409a188c7701b68c4dec158c9ced3"><code>6e3f985</code></a>
Add wrapper for <code>test.macro</code></li>
<li><a
href="https://github.com/github/codeql-action/commit/e7a347dfb1bfb7a858347623fcb4f650effca6b5"><code>e7a347d</code></a>
Merge pull request <a
href="https://redirect.github.com/github/codeql-action/issues/3881">#3881</a>
from github/update-bundle/codeql-bundle-v2.25.4</li>
<li><a
href="https://github.com/github/codeql-action/commit/17eabb2500031486a71e00ecbcb72c73804a6c9f"><code>17eabb2</code></a>
Rebuild</li>
<li><a
href="https://github.com/github/codeql-action/commit/aaef09c48db2dd7f0100363de1785963a34cd706"><code>aaef09c</code></a>
Bump ruby/setup-ruby</li>
<li>Additional commits viewable in <a
href="https://github.com/github/codeql-action/compare/e46ed2cbd01164d986452f91f178727624ae40d7...68bde559dea0fdcac2102bfdf6230c5f70eb485e">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github/codeql-action&package-manager=github_actions&previous-version=4.35.3&new-version=4.35.4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Update the Meson project version from 0.2.0 to 0.3.0 to match the CMake project version.
## Summary

- keep `UseRef()` snapshot IDs as `int64_t` while resolving refs
- add a regression test for a ref pointing at `INT32_MAX + 42`

## Root Cause

`TableScanBuilder::UseRef()` copied `SnapshotRef::snapshot_id` into an
`int32_t`, even though snapshot IDs are modeled as `int64_t` in refs,
metadata lookup, and scan context.

## Testing

- `uvx cmake --build build --target scan_test -j 8`
- `./build/src/iceberg/test/scan_test
--gtest_filter=*UseRefPreservesInt64SnapshotIds*`
- `./build/src/iceberg/test/scan_test`

Fixes apache#660
…pache#652)

Implement two manifest management classes for table write operations:

- ManifestFilterManager: filters manifest entries by row filter
expression, file path, or partition value; supports
FailMissingDeletePaths validation. Rewrites manifests that contain
matching files, marking entries as DELETED; passes through manifests
that cannot contain matching files unchanged.

- ManifestMergeManager: merges small manifests using greedy bin-packing,
grouping by partition_spec_id (manifests with different specs are never
merged). Oversized manifests pass through unchanged. ADDED entries from
prior manifests become EXISTING when merged (matching Java semantics).
…hots::Finalize (apache#648)

Mirrors Java's IncrementalFileCleanup for the linear-ancestry case: each
manifest is attributed to its writer snapshot, so two passes are enough
instead of the full reachability scan. Cherry-pick protection via
SnapshotSummaryFields::kSourceSnapshotId is preserved.

Finalize() now picks IncrementalFileCleanup when the expiration is
"simple" (no explicit snapshot IDs, no removed snapshots outside the
current main ancestry, and no retained snapshots outside the current
main ancestry), and falls back to ReachableFileCleanup otherwise. The
dispatch matches Java RemoveSnapshots.cleanExpiredSnapshots.

Two existing cleanup tests (DeletesExpiredFiles,
IgnoresExpiredDeleteManifestReadFailures) used an empty current manifest
list, which is an unreachable-orphan scenario that only
ReachableFileCleanup can resolve. They now call ExpireSnapshotId() to
force the reachable path, which keeps their original intent and matches
Java behavior. New tests cover both dispatch branches.

---------

Co-authored-by: shangxinli <shangxinli@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## Summary
- Check Parquet compression codec availability with Arrow before opening
the file writer.
- Return a clear `InvalidArgument` when a requested Parquet codec is not
built into Arrow.
- Add a regression test that exercises an unavailable codec in the
current Arrow build.

## Test Plan
- `cmake --build build --target parquet_test data_test && ctest
--test-dir build -R 'parquet_test|data_test' --output-on-failure`
- `git diff --check`
## Summary

Add a new `FileIO::DeleteFiles(...)` API as a bulk deletion entry point.

The default implementation deletes files sequentially by calling the
existing
`DeleteFile(...)` method and returns the first deletion error
encountered.

This PR only adds the API and backward-compatible fallback behavior. It
does not
yet update `ExpireSnapshots` to use `DeleteFiles(...)`, and it does not
introduce
parallel deletion.

Fixed: apache#658 

## Motivation

`ExpireSnapshots` and other cleanup flows may need to delete many files.
A
bulk deletion API gives FileIO implementations a common extension point
for
future optimized deletion strategies, such as storage-native batch
deletion or
parallel fallback deletion.
Abstract base for merge-based snapshot operations (MergeAppend,
OverwriteFiles, RowDelta, etc.), implementing the filter → write →
merge pipeline and conflict-detection helpers consistent with Java's
MergingSnapshotProducer.

Also adds ReplacedManifestsCount to ManifestFilterManager and
ManifestMergeManager, and SetManifestCounts to SnapshotSummaryBuilder.
@gty404 gty404 closed this May 26, 2026
@github-actions
Copy link
Copy Markdown

Cpp-Linter Report ⚠️

Some files did not pass the configured checks!

clang-tidy (v22.1.6) reports: 31 concern(s)
  • src/iceberg/test/manifest_filter_manager_test.cc:454:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "DropDeleteFilesOlderThan_KeepsNewerEntries" according to Googletest FAQ

      454 | TEST_F(ManifestFilterManagerTest, DropDeleteFilesOlderThan_KeepsNewerEntries) {
          |                                   ^
  • src/iceberg/test/manifest_filter_manager_test.cc:512:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "RemoveDanglingDeletesFor_FiltersDanglingDV" according to Googletest FAQ

      512 | TEST_F(ManifestFilterManagerTest, RemoveDanglingDeletesFor_FiltersDanglingDV) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:181:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "AddsDataFiles_InitiallyFalse" according to Googletest FAQ

      181 | TEST_F(MergingSnapshotUpdateTest, AddsDataFiles_InitiallyFalse) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:188:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "AddsDataFiles_TrueAfterAdd" according to Googletest FAQ

      188 | TEST_F(MergingSnapshotUpdateTest, AddsDataFiles_TrueAfterAdd) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:195:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "AddsDeleteFiles_TrueAfterAdd" according to Googletest FAQ

      195 | TEST_F(MergingSnapshotUpdateTest, AddsDeleteFiles_TrueAfterAdd) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:203:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "DeletesDataFiles_TrueAfterRegisterDelete" according to Googletest FAQ

      203 | TEST_F(MergingSnapshotUpdateTest, DeletesDataFiles_TrueAfterRegisterDelete) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:296:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "CleanUncommitted_ClearsCaches" according to Googletest FAQ

      296 | TEST_F(MergingSnapshotUpdateTest, CleanUncommitted_ClearsCaches) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:309:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "CommitDeleteFile_SummaryHasAddedDeleteFiles" according to Googletest FAQ

      309 | TEST_F(MergingSnapshotUpdateTest, CommitDeleteFile_SummaryHasAddedDeleteFiles) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:326:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "CommitDeletesDeleteFile_SummaryHasRemovedDeleteFiles" according to Googletest FAQ

      326 | TEST_F(MergingSnapshotUpdateTest, CommitDeletesDeleteFile_SummaryHasRemovedDeleteFiles) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:352:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "DuplicateDataFile_OnlyCountedOnce" according to Googletest FAQ

      352 | TEST_F(MergingSnapshotUpdateTest, DuplicateDataFile_OnlyCountedOnce) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:398:37: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "ValidateNewDeleteFile_V1Rejected" according to Googletest FAQ

      398 | TEST_F(MergingSnapshotUpdateV1Test, ValidateNewDeleteFile_V1Rejected) {
          |                                     ^
  • src/iceberg/test/merging_snapshot_update_test.cc:404:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "ValidateNewDeleteFile_V2RejectsDeletionVector" according to Googletest FAQ

      404 | TEST_F(MergingSnapshotUpdateTest, ValidateNewDeleteFile_V2RejectsDeletionVector) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:413:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "ValidateNewDeleteFile_V2AllowsEqualityDelete" according to Googletest FAQ

      413 | TEST_F(MergingSnapshotUpdateTest, ValidateNewDeleteFile_V2AllowsEqualityDelete) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:425:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "AddManifest_RejectsDeleteManifest" according to Googletest FAQ

      425 | TEST_F(MergingSnapshotUpdateTest, AddManifest_RejectsDeleteManifest) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:436:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "AddManifest_RejectsManifestWithExistingFiles" according to Googletest FAQ

      436 | TEST_F(MergingSnapshotUpdateTest, AddManifest_RejectsManifestWithExistingFiles) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:448:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "AddManifest_RejectsManifestWithDeletedFiles" according to Googletest FAQ

      448 | TEST_F(MergingSnapshotUpdateTest, AddManifest_RejectsManifestWithDeletedFiles) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:459:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "AddManifest_RejectsManifestWithAssignedSnapshotId" according to Googletest FAQ

      459 | TEST_F(MergingSnapshotUpdateTest, AddManifest_RejectsManifestWithAssignedSnapshotId) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:469:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "AddManifest_RejectsManifestWithFirstRowId" according to Googletest FAQ

      469 | TEST_F(MergingSnapshotUpdateTest, AddManifest_RejectsManifestWithFirstRowId) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:484:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "AppendManifest_EmptyTable" according to Googletest FAQ

      484 | TEST_F(MergingSnapshotUpdateTest, AppendManifest_EmptyTable) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:504:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "AppendManifest_WithDataFiles" according to Googletest FAQ

      504 | TEST_F(MergingSnapshotUpdateTest, AppendManifest_WithDataFiles) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:527:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "AppendManifest_MergeWithMinCountOne" according to Googletest FAQ

      527 | TEST_F(MergingSnapshotUpdateTest, AppendManifest_MergeWithMinCountOne) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:551:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "AppendManifest_DoNotMergeMinCount" according to Googletest FAQ

      551 | TEST_F(MergingSnapshotUpdateTest, AppendManifest_DoNotMergeMinCount) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:585:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "ManifestMerge_MergesIntoOne" according to Googletest FAQ

      585 | TEST_F(MergingSnapshotUpdateTest, ManifestMerge_MergesIntoOne) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:609:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "ManifestMerge_DoesNotMergeWhenBelowMinCount" according to Googletest FAQ

      609 | TEST_F(MergingSnapshotUpdateTest, ManifestMerge_DoesNotMergeWhenBelowMinCount) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:625:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "ManifestMerge_DoesNotMergeWhenSizeTargetTooSmall" according to Googletest FAQ

      625 | TEST_F(MergingSnapshotUpdateTest, ManifestMerge_DoesNotMergeWhenSizeTargetTooSmall) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:649:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "Summary_ManifestCountsOnFirstCommit" according to Googletest FAQ

      649 | TEST_F(MergingSnapshotUpdateTest, Summary_ManifestCountsOnFirstCommit) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:661:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "Summary_ManifestCountsOnSecondCommitNoMerge" according to Googletest FAQ

      661 | TEST_F(MergingSnapshotUpdateTest, Summary_ManifestCountsOnSecondCommitNoMerge) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:676:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "Summary_ManifestCountsAfterMerge" according to Googletest FAQ

      676 | TEST_F(MergingSnapshotUpdateTest, Summary_ManifestCountsAfterMerge) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:696:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "Summary_ManifestCountsAfterDelete" according to Googletest FAQ

      696 | TEST_F(MergingSnapshotUpdateTest, Summary_ManifestCountsAfterDelete) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:721:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "DataSpec_ThrowsWithMultipleSpecs" according to Googletest FAQ

      721 | TEST_F(MergingSnapshotUpdateTest, DataSpec_ThrowsWithMultipleSpecs) {
          |                                   ^
  • src/iceberg/test/merging_snapshot_update_test.cc:729:35: warning: [google-readability-avoid-underscore-in-googletest-name]

    avoid using "_" in test name "DataSpec_ThrowsWhenEmpty" according to Googletest FAQ

      729 | TEST_F(MergingSnapshotUpdateTest, DataSpec_ThrowsWhenEmpty) {
          |                                   ^

Have any feedback or feature suggestions? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants