Skip to content

NIFI-16344 Preserve Swap File Ordering during Selective Drop - #11680

Merged
bobpaulin merged 3 commits into
apache:mainfrom
markap14:codex/nifi-16344-preserve-selective-drop-ordering
Sep 15, 2026
Merged

bobpaulin merged 3 commits into
apache:mainfrom
markap14:codex/nifi-16344-preserve-selective-drop-ordering

Conversation

@markap14

Copy link
Copy Markdown
Contributor

Summary

NIFI-16344

Selective FlowFile drop rewrites swap files containing matching records. Rewritten files were appended after unchanged swap files, allowing newer FlowFiles to be polled before retained older FlowFiles. This change replaces rewritten swap locations in their original positions and adds a multiple-swap-file regression test that verifies FIFO polling order.

Tracking

Issue Tracking

  • Apache NiFi Jira issue created

Pull Request Tracking

  • Pull Request title starts with Apache NiFi Jira issue number
  • Pull Request commit message starts with Apache NiFi Jira issue number
  • Pull request contains a commit with Verified signature status

Pull Request Formatting

  • Pull Request based on current revision of the main branch
  • Pull Request refers to a feature branch with one commit containing changes

Verification

Build

Targeted reactor build and queue tests completed using JDK 21:

./mvnw -pl nifi-framework-bundle/nifi-framework/nifi-framework-core -am -Dtest=org.apache.nifi.controller.queue.clustered.TestSwappablePriorityQueue,org.apache.nifi.controller.TestStandardFlowFileQueue,org.apache.nifi.controller.queue.clustered.TestSocketLoadBalancedFlowFileQueue -Dsurefire.failIfNoSpecifiedTests=false test

Tests run: 73, Failures: 0, Errors: 0, Skipped: 0.

Licensing

  • No new dependencies

Documentation

  • No documentation changes required

@bobpaulin bobpaulin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @markap14 the new code preserves the order as intended. One nit comment just for readability. Otherwise test coverage and functionality matches as expected.

markap14 and others added 2 commits September 15, 2026 11:57
…ain/java/org/apache/nifi/controller/queue/SwappablePriorityQueue.java

Co-authored-by: Bob Paulin <bob@bobpaulin.com>
Update remaining references in dropFlowFiles after renaming the local variable per review feedback.

Co-authored-by: Cursor <cursoragent@cursor.com>
@markap14

Copy link
Copy Markdown
Contributor Author

[grok-4.6] Good catch on the incomplete rename from @bobpaulin's suggestion — the declaration was updated to rewrittenSwapLocations but three uses in dropFlowFiles still referenced updatedSwapLocations, which failed code-compliance / Validate with:

[WARNING] SwappablePriorityQueue.java:[1042,19] (coding) UnusedLocalVariable: Unused named local variable 'rewrittenSwapLocations'.

Fixed in 28249a3 (signed); all references in that block now use rewrittenSwapLocations.

@bobpaulin bobpaulin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update @markap14. This looks good.

@bobpaulin
bobpaulin merged commit d323fce into apache:main Sep 15, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants