test(atecontroller): retry the WorkerPool update on conflict - #995
Open
NekoPunch (orangeCatDeveloper) wants to merge 1 commit into
Open
Conversation
Benjamin Elder (BenTheElder)
requested a review
from Julian Gutierrez Oschmann (juli4n)
August 17, 2026 19:33
NekoPunch (orangeCatDeveloper)
force-pushed
the
fix/workerpool-template-update-conflict
branch
3 times, most recently
from
August 18, 2026 01:31
62d71d3 to
9b30748
Compare
TestWorkerPoolPodTemplateUpdate did a bare get-then-update while the reconciler writes the same object's status, so a reconcile landing between the read and the write rejected it with "the object has been modified". The file already has updateWorkerPoolSpec for this; use it. Claude-Session: https://claude.ai/code/session_01XuQqkwLf5Zx6CSZFHSC6hb
NekoPunch (orangeCatDeveloper)
force-pushed
the
fix/workerpool-template-update-conflict
branch
from
August 18, 2026 01:58
9b30748 to
67e1739
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #994
TestWorkerPoolPodTemplateUpdateintermittently fails on main with:Three runs on
maincarry it, each verified to fail on this test with this signature:The test read the WorkerPool and wrote it back without handling conflicts, while
WorkerPoolReconcilerwrites the same object's status (workerpool_controller.go:142) and bumps itsresourceVersion. A reconcile landing between the read and the write rejects it. The file already hasupdateWorkerPoolSpec, which re-reads and retries on conflict — and the comment on its sibling describes this exact failure — so this just uses it.Verification
-count=Nproves nothing here: the unfixed test passes at-count=100 -racelocally because the window is too narrow to hit by chance. I constructed the conflict instead, inserting a write between the test's read and its write:Operation cannot be fulfilled on workerpools.ate.dev "race-probe-bare": the object has been modified; please apply your changes to the latest version and try againupdateWorkerPoolSpecNodeSelectorisupdatedafterwards)The probe tests were only for this and are not included.
Scope: this shows the race is real and that the helper handles it. It does not prove the three CI failures took this exact path — the constructed contention is a manual write, whereas on CI it is the reconciler's status update. The error signature, the cause in the code, and the pre-existing helper all line up, but that last step is inference.
I checked the other
Updatecall sites in this file; they already go through the retry helpers.