test(integration): tolerate a slow server in three latency-sensitive suites - #178
Open
ambiorix2099 wants to merge 2 commits into
Open
test(integration): tolerate a slow server in three latency-sensitive suites#178ambiorix2099 wants to merge 2 commits into
ambiorix2099 wants to merge 2 commits into
Conversation
These four waits are the only ones in the integration suite that override waitForWorkflowCompletion's 5-minute default, at 30s. Each waits on a worker to poll for and complete its tasks, so they are the most sensitive in the suite to server latency, and they lose races on a contended shared server: shard 3/3 has intermittently failed on 'multi worker example', 'Should test a workflow' and two WorkerAdvanced cases across Node 20 and 22 since mid-August. Locally the slowest of them finishes in 3.4s. Raise the budget to 90s and the file's jest timeout to 120s, matching WorkerAdvanced.test.ts.
… server WorkflowExecutor.test.ts: the Return Strategy describe sets a 300s jest timeout, but its beforeAll passed an explicit 30000 override. Registering several workflow definitions against a slow sdkdev outran it, and a dead hook fails every test in the file. Inherit the describe's timeout. LeaseExtension.validation.test.ts: both tests batchPoll with a 200ms long-poll window immediately after startWorkflow, so a queue that takes longer than that to surface the task yields no task and the suite fails on expect(task).toBeDefined(). Widen the window to 5s; the 20s execution and heartbeat behaviour under test is unchanged. Verified against a local Conductor: TaskManager and LeaseExtension pass, and WorkflowExecutor now gets past its hook.
Author
|
This CI issue was uncovered while working on this other PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request type
Summary
WorkflowExecutor.test.ts(the whole file),LeaseExtension.validation.test.tsandTaskManager.test.ts.WorkflowExecutor'sbeforeAllpassed an explicit30000that overrode its own describe'sjest.setTimeout(300000), and a dead hook fails every test in the file. BothLeaseExtensiontestsbatchPollwith a 200ms long-poll window immediately afterstartWorkflow, so a queue slower than that yields no task.TaskManager's four waits were the only overrides ofwaitForWorkflowCompletion's 5-minute default, at 30s.User impact
Shard 3/3 has lost a rotating test since mid-August, on
mainand on unrelated branches. Once #176 gatedREGION_DURABLE, these became the visible failure and tookWorkflowExecutor'sSYNCcases down with them, since one expired hook fails every test in its file.Changes
WorkflowExecutor.test.ts: drop thebeforeAlltimeout override so it inherits the describe's 300s.LeaseExtension.validation.test.ts:batchPollwindow 200ms to 5s, namedPOLL_WAIT_MS. The 20s execution and heartbeat behaviour under test is unchanged.TaskManager.test.ts: four waits to 90s viaWF_WAIT_MS, jest timeout to 120s, matchingWorkerAdvanced.test.ts.Test plan
integration v5 sdkdevshard passed on Node 20, 22 and 24, against the same server that had been failing them.TaskManagerandLeaseExtensionpass, andWorkflowExecutorgets past the hook that was expiring.