Fix module-level state leaks that broke randomized test order - #362
Merged
Conversation
Bun swaps a mocked module's exports onto the same namespace object it hands back from import(), so a file that captures "the real module" via a bare import() before mocking is holding a reference that later mutates into the mock. Restoring with that reference in afterAll was therefore a no-op, and files loaded after left every other test in the process running against the mock: authz, verify, secret-guard, and read-file-guard plugins collapsed to their fakes, mcpClientToAgentTools lost several exports, and any file statically importing mcp/client.js after web-search.test.ts lost unwrapToolContent and connectMCPServers entirely. Shallow-copying the captured exports at capture time freezes a real snapshot before the mock overwrites the shared object, so afterAll can actually put the original back. The web-search mock also now spreads the real module instead of replacing it outright, and the mouse-reporting-disabled mock (already spreading real exports) gets the same snapshot fix so its existing afterAll restore is not silently defeated the same way. The dead mock of a nonexistent src/web/plugin.js module is removed along with its unusable real-module capture.
schedulePricingMetadataRefresh guards itself with a module-level one-shot flag shared by the whole test process. Any other file that exercises loadConfig's real bootstrap path leaves that flag set to true and never clears it, so this file's own first test silently inherited "already scheduled" from whichever file ran before it and its offlineFetch was never called.
A file order that happens to work is not evidence of isolation; only running with --randomize catches a test that leans on another file's side effects. Fixing the leaks this uncovered is only durable if the check keeps running, so add a CI step alongside the canonical run and write the underlying rule into AGENTS.md.
config.test.ts captured node:os with a static `import * as nodeOs`
and later "restored" it with `mock.module("node:os", () => nodeOs)`
-- the same defect as the dynamic-import case already fixed
elsewhere in this branch, since Bun mutates that namespace object in
place regardless of how it was imported. The capture is now a
shallow copy taken before anything mocks node:os.
state.test.ts mocked node:fs/promises with no restore at all. Its
delegate functions were captured before the mock took effect so the
leak was inert here, but the file still violated the isolation rule
this branch exists to enforce, so it gets the same shallow-copy
capture and an afterAll restore.
The AGENTS.md wording only mentioned the dynamic-import capture shape, which reads as though that is the only form at risk -- it is not; a static `import * as ns` binding is mutated by mock.module the same way, and config.test.ts had exactly that bug. State the rule in terms of any live namespace binding instead of one idiom. The fixed-seed CI step is deterministic and worth keeping, but it only ever exercises one shuffle of the suite, so a leak that particular order does not disturb stays invisible forever. Add a nightly job with a fresh random seed each run, printed up front so a failure reproduces locally with the exact same --seed.
TheGreatAxios
force-pushed
the
cl-5579-test-ordering
branch
from
August 7, 2026 08:26
13e1701 to
9e3b9db
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
bun test --randomizefailed with over a hundred failures spreadacross unrelated files (measured at seed 42 on unmodified
main:~107-119 failures across repeated runs — the count is not perfectly
stable run-to-run, meaning some failures are timing-sensitive rather
than purely a function of the shuffle order). All of it traced to
two shared-state bugs, plus two further instances of the first one:
mock.modulerestores that captured "the real module" via a livenamespace binding — either
await import()or a staticimport * as ns— were no-ops, because Bun mutates that samenamespace object in place when the module gets mocked. The
captured reference silently turns into the mock too, so a later
mock.module(path, () => capturedReference)"restore" changesnothing. This broke:
tests/unit/tui/agent-tools.test.ts(authz/verify/secret-guard/read-file-guard plugins collapsed to fakes for every file that
ran afterward)
src/tools/web-search.test.ts(droppedunwrapToolContentandconnectMCPServersfrommcp/client.jsprocess-wide)tests/unit/config.test.ts(node:os capture via a staticimport * as nodeOs, same defect, different capture shape)mouse-reporting-disabled.test.ts(latent, not yet triggering afailure — fixed proactively)
src/session/state.test.tsmockednode:fs/promiseswith norestore at all; harmless in practice because its delegate was
captured before the mock took effect, but still fixed to restore
Fixed by shallow-copying the capture immediately (
{ ...ns }),regardless of which import form produced it, and spreading the
real module into the mock instead of replacing it outright. Also
removed a dead mock of a
src/web/plugin.jsmodule that no longerexists.
schedulePricingMetadataRefresh's one-shot latch is module-leveland shared by the whole test process; any file that exercises the
real
loadConfigbootstrap path leaves it set, sopricing-metadata.test.ts's own dedupe test silently no-op'd whenit ran after one of those files. Fixed with a
beforeEachresetalongside the existing
afterEachone.AGENTS.mddocuments the rule in terms of any live namespacebinding, not just the dynamic-import idiom, so the next author
following the guidance doesn't reproduce the config.test.ts bug in a
different shape. CI keeps the deterministic fixed-seed step and adds
a nightly job with a fresh random seed each run, printed up front so
a failure reproduces locally with the same
--seed.No assertions were weakened. Each fix restores real isolation — the
tests that exercise the fixed plugins/latch still fail if that
isolation regresses.
Verification
bun run typecheck,bun run build,bun run testall passbun test ./src ./tests ./evals --randomizepasses across 20seeds (42, 1337, 1361063567, 1-10, 100, 200, 300, 7, 555, 909090,
2026, 314159, 8675309, 424242)
Closes CL-5579