Skip to content

Fix incremental build/publish test flakiness by isolating NuGet cache#54870

Open
MichaelSimons wants to merge 1 commit into
mainfrom
michaelsimons/fix-incremental-build-test-flakiness
Open

Fix incremental build/publish test flakiness by isolating NuGet cache#54870
MichaelSimons wants to merge 1 commit into
mainfrom
michaelsimons/fix-incremental-build-test-flakiness

Conversation

@MichaelSimons

Copy link
Copy Markdown
Member

Summary

Fixes #54823

Three incremental build/publish tests are intermittently failing because their timestamp assertions are invalidated by concurrent NuGet restore activity from other test classes.

Root Cause

xUnit runs test classes in parallel by default, and all tests share a single global NuGet packages cache (SdkTestContext.NuGetCachePath). NuGet's restore no-op optimization checks whether the packages folder has changed since the last restore. When a concurrent test class writes to the shared cache, NuGet's no-op check is invalidated, causing restore to rewrite project.assets.json — which in turn triggers MSBuild to regenerate downstream files like .assets.cache and .runtimeconfig.dev.json, breaking the timestamp equality assertions.

Fix

Added a WithIsolatedNuGetCache(string baseDirectory) method to TestCommand that sets the NUGET_PACKAGES environment variable to a per-test directory. This ensures each test's restore operates against its own cache, preventing concurrent restores from interfering with no-op detection.

Applied to the three affected tests:

  • ResolvePackageAssets_runs_incrementally
  • GenerateBuildRuntimeConfigurationFiles_runs_incrementally
  • GeneratePublishDependencyFile_runs_incrementally

This follows the same pattern used by AspNetSdkTest.ApplyDefaults(), which already isolates its NuGet cache for ASP.NET SDK tests.

Add WithIsolatedNuGetCache() to TestCommand, which sets the NUGET_PACKAGES
environment variable to a per-test directory. This prevents concurrent test
restores (xUnit runs test classes in parallel) from interfering with NuGet's
no-op detection via the shared global packages folder, which can cause NuGet
to touch project.assets.json timestamps and invalidate the assets cache.

Apply it to the three affected incremental build/publish tests:
- ResolvePackageAssets_runs_incrementally
- GenerateBuildRuntimeConfigurationFiles_runs_incrementally
- GeneratePublishDependencyFile_runs_incrementally

Fixes #54823

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 18, 2026 16:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses intermittent incremental build/publish test failures caused by parallel test execution sharing a global NuGet packages cache, which can invalidate NuGet restore no-op detection and rewrite restore outputs that the tests assert are unchanged.

Changes:

  • Added TestCommand.WithIsolatedNuGetCache(string baseDirectory) to set NUGET_PACKAGES to a per-test cache directory.
  • Updated two incremental build tests to use an isolated NuGet cache rooted under the test asset directory.
  • Updated one incremental publish test to use an isolated NuGet cache rooted under the test asset directory.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
test/Microsoft.NET.TestFramework/Commands/TestCommand.cs Adds a helper to isolate NUGET_PACKAGES per test command to avoid cross-test restore interference.
test/Microsoft.NET.Build.Tests/GivenThatWeWantBuildsToBeIncremental.cs Uses the isolated NuGet cache for incremental build timestamp assertions.
test/Microsoft.NET.Publish.Tests/GivenThatWeWantToPublishIncrementally.cs Uses the isolated NuGet cache for incremental publish timestamp assertions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Test flakiness witt Microsoft.NET.Build.Tests.GivenThatWeWantBuildsToBeIncremental tests

2 participants