Fix incremental build/publish test flakiness by isolating NuGet cache#54870
Open
MichaelSimons wants to merge 1 commit into
Open
Fix incremental build/publish test flakiness by isolating NuGet cache#54870MichaelSimons wants to merge 1 commit into
MichaelSimons wants to merge 1 commit into
Conversation
Add WithIsolatedNuGetCache() to TestCommand, which sets the NUGET_PACKAGES environment variable to a per-test directory. This prevents concurrent test restores (xUnit runs test classes in parallel) from interfering with NuGet's no-op detection via the shared global packages folder, which can cause NuGet to touch project.assets.json timestamps and invalidate the assets cache. Apply it to the three affected incremental build/publish tests: - ResolvePackageAssets_runs_incrementally - GenerateBuildRuntimeConfigurationFiles_runs_incrementally - GeneratePublishDependencyFile_runs_incrementally Fixes #54823 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR addresses intermittent incremental build/publish test failures caused by parallel test execution sharing a global NuGet packages cache, which can invalidate NuGet restore no-op detection and rewrite restore outputs that the tests assert are unchanged.
Changes:
- Added
TestCommand.WithIsolatedNuGetCache(string baseDirectory)to setNUGET_PACKAGESto a per-test cache directory. - Updated two incremental build tests to use an isolated NuGet cache rooted under the test asset directory.
- Updated one incremental publish test to use an isolated NuGet cache rooted under the test asset directory.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| test/Microsoft.NET.TestFramework/Commands/TestCommand.cs | Adds a helper to isolate NUGET_PACKAGES per test command to avoid cross-test restore interference. |
| test/Microsoft.NET.Build.Tests/GivenThatWeWantBuildsToBeIncremental.cs | Uses the isolated NuGet cache for incremental build timestamp assertions. |
| test/Microsoft.NET.Publish.Tests/GivenThatWeWantToPublishIncrementally.cs | Uses the isolated NuGet cache for incremental publish timestamp assertions. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #54823
Three incremental build/publish tests are intermittently failing because their timestamp assertions are invalidated by concurrent NuGet restore activity from other test classes.
Root Cause
xUnit runs test classes in parallel by default, and all tests share a single global NuGet packages cache (
SdkTestContext.NuGetCachePath). NuGet's restore no-op optimization checks whether the packages folder has changed since the last restore. When a concurrent test class writes to the shared cache, NuGet's no-op check is invalidated, causing restore to rewriteproject.assets.json— which in turn triggers MSBuild to regenerate downstream files like.assets.cacheand.runtimeconfig.dev.json, breaking the timestamp equality assertions.Fix
Added a
WithIsolatedNuGetCache(string baseDirectory)method toTestCommandthat sets theNUGET_PACKAGESenvironment variable to a per-test directory. This ensures each test's restore operates against its own cache, preventing concurrent restores from interfering with no-op detection.Applied to the three affected tests:
ResolvePackageAssets_runs_incrementallyGenerateBuildRuntimeConfigurationFiles_runs_incrementallyGeneratePublishDependencyFile_runs_incrementallyThis follows the same pattern used by
AspNetSdkTest.ApplyDefaults(), which already isolates its NuGet cache for ASP.NET SDK tests.