Skip to content

[Experiment] Combine CI Build and Queue Tests into a single invocation #54933

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
mmitche wants to merge 3 commits into
base: main
Choose a base branch
from
+106 −70
Draft

[Experiment] Combine CI Build and Queue Tests into a single invocation #54933

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3f14c1d
[Experiment] Combine CI Build and Queue Tests into a single invocation
mmitche Jun 23, 2026
7c82fad
Fix: build product solution + orchestrator via /p:Projects instead of…
mmitche Jun 23, 2026
e8c253a
Fix MSB1006: quote the multi-project /p:Projects value in the bash leg
mmitche Jun 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion eng/XUnitV3/XUnitV3.Runner.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

<Target Name="RunTests"
Outputs="%(TestToRun.ResultsStdOutPath)"
Condition="'@(TestToRun)' != ''">
Condition="'@(TestToRun)' != '' and '$(CustomHelixTargetQueue)' == ''">
<Telemetry EventName="NETCORE_ENGINEERING_TELEMETRY" EventData="Category=Test" />

<PropertyGroup Condition="'$(UseMicrosoftTestingPlatformRunner)' == 'true'">
Expand Down
174 changes: 105 additions & 69 deletions eng/pipelines/templates/jobs/sdk-build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,96 +92,132 @@ jobs:
checkDownloadedFiles: true
displayName: 🟣 Download Manifest MSI Packages

############### BUILDING ###############
############### BUILDING (+ TESTING) ###############
# EXPERIMENT: The former separate 'Build' and 'Queue Tests' steps are combined into a
# single build.{ps1,sh} invocation to evaluate eliminating the duplicate restore/build
# between them (the test publish in test/UnitTests.proj re-restores and recompiles the
# product with a RuntimeIdentifier, which previously happened only in the second step).
#
# When running tests, the combined step builds the product solution (sdk.slnx) AND the
# Helix orchestrator (test/UnitTests.proj) in a single invocation via /p:Projects, with
# -test. Because MSBuild builds the listed projects in order, the product is built before
# UnitTests.proj publishes/queues the tests. Per-project in-process test execution is
# suppressed via the RunTests guard in eng/XUnitV3/XUnitV3.Runner.targets (keyed off
# CustomHelixTargetQueue), so the individual *.Tests.csproj do not run on the build agent;
# UnitTests.proj still submits them to Helix.
#
# Build-only legs (runTests=false) build only sdk.slnx, exactly as the former 'Build'
# step did, so they neither queue Helix work nor pull in the orchestrator.
- ${{ if eq(parameters.pool.os, 'windows') }}:
- powershell: eng/common/build.ps1
-restore -build -pack -ci -nativeToolsOnMachine
-configuration $(buildConfiguration)
${{ parameters.publishArgument }}
${{ parameters.signArgument }}
/p:EnableDefaultArtifacts=${{ parameters.enableDefaultArtifacts }}
/p:TargetArchitecture=${{ parameters.targetArchitecture }}
/p:PgoInstrument=${{ parameters.pgoInstrument }}
${{ parameters.runtimeSourceProperties }}
${{ parameters.officialBuildProperties }}
/p:DotNetSignType=$(_SignType) /p:TeamName=$(_TeamName)
displayName: 🟣 Build
env:
BuildConfig: $(buildConfiguration)
TestFullMSBuild: ${{ parameters.testFullMSBuild }}
MSBUILDALWAYSRETRY: true

- ${{ else }}:
- script: |
source $(Build.SourcesDirectory)/eng/common/native/init-os-and-arch.sh
source $(Build.SourcesDirectory)/eng/common/native/init-distro-rid.sh
initDistroRidGlobal "$os" "$arch" ""

. $(Build.SourcesDirectory)/eng/common/build.sh \
-restore -build -pack -ci \
-configuration $(buildConfiguration) \
${{ parameters.publishArgument }} \
${{ parameters.signArgument }} \
/p:EnableDefaultArtifacts=${{ parameters.enableDefaultArtifacts }} \
/p:TargetArchitecture=${{ parameters.targetArchitecture }} \
/p:PgoInstrument=${{ parameters.pgoInstrument }} \
/p:TargetRid=${{ parameters.runtimeIdentifier }} \
${{ parameters.osProperties }} \
${{ parameters.runtimeSourceProperties }} \
${{ parameters.officialBuildProperties }} \
/p:DotNetSignType=$(_SignType) /p:TeamName=$(_TeamName)
displayName: 🟣 Build
env:
BuildConfig: $(buildConfiguration)
OPENSSL_ENABLE_SHA1_SIGNATURES: 1
MSBUILDALWAYSRETRY: true
TARGET_ARCHITECTURE: ${{ parameters.targetArchitecture }}

############### TESTING ###############
- ${{ if eq(parameters.runTests, true) }}:

# For the /p:Projects syntax for PowerShell, see: https://github.com/dotnet/msbuild/issues/471#issuecomment-1146466335
- ${{ if eq(parameters.pool.os, 'windows') }}:
- ${{ if eq(parameters.runTests, true) }}:
# For the /p:Projects syntax for PowerShell, see: https://github.com/dotnet/msbuild/issues/471#issuecomment-1146466335
- powershell: eng/common/build.ps1
-restore -test -ci -prepareMachine -nativeToolsOnMachine
-restore -build -pack -test -ci -prepareMachine -nativeToolsOnMachine
-configuration $(buildConfiguration)
/p:Projects=\`"${{ replace(parameters.testProjects, ';', '`;') }}\`"
/p:TestRunnerAdditionalArguments="${{ parameters.testRunnerAdditionalArguments }}"
${{ parameters.publishArgument }}
${{ parameters.signArgument }}
/p:Projects=\`"$(Build.SourcesDirectory)/sdk.slnx`;$(Build.SourcesDirectory)/test/UnitTests.proj\`"
/p:EnableDefaultArtifacts=${{ parameters.enableDefaultArtifacts }}
/p:TargetArchitecture=${{ parameters.targetArchitecture }}
/p:PgoInstrument=${{ parameters.pgoInstrument }}
${{ parameters.runtimeSourceProperties }}
${{ parameters.officialBuildProperties }}
/p:DotNetSignType=$(_SignType) /p:TeamName=$(_TeamName)
/p:TestRunnerAdditionalArguments="${{ parameters.testRunnerAdditionalArguments }}"
/p:CustomHelixTargetQueue=${{ parameters.helixTargetQueue }}
/p:EnableHelixJobMonitor=true
/bl:$(Build.SourcesDirectory)/artifacts/log/$(buildConfiguration)/${{ parameters.categoryName }}Tests.binlog
displayName: 🟣 Queue Tests
/bl:$(Build.SourcesDirectory)/artifacts/log/$(buildConfiguration)/${{ parameters.categoryName }}Build.binlog
displayName: 🟣 Build and Test
env:
BuildConfig: $(buildConfiguration)
TestFullMSBuild: ${{ parameters.testFullMSBuild }}
MSBUILDALWAYSRETRY: true
# Required by Arcade for running in Helix.
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
HelixAccessToken: $(HelixApiAccessToken)
RunAoTTests: ${{ parameters.runAoTTests }}
TestFullMSBuild: ${{ parameters.testFullMSBuild }}

- ${{ else }}:
# For the /p:Projects syntax for Bash, see: https://github.com/dotnet/msbuild/issues/471#issuecomment-1690189034
# The /p:CustomHelixTargetQueue syntax is: <queue-name>@<container-url>
# For the Helix containers, see the 'simpleTags' arrays here: https://github.com/dotnet/versions/blob/main/build-info/docker/image-info.dotnet-dotnet-buildtools-prereqs-docker-main.json
- script: eng/common/build.sh
-restore -test -ci -prepareMachine
- powershell: eng/common/build.ps1
-restore -build -pack -ci -nativeToolsOnMachine
-configuration $(buildConfiguration)
'/p:Projects="${{ parameters.testProjects }}"'
/p:TestRunnerAdditionalArguments="${{ parameters.testRunnerAdditionalArguments }}"
${{ parameters.publishArgument }}
${{ parameters.signArgument }}
/p:Projects=$(Build.SourcesDirectory)/sdk.slnx
/p:EnableDefaultArtifacts=${{ parameters.enableDefaultArtifacts }}
/p:TargetArchitecture=${{ parameters.targetArchitecture }}
/p:TargetRid=${{ parameters.runtimeIdentifier }}
${{ parameters.osProperties }}
/p:PgoInstrument=${{ parameters.pgoInstrument }}
${{ parameters.runtimeSourceProperties }}
/p:CustomHelixTargetQueue=${{ parameters.helixTargetQueue }}${{ parameters.helixTargetContainer }}
/p:EnableHelixJobMonitor=true
/bl:$(Build.SourcesDirectory)/artifacts/log/$(buildConfiguration)/${{ parameters.categoryName }}Tests.binlog
displayName: 🟣 Queue Tests
${{ parameters.officialBuildProperties }}
/p:DotNetSignType=$(_SignType) /p:TeamName=$(_TeamName)
displayName: 🟣 Build
env:
BuildConfig: $(buildConfiguration)
TestFullMSBuild: ${{ parameters.testFullMSBuild }}
MSBUILDALWAYSRETRY: true

- ${{ else }}:
# The /p:CustomHelixTargetQueue syntax is: <queue-name>@<container-url>
# For the Helix containers, see the 'simpleTags' arrays here: https://github.com/dotnet/versions/blob/main/build-info/docker/image-info.dotnet-dotnet-buildtools-prereqs-docker-main.json
- ${{ if eq(parameters.runTests, true) }}:
# For the /p:Projects syntax for Bash, see: https://github.com/dotnet/msbuild/issues/471#issuecomment-1690189034
- script: |
source $(Build.SourcesDirectory)/eng/common/native/init-os-and-arch.sh
source $(Build.SourcesDirectory)/eng/common/native/init-distro-rid.sh
initDistroRidGlobal "$os" "$arch" ""

. $(Build.SourcesDirectory)/eng/common/build.sh \
-restore -build -pack -test -ci -prepareMachine \
-configuration $(buildConfiguration) \
${{ parameters.publishArgument }} \
${{ parameters.signArgument }} \
'/p:Projects="$(Build.SourcesDirectory)/sdk.slnx;$(Build.SourcesDirectory)/test/UnitTests.proj"' \
/p:EnableDefaultArtifacts=${{ parameters.enableDefaultArtifacts }} \
/p:TargetArchitecture=${{ parameters.targetArchitecture }} \
/p:PgoInstrument=${{ parameters.pgoInstrument }} \
/p:TargetRid=${{ parameters.runtimeIdentifier }} \
${{ parameters.osProperties }} \
${{ parameters.runtimeSourceProperties }} \
${{ parameters.officialBuildProperties }} \
/p:DotNetSignType=$(_SignType) /p:TeamName=$(_TeamName) \
/p:TestRunnerAdditionalArguments="${{ parameters.testRunnerAdditionalArguments }}" \
/p:CustomHelixTargetQueue=${{ parameters.helixTargetQueue }}${{ parameters.helixTargetContainer }} \
/p:EnableHelixJobMonitor=true \
/bl:$(Build.SourcesDirectory)/artifacts/log/$(buildConfiguration)/${{ parameters.categoryName }}Build.binlog
displayName: 🟣 Build and Test
env:
BuildConfig: $(buildConfiguration)
OPENSSL_ENABLE_SHA1_SIGNATURES: 1
MSBUILDALWAYSRETRY: true
TARGET_ARCHITECTURE: ${{ parameters.targetArchitecture }}
# Required by Arcade for running in Helix.
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
HelixAccessToken: $(HelixApiAccessToken)
RunAoTTests: ${{ parameters.runAoTTests }}
- ${{ else }}:
- script: |
source $(Build.SourcesDirectory)/eng/common/native/init-os-and-arch.sh
source $(Build.SourcesDirectory)/eng/common/native/init-distro-rid.sh
initDistroRidGlobal "$os" "$arch" ""

. $(Build.SourcesDirectory)/eng/common/build.sh \
-restore -build -pack -ci \
-configuration $(buildConfiguration) \
${{ parameters.publishArgument }} \
${{ parameters.signArgument }} \
'/p:Projects=$(Build.SourcesDirectory)/sdk.slnx' \
/p:EnableDefaultArtifacts=${{ parameters.enableDefaultArtifacts }} \
/p:TargetArchitecture=${{ parameters.targetArchitecture }} \
/p:PgoInstrument=${{ parameters.pgoInstrument }} \
/p:TargetRid=${{ parameters.runtimeIdentifier }} \
${{ parameters.osProperties }} \
${{ parameters.runtimeSourceProperties }} \
${{ parameters.officialBuildProperties }} \
/p:DotNetSignType=$(_SignType) /p:TeamName=$(_TeamName)
displayName: 🟣 Build
env:
BuildConfig: $(buildConfiguration)
OPENSSL_ENABLE_SHA1_SIGNATURES: 1
MSBUILDALWAYSRETRY: true
TARGET_ARCHITECTURE: ${{ parameters.targetArchitecture }}

############### POST ###############
Expand Down
Loading