Skip to content

feat: run steps in parallel with background, wait, wait-all, cancel and parallel - #6161

Draft
McNultyyy wants to merge 4 commits into
nektos:masterfrom
McNultyyy:act-split/6-parallel-steps
Draft

feat: run steps in parallel with background, wait, wait-all, cancel and parallel#6161
McNultyyy wants to merge 4 commits into
nektos:masterfrom
McNultyyy:act-split/6-parallel-steps

Conversation

@McNultyyy

Copy link
Copy Markdown

Closes #6124.

GitHub shipped step level parallelism in June 2026. act rejects the keys as unknown properties, so a workflow using them cannot be run locally at all — the reporter of #6124 gets a schema error before anything executes.

Draft, because this is stacked. The first three commits are #6153 and #6154, already open on their own. Only the last commit, 598dd85, is new here. I will rebase and mark this ready once those two land. Opening it now so #6124 has something concrete attached to it.

What it adds

steps:
  - id: server
    run: npm start
    background: true        # job continues without waiting

  - run: npm test           # runs while the server is up

  - parallel:               # all at once, implicit wait after the group
      - run: npm run build:frontend
      - run: npm run build:backend

  - wait: server            # or a list, or `wait-all:` for everything
  - cancel: server          # graceful stop

background: true on run and regular steps; wait as string-or-sequence; wait-all as null-or-boolean; cancel; and parallel.

This is upstream GitHub syntax, not an act extension — actions/languageservices workflow-v1.0.json defines steps-item as the same six member one-of, with those exact value types. Composite actions still reject all of it, because GitHub's own action-v1.0.json does too; that is fidelity, not a gap.

How it works

A per job registry tracks background steps behind a ten slot semaphore, matching GitHub's limit. expandParallelStepGroups desugars a parallel: block into background steps plus a synthesized wait for the group, so a group is not a second execution path — it is the same one, spelled shorter. A background step's failure is taken by the first wait that includes it; deliberately cancelled steps do not fail the job; anything still running when the job ends is stopped.

Steps now genuinely overlap, so the state they share needs guarding. Expression evaluators read env and step-result snapshots rather than the live maps, and log writers travel on the context instead of being swapped into the execution environment's single global writer slot.

That last part is the subtle one. Once context writers take precedence, JobContainer.ReplaceLogWriter silently stops having any effect. newCompositeCommandExecutor used it to install the handler that parses a composite action's inner ::set-output, so without the matching change those outputs get attributed to the outer step. This PR moves that call site — and the two other capture sites in expression.go and run_context.go — onto container.WithLogWriters. I mention it because it is exactly the kind of change that looks unrelated in a diff and is not.

Testing

TestExpandParallelStepGroups plus new cases for nesting rejection and for expansion not writing through to the job model. background-steps-container and a new background-steps-matrix fixture (two parallel groups and a loose background step across three matrix entries) are in the TestRunEvent table.

Verified before opening:

  • The composite suite — uses-composite, uses-nested-composite, composite-fail-with-output, act-composite-env-test, do-not-leak-step-env-in-composite, outputs, composite-undeclared-outputs — all green under docker. This is the acceptance gate for the log-writer change.
  • go test -race ./pkg/runner/ in a linux container reports three races, all in GoGitActionCache (GoGitActionCache race condition with concurrent matrix jobs #6028) and all reproducing identically on a clean master checkout. Nothing in the code this PR adds.
  • The -short failure set matches a master baseline exactly. TestActionCache/Fetch_HEAD and Fetch_Sha are flaky run to run on master as well.

Notes for review

  • Not a fix for Not able to use background tasks #2287. A shell & inside a run step still dies with the exec. This is a different, opt-in mechanism that requires rewriting the workflow.
  • The schema types parallel group members as steps-item, as GitHub does, which means a nested parallel: or a wait: inside a group validates. Since neither has a meaning as a background step, the expansion rejects them with a named error rather than dropping them silently. Happy to narrow the schema instead if you would rather it fail at validation time.
  • parallel-step permits name/id where GitHub's permits only parallel. The synthesized wait step reuses the group's id, so matching GitHub exactly would need a second id scheme — I did not think that was worth it, but say the word.
  • Auto-assigned ids for unnamed steps in a group follow act's existing fmt.Sprintf("%d", i) convention and land on the index the step ends up at.

McNultyyy and others added 4 commits August 5, 2026 08:52
…cess

Three data races that are reachable today, with no behaviour change:

* ptyWriter.AutoStop is written by the goroutine running the command
  once it finished, while the goroutine copying the pty output reads it
  on every write. It becomes an atomic.Bool. The new test reproduces
  this reliably under -race.
* containerReference.ReplaceLogWriter swaps two fields that the
  goroutines copying container output read concurrently, and
  HostEnvironment.ReplaceLogWriter swaps StdOut while a command is
  running. Both are now guarded, and the readers take a consistent
  snapshot through an accessor.
* RunContext.Masks is appended to by a running step while the log
  formatter iterates it for every emitted line, and composite run
  contexts aliased the parent's slice and appended to it. Appends and
  reads now take a lock, and composite contexts copy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every step wrote its file commands to the same paths in the job
container: workflow/outputcmd.txt, statecmd.txt, pathcmd.txt, envs.txt
and SUMMARY.md. A step therefore read whatever the previously executed
step had left in them.

The visible consequence is composite actions leaking outputs. GitHub
scopes an output written by a composite action's inner step to that
inner step, and exposes only the outputs the action declares. In act the
inner step's writes to $GITHUB_OUTPUT were still sitting in the shared
file when the enclosing job step processed it, so they appeared as
outputs of the step that used the action.

Each step execution now gets its own directory. The file commands are
also recorded against the id of the step they belong to instead of the
mutable CurrentStep field, and composite script names are derived from
the stable id of the step the composite runs for rather than from
CurrentStep.

The new composite-undeclared-outputs fixture covers a plain and a nested
composite action, over both $GITHUB_OUTPUT and ::set-output. It fails on
master and passes here.

Refs nektos#2184, nektos#2697, nektos#2553

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ommand-dirs' into act-split/6-parallel-steps
…nd parallel

GitHub shipped step level parallelism in June 2026. act rejects the keys as
unknown properties, so a workflow using them cannot be run locally at all.

Model and schema learn the six member `steps-item` one-of that GitHub's own
`workflow-v1.0.json` defines: `background` on run and regular steps, `wait` as
string-or-sequence, `wait-all` as null-or-boolean, `cancel`, and `parallel`.

At run time a per job registry tracks background steps behind a ten slot
semaphore. `expandParallelStepGroups` desugars a `parallel:` block into
background steps plus a synthesized `wait` for the group, so the group has an
implicit join without a second execution path. A background step's failure is
taken by the first `wait` that includes it, cancelled steps do not fail the
job, and anything still running when the job ends is stopped.

Steps now genuinely overlap, so the job state they share needs guarding:
expression evaluators read env and step result snapshots, the runner file
command directories are already per execution, and log writers travel on the
context instead of being swapped into the execution environment's single
global slot. That last part is why `newCompositeCommandExecutor` publishes its
handler through `container.WithLogWriters` as well - with the context writers
in place, replacing the writer slot no longer has any effect, and a composite
action's inner `::set-output` would otherwise be attributed to the outer step.

Composite actions still reject these keys, matching GitHub's action schema.

Closes nektos#6124
McNultyyy added a commit to McNultyyy/act that referenced this pull request Aug 17, 2026
The composite suite and both background-step fixtures pass under docker, and
the three races the detector reports reproduce identically on a clean master
clone - they are the known nektos#6028 GoGitActionCache race, not anything this PR
adds.

Also records that TestActionCache is flaky enough to fake a regression on a
single baseline run, so baseline twice before believing a failure-set delta.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for concurrent steps execution

1 participant