Skip to content

Add KernelAbstractions.@spawn and record_event/wait_event - #750

Merged
christiangnrd merged 12 commits into
mainfrom
vc/spawn
Sep 21, 2026
Merged

Add KernelAbstractions.@spawn and record_event/wait_event#750
christiangnrd merged 12 commits into
mainfrom
vc/spawn

Conversation

@vchuravy

@vchuravy vchuravy commented Sep 7, 2026

Copy link
Copy Markdown
Member

Summary

Adds KernelAbstractions.@spawn backend expr, which runs expr on a new Julia task while keeping the work queued on backend ordered between the two tasks. It encodes the discipline users otherwise have to write by hand:

synchronize(backend)
Threads.@spawn begin
    ...
    synchronize(backend)
end |> wait

The macro's protocol:

  1. The spawning task calls record_event(backend).
  2. The new task selects the spawning task's device with device!, then calls wait_event(backend, event).
  3. After expr returns, the task calls synchronize(backend), so wait(task) / fetch(task) imply that all of the task's device work has completed.

An optional first argument is forwarded to Threads.@spawn as the threadpool (@spawn :interactive backend expr). The macro is not exported, to avoid clashing with Threads.@spawn.

Backend opt-in

Two new optional functions in KernelInterface:

  • record_event(backend) defaults to a full synchronize returning nothing, which is always correct.
  • wait_event(::Backend, ::Nothing) is a no-op.

A backend with task-local streams (CUDA.jl, AMDGPU.jl, ...) can override record_event to record an event on the current stream without blocking, and implement wait_event for that event type as a stream wait, turning the protocol into

event = record()
@spawn begin
    wait(event)
    ...
    synchronize()
end |> wait

Backends with a single global queue (POCL) need no changes.

Docs

  • synchronize docstring now recommends a cooperative, non-blocking implementation, since a blocking one serializes otherwise independent spawned tasks.
  • API and KernelInterface pages list the new functions; the implementer notes gain a "Task-local queues and @spawn" section; the quickstart's task-programming section now recommends @spawn.

Test plan

  • New Spawn testsuite entry (test/spawn.jl): ordering after the parent's queued work, visibility after wait, device pinning, threadpool forms, single evaluation of the backend expression, error propagation, many concurrent tasks. Passes on the CPU backend.
  • KernelInterface standalone tests cover the record_event/wait_event fallbacks.
  • Full CPU testsuite: all pass except a pre-existing Private error caused by my local GPUCompiler dev checkout lacking alloca (unrelated to this change).
  • CI on GPU backends (they use the default synchronize-based fallback until they opt in).

🤖 Generated with Claude Code

https://claude.ai/code/session_01YQWT6DHjjRoh1YaAsFEUN5

@github-actions

github-actions Bot commented Sep 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmark Results

Show table
main c20c359... main / c20c359...
const/@Const/Float32/262144 0.167 ± 0.017 ms 0.247 ± 0.036 ms 0.678 ± 0.12
const/@Const/Float32/65536 0.0673 ± 0.019 ms 0.0697 ± 0.02 ms 0.964 ± 0.39
const/@Const/Float64/262144 0.273 ± 0.018 ms 0.391 ± 0.046 ms 0.699 ± 0.094
const/@Const/Float64/65536 0.101 ± 0.017 ms 0.101 ± 0.02 ms 1 ± 0.26
const/unmarked/Float32/262144 0.247 ± 0.016 ms 0.261 ± 0.038 ms 0.946 ± 0.15
const/unmarked/Float32/65536 0.0945 ± 0.018 ms 0.0975 ± 0.02 ms 0.969 ± 0.27
const/unmarked/Float64/262144 0.447 ± 0.036 ms 0.455 ± 0.063 ms 0.983 ± 0.16
const/unmarked/Float64/65536 0.146 ± 0.023 ms 0.14 ± 0.016 ms 1.04 ± 0.2
launch/3D static workgroup, dynamic ndrange 21.3 ± 17 μs 20.8 ± 14 μs 1.02 ± 1
launch/3D static workgroup, static ndrange 20.5 ± 12 μs 21.3 ± 16 μs 0.96 ± 0.89
launch/dynamic workgroup, dynamic ndrange 21 ± 8.7 μs 20.9 ± 9.5 μs 1.01 ± 0.62
launch/dynamic workgroup, dynamic ndrange, workgroupsize given 20.4 ± 15 μs 20.2 ± 15 μs 1.01 ± 1.1
launch/static workgroup, dynamic ndrange 19.9 ± 13 μs 20.8 ± 15 μs 0.957 ± 0.95
launch/static workgroup, static ndrange 20.3 ± 14 μs 20.5 ± 12 μs 0.99 ± 0.89
partition/dynamic workgroup, dynamic ndrange 0.0452 ± 0.0066 μs 0.0487 ± 0.0062 μs 0.927 ± 0.18
partition/static workgroup, dynamic ndrange 0.0521 ± 0.01 μs 0.0549 ± 0.01 μs 0.949 ± 0.25
partition/static workgroup, static ndrange 1.13 ± 0.029 ns 1.31 ± 0.083 ns 0.864 ± 0.059
saxpy/default/Float16/1024 24 ± 18 μs 25.2 ± 22 μs 0.952 ± 1.1
saxpy/default/Float16/1048576 0.316 ± 0.026 ms 0.318 ± 0.029 ms 0.993 ± 0.12
saxpy/default/Float16/16384 0.0439 ± 0.024 ms 0.0509 ± 0.026 ms 0.862 ± 0.64
saxpy/default/Float16/2048 27.4 ± 22 μs 29.1 ± 22 μs 0.94 ± 1
saxpy/default/Float16/256 22.2 ± 8.3 μs 22.3 ± 11 μs 0.998 ± 0.63
saxpy/default/Float16/262144 0.112 ± 0.022 ms 0.112 ± 0.022 ms 1 ± 0.28
saxpy/default/Float16/32768 0.0523 ± 0.026 ms 0.0539 ± 0.027 ms 0.97 ± 0.68
saxpy/default/Float16/4096 0.0364 ± 0.022 ms 0.0436 ± 0.021 ms 0.836 ± 0.65
saxpy/default/Float16/512 22 ± 9.8 μs 23.6 ± 13 μs 0.933 ± 0.65
saxpy/default/Float16/64 21.6 ± 9.6 μs 22.8 ± 16 μs 0.945 ± 0.78
saxpy/default/Float16/65536 0.0619 ± 0.026 ms 0.0548 ± 0.021 ms 1.13 ± 0.64
saxpy/default/Float32/1024 23.9 ± 23 μs 27.3 ± 24 μs 0.875 ± 1.1
saxpy/default/Float32/1048576 0.335 ± 0.066 ms 0.328 ± 0.052 ms 1.02 ± 0.26
saxpy/default/Float32/16384 0.0484 ± 0.024 ms 0.0429 ± 0.024 ms 1.13 ± 0.84
saxpy/default/Float32/2048 0.034 ± 0.022 ms 0.0416 ± 0.022 ms 0.816 ± 0.69
saxpy/default/Float32/256 21.8 ± 10 μs 22.6 ± 11 μs 0.962 ± 0.65
saxpy/default/Float32/262144 0.106 ± 0.02 ms 0.104 ± 0.021 ms 1.01 ± 0.28
saxpy/default/Float32/32768 0.0522 ± 0.025 ms 0.0526 ± 0.025 ms 0.994 ± 0.68
saxpy/default/Float32/4096 0.035 ± 0.025 ms 0.0355 ± 0.025 ms 0.986 ± 0.97
saxpy/default/Float32/512 22.4 ± 18 μs 24 ± 23 μs 0.931 ± 1.2
saxpy/default/Float32/64 22 ± 9.6 μs 21.8 ± 13 μs 1.01 ± 0.74
saxpy/default/Float32/65536 0.0572 ± 0.026 ms 0.058 ± 0.026 ms 0.987 ± 0.63
saxpy/default/Float64/1024 27.8 ± 25 μs 0.0332 ± 0.026 ms 0.838 ± 0.99
saxpy/default/Float64/1048576 0.583 ± 0.069 ms 0.616 ± 0.088 ms 0.945 ± 0.18
saxpy/default/Float64/16384 0.0492 ± 0.026 ms 0.0501 ± 0.026 ms 0.982 ± 0.73
saxpy/default/Float64/2048 0.0391 ± 0.025 ms 0.0327 ± 0.023 ms 1.2 ± 1.1
saxpy/default/Float64/256 21.9 ± 10 μs 22.2 ± 13 μs 0.984 ± 0.72
saxpy/default/Float64/262144 0.173 ± 0.036 ms 0.176 ± 0.035 ms 0.984 ± 0.28
saxpy/default/Float64/32768 0.0558 ± 0.028 ms 0.0569 ± 0.029 ms 0.981 ± 0.7
saxpy/default/Float64/4096 0.0356 ± 0.025 ms 0.038 ± 0.025 ms 0.938 ± 0.91
saxpy/default/Float64/512 21.6 ± 8.1 μs 22.3 ± 14 μs 0.969 ± 0.7
saxpy/default/Float64/64 22.2 ± 6.1 μs 21.4 ± 10 μs 1.04 ± 0.58
saxpy/default/Float64/65536 0.0671 ± 0.026 ms 0.07 ± 0.027 ms 0.957 ± 0.52
saxpy/static workgroup=(1024,)/Float16/1024 23 ± 20 μs 25 ± 23 μs 0.922 ± 1.2
saxpy/static workgroup=(1024,)/Float16/1048576 0.321 ± 0.036 ms 0.314 ± 0.03 ms 1.02 ± 0.15
saxpy/static workgroup=(1024,)/Float16/16384 0.0477 ± 0.024 ms 0.0482 ± 0.024 ms 0.99 ± 0.7
saxpy/static workgroup=(1024,)/Float16/2048 27.6 ± 20 μs 30.6 ± 20 μs 0.902 ± 0.89
saxpy/static workgroup=(1024,)/Float16/256 22.1 ± 15 μs 23.9 ± 22 μs 0.926 ± 1.1
saxpy/static workgroup=(1024,)/Float16/262144 0.11 ± 0.021 ms 0.109 ± 0.022 ms 1.01 ± 0.28
saxpy/static workgroup=(1024,)/Float16/32768 0.0505 ± 0.024 ms 0.0528 ± 0.025 ms 0.958 ± 0.65
saxpy/static workgroup=(1024,)/Float16/4096 0.039 ± 0.021 ms 0.0383 ± 0.021 ms 1.02 ± 0.78
saxpy/static workgroup=(1024,)/Float16/512 21.7 ± 16 μs 24.3 ± 22 μs 0.893 ± 1.1
saxpy/static workgroup=(1024,)/Float16/64 22.4 ± 15 μs 24 ± 19 μs 0.934 ± 0.98
saxpy/static workgroup=(1024,)/Float16/65536 0.0517 ± 0.021 ms 0.0526 ± 0.02 ms 0.983 ± 0.55
saxpy/static workgroup=(1024,)/Float32/1024 23.4 ± 22 μs 25.5 ± 24 μs 0.92 ± 1.2
saxpy/static workgroup=(1024,)/Float32/1048576 0.333 ± 0.063 ms 0.322 ± 0.051 ms 1.03 ± 0.26
saxpy/static workgroup=(1024,)/Float32/16384 0.046 ± 0.024 ms 0.0473 ± 0.024 ms 0.972 ± 0.71
saxpy/static workgroup=(1024,)/Float32/2048 0.0421 ± 0.021 ms 30.8 ± 20 μs 1.37 ± 1.1
saxpy/static workgroup=(1024,)/Float32/256 22 ± 18 μs 23.2 ± 19 μs 0.947 ± 1.1
saxpy/static workgroup=(1024,)/Float32/262144 0.101 ± 0.02 ms 0.0999 ± 0.019 ms 1.01 ± 0.28
saxpy/static workgroup=(1024,)/Float32/32768 0.0493 ± 0.024 ms 0.0443 ± 0.022 ms 1.11 ± 0.78
saxpy/static workgroup=(1024,)/Float32/4096 0.0326 ± 0.022 ms 31.3 ± 21 μs 1.04 ± 0.97
saxpy/static workgroup=(1024,)/Float32/512 22.4 ± 19 μs 23.5 ± 23 μs 0.951 ± 1.2
saxpy/static workgroup=(1024,)/Float32/64 22.2 ± 13 μs 22.7 ± 15 μs 0.977 ± 0.85
saxpy/static workgroup=(1024,)/Float32/65536 0.0529 ± 0.022 ms 0.0571 ± 0.024 ms 0.928 ± 0.55
saxpy/static workgroup=(1024,)/Float64/1024 26.5 ± 24 μs 0.0326 ± 0.024 ms 0.812 ± 0.96
saxpy/static workgroup=(1024,)/Float64/1048576 0.629 ± 0.092 ms 0.605 ± 0.085 ms 1.04 ± 0.21
saxpy/static workgroup=(1024,)/Float64/16384 0.0474 ± 0.024 ms 0.0475 ± 0.025 ms 0.997 ± 0.73
saxpy/static workgroup=(1024,)/Float64/2048 0.0325 ± 0.023 ms 0.0326 ± 0.022 ms 0.998 ± 0.99
saxpy/static workgroup=(1024,)/Float64/256 25.7 ± 22 μs 25.2 ± 21 μs 1.02 ± 1.2
saxpy/static workgroup=(1024,)/Float64/262144 0.172 ± 0.034 ms 0.168 ± 0.031 ms 1.03 ± 0.28
saxpy/static workgroup=(1024,)/Float64/32768 0.0498 ± 0.022 ms 0.056 ± 0.026 ms 0.889 ± 0.57
saxpy/static workgroup=(1024,)/Float64/4096 0.0406 ± 0.023 ms 0.0367 ± 0.025 ms 1.11 ± 0.97
saxpy/static workgroup=(1024,)/Float64/512 24.2 ± 21 μs 24.8 ± 22 μs 0.977 ± 1.2
saxpy/static workgroup=(1024,)/Float64/64 21.3 ± 9.1 μs 22.8 ± 18 μs 0.932 ± 0.83
saxpy/static workgroup=(1024,)/Float64/65536 0.0656 ± 0.024 ms 0.0639 ± 0.023 ms 1.03 ± 0.52
time_to_load 0.65 ± 0.027 s 0.663 ± 0.019 s 0.98 ± 0.049
main c20c359... main / c20c359...
const/@Const/Float32/262144 1 allocs: 16 B 1 allocs: 16 B 1
const/@Const/Float32/65536 1 allocs: 16 B 1 allocs: 16 B 1
const/@Const/Float64/262144 1 allocs: 16 B 1 allocs: 16 B 1
const/@Const/Float64/65536 1 allocs: 16 B 1 allocs: 16 B 1
const/unmarked/Float32/262144 1 allocs: 16 B 1 allocs: 16 B 1
const/unmarked/Float32/65536 1 allocs: 16 B 1 allocs: 16 B 1
const/unmarked/Float64/262144 1 allocs: 16 B 1 allocs: 16 B 1
const/unmarked/Float64/65536 1 allocs: 16 B 1 allocs: 16 B 1
launch/3D static workgroup, dynamic ndrange 5 allocs: 0.141 kB 5 allocs: 0.141 kB 1
launch/3D static workgroup, static ndrange 6 allocs: 0.172 kB 6 allocs: 0.172 kB 1
launch/dynamic workgroup, dynamic ndrange 15 allocs: 0.312 kB 15 allocs: 0.312 kB 1
launch/dynamic workgroup, dynamic ndrange, workgroupsize given 1 allocs: 16 B 1 allocs: 16 B 1
launch/static workgroup, dynamic ndrange 1 allocs: 16 B 1 allocs: 16 B 1
launch/static workgroup, static ndrange 2 allocs: 0.0469 kB 2 allocs: 0.0469 kB 1
partition/dynamic workgroup, dynamic ndrange 2 allocs: 0.0625 kB 2 allocs: 0.0625 kB 1
partition/static workgroup, dynamic ndrange 2 allocs: 32 B 2 allocs: 32 B 1
partition/static workgroup, static ndrange 0 allocs: 0 B 0 allocs: 0 B
saxpy/default/Float16/1024 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/1048576 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/16384 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/2048 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/256 16 allocs: 0.328 kB 16 allocs: 0.328 kB 1
saxpy/default/Float16/262144 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/32768 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/4096 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/512 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/64 16 allocs: 0.328 kB 16 allocs: 0.328 kB 1
saxpy/default/Float16/65536 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/1024 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/1048576 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/16384 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/2048 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/256 16 allocs: 0.328 kB 16 allocs: 0.328 kB 1
saxpy/default/Float32/262144 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/32768 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/4096 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/512 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/64 16 allocs: 0.328 kB 16 allocs: 0.328 kB 1
saxpy/default/Float32/65536 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/1024 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/1048576 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/16384 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/2048 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/256 16 allocs: 0.328 kB 16 allocs: 0.328 kB 1
saxpy/default/Float64/262144 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/32768 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/4096 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/512 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/64 16 allocs: 0.328 kB 16 allocs: 0.328 kB 1
saxpy/default/Float64/65536 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/static workgroup=(1024,)/Float16/1024 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/1048576 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/16384 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/2048 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/256 1 allocs: 16 B 1 allocs: 16 B 1
saxpy/static workgroup=(1024,)/Float16/262144 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/32768 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/4096 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/512 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/64 1 allocs: 16 B 1 allocs: 16 B 1
saxpy/static workgroup=(1024,)/Float16/65536 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/1024 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/1048576 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/16384 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/2048 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/256 1 allocs: 16 B 1 allocs: 16 B 1
saxpy/static workgroup=(1024,)/Float32/262144 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/32768 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/4096 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/512 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/64 1 allocs: 16 B 1 allocs: 16 B 1
saxpy/static workgroup=(1024,)/Float32/65536 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/1024 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/1048576 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/16384 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/2048 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/256 1 allocs: 16 B 1 allocs: 16 B 1
saxpy/static workgroup=(1024,)/Float64/262144 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/32768 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/4096 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/512 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/64 1 allocs: 16 B 1 allocs: 16 B 1
saxpy/static workgroup=(1024,)/Float64/65536 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
time_to_load 0.201 k allocs: 11.8 kB 0.205 k allocs: 12 kB 0.984

Benchmark Plots

A plot of the benchmark results have been uploaded as an artifact to the workflow run for this PR.
Go to "Actions"->"Benchmark a pull request"->[the most recent run]->"Artifacts" (at the bottom).

vchuravy
vchuravy commented Sep 7, 2026
View reviewed changes
Comment thread docs/src/implementations.md Outdated
vchuravy
vchuravy commented Sep 7, 2026
View reviewed changes
Comment thread docs/src/implementations.md Outdated
vchuravy
vchuravy commented Sep 7, 2026
View reviewed changes
Comment thread docs/src/implementations.md Outdated
vchuravy
vchuravy commented Sep 7, 2026
View reviewed changes
Comment thread docs/src/quickstart.md Outdated
@vchuravy vchuravy added this to the 0.10.0 milestone Sep 7, 2026
@vchuravy

vchuravy commented Sep 8, 2026

Copy link
Copy Markdown
Member Author

@christiangnrd we should have this for KI 0.2 as well

@christiangnrd

christiangnrd commented Sep 8, 2026

Copy link
Copy Markdown
Member

@christiangnrd we should have this for KI 0.2 as well

Sure! Is the idea that things you write your code assuming events are supported, and on backends where it isn't it just silently falls back to sequential operation?

@vchuravy

vchuravy commented Sep 8, 2026

Copy link
Copy Markdown
Member Author

Yeah the sequential ordering is the correct thing, and the events are an optimization.

christiangnrd
christiangnrd reviewed Sep 8, 2026
View reviewed changes
Comment thread lib/KernelInterface/src/host.jl Outdated
christiangnrd
christiangnrd reviewed Sep 8, 2026
View reviewed changes

@christiangnrd christiangnrd left a comment •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude seems to find it very important that someone reading any part of the documentation know that this defaults to synchronize and that if record_event is implemented, wait_event must also be implemented.

I think the docs should focus more on desired behaviour, and any implementation recommendations/guidelines should be tucked away and clearly marked as such since realistically it'll be one of us adding backend support and users probably won't care about implementation unless things break

@christiangnrd

christiangnrd commented Sep 8, 2026

Copy link
Copy Markdown
Member

It'll probably end up in KI 0.2.0, but if I understand correctly, since the backend implementations are just an optimization (and optional), this can release whenever and backends just set compat to whatever version this is released in once they add support

@christiangnrd

christiangnrd commented Sep 11, 2026

Copy link
Copy Markdown
Member

I've added support for the device selection interface in KA in my Opencl branches (feel free to review JuliaGPU/OpenCL.jl@fe59d68) so once #754 is in I'll rebase this and the OpenCL CUDA tests should pass!

@christiangnrd christiangnrd mentioned this pull request Sep 12, 2026
Merged
@christiangnrd
christiangnrd force-pushed the vc/spawn branch 3 times, most recently from 42e70b7 to c46c49b Compare September 12, 2026 17:54
@christiangnrd

christiangnrd commented Sep 12, 2026

Copy link
Copy Markdown
Member

@vchuravy I just pushed a few commits that address my review comments and also some stuff that Claude found. They're all documentation changes, except for 586d110, which you should review particularly more closely.

It also brought up another issue that I didn't fix since I'm not sure what the best approach would be. Bot explanation related to @spawn:

If expr throws, the trailing synchronize is skipped, so the task's queued kernels may still be running when wait(task) throws. The docstring states the completion guarantee unconditionally. Either document that it only holds when expr returns, or wrap the body in try/finally, which risks a failing synchronize masking the original device error. I lean towards documenting it.

@vchuravy

vchuravy commented Sep 12, 2026

Copy link
Copy Markdown
Member Author

hich risks a failing synchronize masking the original device error. I lean towards documenting it.

We could do

try
finally
   sync
end

but then what do we do if sync throws as well. I tend to agree the right thing is to say: In case of an exceptional exit the task is not synced.

@christiangnrd

christiangnrd commented Sep 12, 2026

Copy link
Copy Markdown
Member

I tend to agree the right thing is to say: In case of an exceptional exit the task is not synced.

Done

vchuravy
vchuravy commented Sep 12, 2026
View reviewed changes
Comment thread docs/src/quickstart.md Outdated
vchuravy
vchuravy commented Sep 12, 2026
View reviewed changes
Comment thread src/spawn.jl
@codecov

codecov Bot commented Sep 12, 2026 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.14286% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 66.81%. Comparing base (ec76083) to head (c20c359).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/spawn.jl 97.14% 1 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #750      +/-   ##
==========================================
+ Coverage   66.18%   66.81%   +0.63%     
==========================================
  Files          23       24       +1     
  Lines        1990     2025      +35     
==========================================
+ Hits         1317     1353      +36     
+ Misses        673      672       -1

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@christiangnrd christiangnrd mentioned this pull request Sep 14, 2026
Merged
@vchuravy @christiangnrd
Add KernelAbstractions.@Spawn and record_event/wait_event
02956c5 
`KernelAbstractions.@Spawn backend expr` runs `expr` on a new Julia task
while keeping the work queued on `backend` ordered between the two tasks:

1. the spawning task calls `record_event(backend)`,
2. the new task selects the same device and calls `wait_event`,
3. after `expr` returns the task calls `synchronize(backend)`, so that
   `wait(task)`/`fetch(task)` imply all of its device work has completed.

`record_event` defaults to a full `synchronize` returning `nothing`, and
`wait_event(::Backend, ::Nothing)` is a no-op, so every backend gets the
"synchronize before, synchronize after" discipline for free. Backends with
task-local streams may opt in by recording an event instead and implementing
`wait_event` for it as a stream wait.

Also document that `synchronize` should be cooperative, since a blocking
implementation would serialize otherwise independent spawned tasks.

Assisted-by: Claude Code (Fable 5.1)
vchuravy and others added 10 commits September 18, 2026 11:12
@vchuravy @christiangnrd
Fix spawn test on Julia versions without interactive threads
fe93e98 
`Threads.@Spawn :interactive` runs the task in the `:default` pool when the
interactive pool has no threads, so asserting `Threads.threadpool() === :interactive`
fails on Julia 1.10 and 1.11, which do not start an interactive thread by default.
Expect whichever pool Julia will actually use.

Assisted-by: Claude Code (Opus 5)
@christiangnrd
Bump KernelInterface to v0.2.1
7b9aa34
@christiangnrd @claude
Expand Threads.@Spawn in the caller's scope so @Spawn works under @sync
7a6b681 
The macro expanded Threads.@Spawn inside its own hygienic scope, so the
sync variable Threads.@Spawn escapes resolved to a global in
KernelAbstractions rather than the caller's @sync block: tasks created by
KernelAbstractions.@Spawn were never awaited by @sync, and their errors
were dropped. The same scoping broke $x interpolation in the body.

Escape the whole expansion and gensym our temporaries instead. Add tests
for @sync and $x.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@christiangnrd @claude
Require synchronize to be cooperative in its docstring
07a482c 
The implementer notes already say synchronize is required to be
cooperative; the docstring said backends "should" make it so. Say
"must" in both places, and link to the implementer notes for the
rationale instead of repeating it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@christiangnrd @claude
Implementer notes: task-local queues wording and device interface req…
6624735 
…uirement

Apply the review suggestions: backends should give each task its own
queue, and drop the paragraph about single-queue backends. Note that a
new task does not necessarily inherit its spawner's device, so
multi-device backends must implement the device interface for @Spawn to
pin the task correctly.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@christiangnrd @claude
Describe @Spawn by its guarantees in user-facing docs
e953da1 
The @Spawn docstring and the quickstart walked through the
record_event/wait_event/synchronize protocol. Users only need the
guarantees: same device, ordered after the spawner's queued work, and
complete once wait/fetch returns. Keep the protocol on the implementer
page and point backend authors there.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@christiangnrd @claude
Trim the record_event and wait_event docstrings
ec99943 
Keep the pairing requirement in both, cross-linked, and point to the
implementer notes for the protocol instead of restating it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@christiangnrd @claude
Document that a task whose body throws is not synchronized
8c7d4d8 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@vchuravy @claude @christiangnrd
Scope the record_event/wait_event contract to the active device
495b077 
`wait_event` is queue-ordered, not task-ordered: it makes the queue of the
device that is active when it is called wait on the event. Say so, and spell
out what follows:

- `device!` is not a synchronization point; work queued before a switch is
  unordered with respect to work queued after it.
- The way to order across a switch is to record, switch, then wait -- which is
  the order `@spawn` already uses, now with a comment saying why.
- A multi-device backend has to accept an event recorded on another device,
  either as a driver-level dependency (CUDA's `cuStreamWaitEvent`) or by
  waiting cooperatively.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KfecYSmYnSAbm5Uu476fQf
@vchuravy @claude @christiangnrd
Let @Spawn take an explicit device
ccccf07 
`@spawn [threadpool] backend [device=id] expr` now accepts a device to run on,
defaulting to the spawning task's as before. The event is still recorded on the
spawning task's device, so the ordering guarantee holds across the switch --
which is what the cross-device `wait_event` requirement is for.

Document the reason `@spawn` selects a device at all: backends hold it in
task-local state, which Julia does not copy into a child task, so a plain
`Threads.@spawn` would run on the backend's default device instead of the
caller's.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KfecYSmYnSAbm5Uu476fQf
@christiangnrd

christiangnrd commented Sep 18, 2026

Copy link
Copy Markdown
Member

@vchuravy I think the order of operations should be merge this, implement in the backend branches, test functionality, if it's all good we release the KernelInterface part as v0.2.1 (or 0.3 if we end up requiring breaking changes)

vchuravy
vchuravy commented Sep 21, 2026
View reviewed changes
Comment thread docs/src/implementations.md Outdated
vchuravy
vchuravy commented Sep 21, 2026
View reviewed changes
Comment thread docs/src/implementations.md
vchuravy
vchuravy commented Sep 21, 2026
View reviewed changes
Comment thread docs/src/quickstart.md
vchuravy
vchuravy commented Sep 21, 2026
View reviewed changes
Comment thread docs/src/quickstart.md Outdated
vchuravy
vchuravy commented Sep 21, 2026
View reviewed changes
Comment thread src/spawn.jl
@vchuravy
Apply batched suggestions from code review
c20c359 
Co-authored-by: Valentin Churavy <v.churavy@gmail.com>
@vchuravy

vchuravy commented Sep 21, 2026

Copy link
Copy Markdown
Member Author

I think this is ready from my side. I adjusted the language on some of the implementation notes.

@christiangnrd
christiangnrd merged commit 7d2cdb6 into main Sep 21, 2026
84 checks passed
@christiangnrd
christiangnrd deleted the vc/spawn branch September 21, 2026 11:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@maleadt maleadt Awaiting requested review from maleadt
@christiangnrd christiangnrd Awaiting requested review from christiangnrd

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

0.10.0

Development

Successfully merging this pull request may close these issues.

2 participants

@vchuravy @christiangnrd