ggml-alloc : share meta compute buffers through a borrower alias

[Piggidragon]

Overview

--phase-aware-workspace with an MTP draft crashes under --split-mode tensor on the first target decode:

ggml/src/ggml-backend.cpp:336: GGML_ASSERT(tensor) failed
  ggml_backend_tensor_set
  ggml_backend_meta_buffer_set_tensor
  ggml_backend_sched_compute_splits

The target and MTP contexts share one resizable compute buffer per buffer type (#36). Under -sm tensor that buffer is a meta buffer. A meta buffer maps each tensor to its per-device tensors, and it keeps that map for one graph at a time: ggml_backend_meta_graph_compute rotates and clears it when a new graph runs. The sequence that breaks:

1. target computes
2. MTP allocates and computes in the same meta buffer, which clears the target mappings
3. target reuses its graph, so nothing re-inits its tensors, and set_inputs gets nullptr

The first commit kept meta buffers out of the shared workspace. That fixed the crash, but it gave up the sharing under tensor split, which is the point of the phase-aware workspace.

The second commit shares the memory again. Only the tensor map can not be shared, the simple per-device buffers can:

• ggml_backend_meta_buffer_alias creates a second meta buffer over the same simple buffers, with its own tensor map and split state cache. It does not own the simple buffers.
• The shared entry keeps one alias for the borrower. The owner uses the real buffer. The alias is freed before the buffer when the entry is resized, and made again after.
• A resize already bumps the shared generation, so neither context reuses a graph on a freed alias.

Testing

Qwen3.8-27B-UD-Q4_K_XL, RTX 3060 + RTX 4070, --device CUDA1,CUDA0 -ts 0.55,0.45 --spec-type draft-mtp --phase-aware-workspace -np 2 -c 16384 -ub 512, KV on GPU. Peak VRAM sampled with nvidia-smi every 200 ms over load and one request:

	result	peak CUDA0	peak CUDA1
llama/dev (1a04ed9), -sm tensor	GGML_ASSERT(tensor) on first decode	-	-
first commit (private meta buffers), -sm tensor	ok, 47.5 t/s, 73.9 % draft acceptance	8479 MiB	10553 MiB
this PR, -sm tensor	ok, 48.2 t/s, 75.5 % draft acceptance	8349 MiB	10423 MiB
this PR, -sm layer	ok, 44.1 t/s, 73.0 % draft acceptance	9169 MiB	9473 MiB

• The alias saves 130 MiB per GPU against private meta buffers at this config.
• -sm tensor with a 2568-token prompt and a follow-up request: ok, the workspace switches prefill/decode several times with the draft attached, draft acceptance 44 % and 90 %.
• test-alloc: test_resizable_buffers_owner_borrower_meta_alias runs owner compute, borrower alloc + compute, then owner compute on the reused graph over a two-device CPU meta backend. Without the alias it fails with the same GGML_ASSERT(tensor) as the server. The rest of test-alloc passes.
• test-llama-archs --test-phase-workspace: passes.
• Host KV (--no-kv-offload --kv-cpu-pinned --recurrent-state-offload --kv-gpu-layers 0) under -sm tensor hits a different assert in warmup on this base, GGML_ASSERT(i_start == cgraph->n_nodes), before MTP runs. ggml-meta : split a host-resident KV cache by head #66 fixes that.

Tested with #66 merged on top (local branch, not part of this PR), same model and devices, -sm tensor --spec-type draft-mtp --phase-aware-workspace -np 2 -c 16384 plus the host KV flags above. Target KV 544 MiB and draft KV 64 MiB on CUDA_Host, recurrent state 673 MiB on Meta(), draft borrows the target workspace:

request	result
64-token prompt	ok, 44.3 t/s, 77.5 % draft acceptance
2568-token prompt	ok, prefill 488 t/s, decode 22.2 t/s, 40.7 % draft acceptance
short follow-up	ok, 43.7 t/s, 76.2 % draft acceptance

test-alloc, test-llama-archs --test-sched-copy-name and --test-phase-workspace pass on the merged build.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES - root cause and fix found and implemented by an agent on my instruction, see the Assisted-by: commit trailers.

🤖 Generated with Claude Code

https://claude.ai/code/session_016qzvRnpuN7dvnXNzK1F8dV