Revert "ggml-cuda : restore prop.integrated on HIP builds (#24233)" - #28604
Merged
Conversation
github-actions
Bot
added
ggml
pwilkin
approved these changes
Sep 8, 2026
taronaeo
approved these changes
Sep 8, 2026
Patt92
pushed a commit
to Patt92/llama.cpp
that referenced
this pull request
Sep 9, 2026
Upstream d4389a4 (ggml-org#28604) reverted prop.integrated for all HIP builds over the corrupted output in ggml-org#15034. On gfx1151 that revert is very expensive, and b10931 shipped it. The part has no VRAM carveout to speak of: mem_info_vram_total reports 0.5 GB and the whole model lives in GTT. With integrated == false, ggml_backend_cuda_device_supports_buft refuses host buffers, so the scheduler keeps a device copy of memory the GPU could have addressed in place. Measured on a 124 GB node with GLM-5.3-Flash and Qwen3.8-Flash-Next co-resident, same configuration either side: b10907 (integrated true) 82 GB used, 42 GB available, no swap b10931 (integrated false) 119 GB used, 4 GB available, 8 GB swap The node then thrashes, and prefill goes with it: within one 46k-token prompt individual segments ran at 325 t/s and at 12.6 t/s depending on whether the ubatch touched a swapped-out page. Restored for RDNA3.5 only, so ggml-org#15034 is not reopened on the architectures it was reported on. The exemption is not a guess: this branch carried prop.integrated from upstream c7d8722 (2026-07-16) through b10907 across GLM, Qwen3.8, DeepSeek-V4 and Ornith with no corrupted output on gfx1151. The assignment sits after the HIP block that parses cc, not next to upstream's line, because cc is not known yet at that point. Verified on gfx1151, with the node still swapping: test-backend-ops 15026/15026 with no failures, test-llama-archs OK for glm5next (6.20e-14), qwen4exp (1.01e-13) and deepseek4 (2.73e-12).
x1250
pushed a commit
to x1250/llama.cpp
that referenced
this pull request
Sep 9, 2026
…4233)" (ggml-org#28604) This reverts commit c7d8722.
zbrad
pushed a commit
to zbrad/llama.cpp
that referenced
this pull request
Sep 10, 2026
…4233)" (ggml-org#28604) This reverts commit 68d90ba.
stew675
added a commit
to stew675/llama.cpp
that referenced
this pull request
Sep 11, 2026
On the 050dde5 baseline this block reverted PR ggml-org#24233 (restore prop.integrated on HIP builds): integrated=true enables the CUDA host-buffer path (zero-copy UMA weights) on APUs, which corrupts full-model results under async execution on this box (PPL 5.9243 -> 8.51+ without HIP_LAUNCH_BLOCKING). Re-based 2026-09-08 onto master 9113cc1: upstream itself reverted ggml-org#24233 in ggml-org#28604 (2026-09-08), making forced-integrated-false the upstream default. The functional delta of this block is therefore now upstream; the block is retained so the fork keeps its 14-block structure and carries only the rationale marker comment above the (now unconditional) integrated = false line, guarding against re-landing prop.integrated without re-testing this box.
tungel
pushed a commit
to tungel/llama.cpp
that referenced
this pull request
Sep 11, 2026
On the 050dde5 baseline this block reverted PR ggml-org#24233 (restore prop.integrated on HIP builds): integrated=true enables the CUDA host-buffer path (zero-copy UMA weights) on APUs, which corrupts full-model results under async execution on this box (PPL 5.9243 -> 8.51+ without HIP_LAUNCH_BLOCKING). Re-based 2026-09-08 onto master 9113cc1: upstream itself reverted ggml-org#24233 in ggml-org#28604 (2026-09-08), making forced-integrated-false the upstream default. The functional delta of this block is therefore now upstream; the block is retained so the fork keeps its 14-block structure and carries only the rationale marker comment above the (now unconditional) integrated = false line, guarding against re-landing prop.integrated without re-testing this box.
pwilkin
added a commit
to pwilkin/llama.cpp
that referenced
this pull request
Sep 12, 2026
Reapplies ggml-org#24233, reverted upstream in ggml-org#28604. The corrupted output that motivated disabling the flag (ggml-org#15034) is the graph input aliasing race the scheduler ring buffer in this branch fixes: an asynchronous backend could still be reading a graph input when the next ubatch overwrote it. With the ring in place an APU can report itself integrated again, which keeps model inputs in host-visible memory instead of copying every one of them into VRAM that is the same physical DRAM. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
woct0rdho
pushed a commit
to woct0rdho/llama.cpp
that referenced
this pull request
Sep 13, 2026
Reapplies ggml-org#24233, reverted upstream in ggml-org#28604. The corrupted output that motivated disabling the flag (ggml-org#15034) is the graph input aliasing race the scheduler ring buffer in this branch fixes: an asynchronous backend could still be reading a graph input when the next ubatch overwrote it. With the ring in place an APU can report itself integrated again, which keeps model inputs in host-visible memory instead of copying every one of them into VRAM that is the same physical DRAM. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This reverts commit c7d8722. (#24233)
see #27311 (comment)
Requirements