Repository navigation
Conversation
…ilds Signed-off-by: Thatgfsj <thatgfsj2324@gmail.com>
Thatgfsj
force-pushed
the
docs/windows-hip-rocm-sdk
branch
from
October 10, 2026 14:10
2596840 to
d8737f1
Compare
Collaborator
|
Thanks for writing this up. The runtime notes are really useful, especially the silent CPU fallback and the
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Fills the Windows gaps in
docs/build.mdHIP section, from a from-source deployment on Windows 11 / RX 7800 XT (gfx1101) last week:x64 Native Tools Prompt+%HIP_PATH%\binfrom an installer) is not reachable for exactly the cards most people want to use this backend on. The documented flow mirrors this fork's ownwindows-rocmrelease job step-by-step (rocm[libraries,devel]wheels,rocm-sdk init,rocm-sdk path --root/--cmake/--bin, the env vars, and the same cmake flags), including the MSVC-triple prerequisite.ggml-hip.dllneeds resolvable next to the exe (amdhip64_7.dll->rocm_kpack.dll+amd_comgr.dll;hipblas.dll->rocsolver.dll; hipBLASLt per-ISA kernel data dirs), and where pip SDK keeps them;LoadLibrarysilently - the backend never registers and inference quietly runs on CPU at 1-2 t/s, easily mistaken for a slow GPU;llama-server --list-devicesis the discriminator;fattn-mmatemplate instances are multi-GB each:--parallel 24on 64 GB RAM aborted withclang frontend command failed due to signal(compiler OOM, not a source error); 6-12 built clean.Verification
9a9394a89: full PQ2_0 Bonsai 27B + MTP serving path on gfx1101 - pp512 461 t/s, tg128 49.6 t/s, operator checks green. Details: Eval bug: Bonsai 2 PQ2_0 produces corrupt logits on Windows HIP gfx1151; CPU --no-repack is coherent #223 (comment)--list-devices: (none)) and disappeared once the DLL set was colocated.--parallel 24twice and built clean at--parallel 6.Docs only; no code touched. Happy to adjust tone/placement - e.g. split the runtime notes into
docs/backend/if maintainers prefer.