Consumer Radeon · living notes
Local Qwen on a real R9700. Not a brochure.
amdai is the independent notebook for home-lab AI on Radeon. Which GGUF, which engine, whether it fits your VRAM — grounded in a two-card test rig, written for people who buy one GPU.
Our test rig is two AMD Radeon AI PRO R9700 cards (gfx1201, 32 GB each) and 256 GB DDR5. The daily driver uses one card.
What works this week
Snapshot 2026-10-12 · viable
Daily driver is Qwen3.6-35B-A3B (UD-Q5_K_XL) on one R9700 via llama.cpp Vulkan. Fast agent is Qwen 3.8 Flash on Strata.
Consumer north star: run local Qwen on Radeon, not a data-center brochure. Daily driver on our test rig — one Radeon AI PRO R9700 (32 GB, gfx1201), Qwen3.6-35B-A3B Unsloth UD-Q5_K_XL, llama.cpp Vulkan, 262144 context, f16 KV, 0% system RAM offload. Measured on our rig, 2026-10: UD-Q5_K_XL 100.8 tok/s short / ~88 at 32K / ~50 at 256K; Q5_K_M 108.3 tok/s (28.5 GB VRAM); Q4_K_M 116.5 tok/s decode, 1814 tok/s prefill at 32K. Fast agents: Qwen 3.8 Flash Unsloth UD-Q6_K_XL on Strata v0.1.42 at 262K (Measured on our rig, 2026-10: ~32.6 tok/s vs llama.cpp ~16 tok/s); llama.cpp Q6 is the fallback. Prefer llama.cpp over vLLM at home. Prefer Q5 over Q4. Never go below Q4. Load GGUFs from fast NVMe. One LLM per GPU unless you deliberately combine. vLLM 0.31 is a serve try, not the daily path. Abliterated weights stay educational, never the default.
- ROCm
- AMD ROCm Core SDK 10.0.0 / current 10.1 matrix line — one RDNA4/gfx120x family package (R9700 = gfx1201; official support confirmed)
- Daily inference
- llama.cpp Vulkan on gfx1201 (HIP build is fine if it configures). Prefer that over official Ollama ROCm and over vLLM for a single-user home box.
- Known good (this week)
- One R9700 + llama.cpp Vulkan + Qwen3.6-35B-A3B UD-Q5_K_XL at 262K f16 KV (~50 tok/s at 256K; Measured on our rig, 2026-10). Strata + Qwen 3.8 Flash UD-Q6_K_XL for agents. vLLM 0.31 = try, not the house default.
- AMD-supported combo
- ROCm 10.0 AI ecosystem matrix — vLLM 0.27 + PyTorch 2.13; gfx1201 listed. 10.1 matrix lists the same Radeon rows.
- Serving
- Consumer default is llama.cpp. Fast-agent try on our rig is Strata v0.1.42. Cutting-edge vLLM try is community/ROCm 0.31 (0.30 previous; AMD-validated 0.27/0.29 still the conservative pin on the 10.0 matrix).
- Qwen (reliability first)
- Qwen/Qwen3.6-35B-A3B · GGUF unsloth/Qwen3.6-35B-A3B-GGUF (file Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf; publisher listing 26.6 GB)Quant · UD-Q5_K_XL (Q5 family). Prefer Q5 over Q4. Floor is Q4_K_M. Never below Q4.Serve · llama.cpp Vulkan (or HIP gfx1201). Measured on our rig, 2026-10 — UD-Q5_K_XL 100.8 / ~88 / ~50 tok/s. TODO: exact command line / UD-Q5_K_XL VRAM from rig owner.
This block is driven by src/data/what-works.yaml. Cards: stacks· calculator· GPU directory. Historical notes live under ROCm and the blog.
From the rig
Tested stacks, measured where we have numbers
Daily driver
Qwen3.6-35B-A3B · UD-Q5_K_XL
llama.cpp Vulkan on one R9700. 262K context, f16 KV, 0% RAM offload. 100.8 tok/s short context, ~88 at 32K, ~50 at 256K — Measured on our rig, 2026-10.
Measured on our rig, 2026-10100.8 / ~88 / ~50 tok/s
Fast agent
Qwen 3.8 Flash · UD-Q6_K_XL
Strata v0.1.42 at 262K. ~32.6 tok/s vs llama.cpp ~16 on our rig — first-party, labeled.
Tested on our rig~32.6 tok/s · Strata
First-party
Benchmarks
How to read pp/tg, then the wtpai1 table: 35B llama.cpp and Flash Strata vs llama.cpp. Empty cells stay not measured.
Measured on our rig, 2026-10No leaderboard spam
The rest of the lab
Directory, troubleshooter, forum, ROCm
Hardware
GPU directory
Radeon cards, gfx IDs, VRAM, official ROCm rows. No invented support claims.
When it breaks
Troubleshooter
Paste a log. Cited patterns for ROCm, Ollama, HIP, and reset hangs.
Community
Forum
Categories, rig badges, and solved marks. Off on this static build until FORUM_ENABLED.
The stack
What is ROCm
Plain-English deep dive: the stack, gfx IDs, Vulkan vs HIP, versions, history, glossary, FAQ.
Consumer house rules
How we pick a stack
- 01Prefer llama.cpp over vLLM for a home box. vLLM is a serve try, not the daily driver.
- 02Prefer Q5 over Q4. Q4_K_M is the floor. Never Q1, Q2, Q3, or IQ1–IQ3 — including draft / MTP heads.
- 03Keep GGUFs on fast NVMe. Do not run a 20-plus-GB file from a spinning disk.
- 04One LLM per GPU unless you deliberately combine (tensor/layer split).
- 05Daily driver on our rig uses 0% system RAM offload. Offload is slow — treat it as a last resort.
Tutorials
Do the work
- Install ROCm on a home Linux box
ROCm 10.1.0 on a Radeon AI PRO R9700 — official Ubuntu 26.04.1 / 24.04.5 / 22.04.5 rows, ROCm CLI v0.2.0, and a do-not-mix-apt decision tree.
- Verify GPU and ROCm
Prove the R9700 is visible, that HIP works, and that PyTorch is using the card — before you download a 20 GB model.
- Run a local Qwen (llama.cpp)
Build llama.cpp first — Vulkan is the daily-driver path on our rig, HIP if rocminfo is clean — then run official Qwen instruct.
- Serve Qwen on the R9700
Official Qwen instruct on a loopback /v1 — llama-server first, Strata for Flash, vLLM only in Advanced.
Dated notes
ROCm / vLLM snapshots
Oct 12, 2026
Consumer stacks — Qwen3.6 daily driver, Flash agent, calculatorNorth star is now the two stacks from our test rig. llama.cpp Vulkan daily driver; Strata for Flash; /calculator and /stacks are first-class pages.
Oct 12, 2026
ROCm 10.1.0 notes, CLI v0.2.0, error paste, version matrix10.1.0 lists R9700 as gfx1201 and Radeon Ubuntu 26.04.1 / 24.04.5 / 22.04.5. Published ROCm CLI is v0.2.0, not a 1.0.0 tag. New pages for search, errors, backends, quants, RSS.
Sep 22, 2026
vLLM ROCm Docker try-tag v0.30.0Dated snapshot. AMD-validated vLLM 0.27 / ROCm 10.0.0 unchanged. Community image moved to vllm/vllm-openai-rocm:v0.30.0. Named Qwen defaults from that week were later withdrawn.