I’ve been doing a lot of writeups about the Strix Halo and various tunings and wins I’ve managed to eke out of it, because it’s a relatively low-cost and accessible system and it’s just fun. That said, obviously there is a HUGE pile of things I tried on the Strix Halo box that didn’t pan out and I think there’s some interesting learnings in that too.

So here we go, a list of failures sorted by how they failed, because on this hardware the how is the useful part.

The failures modes fall into three buckets:

  • Type 1, it ran and was slower or worse.
  • Type 2, it silently did nothing, which is the failure mode that ate the most days.
  • Type 3, it worked and I measured it wrong.

Reminder, because it drives everything below: this box is not bandwidth-starved in the abstract, it is bandwidth-pinned. I measured decode reading memory at ~95% of the read ceiling (242.7 of 256 GB/s). We cannot stream faster. So every real win has to move fewer bytes, and most of what follows is me learning that (over and over again) the hard way.

Contents

Type 1: it ran, and it was worse

The first entry is a bit of a cheat, it never even got to run, but it’s the wall the whole project is pinned against, so it goes first.

Memory and fabric overclock: architecturally locked

The obvious first move on a bandwidth-pinned box is to raise the memory clock. You can’t. Infinity Fabric on this part is rate-matched to the memory clock, there’s no independent FCLK knob, and memory frequency is SKU-binned at 8000 (AMD ships 8533 as separate part numbers). I went as far as the AMI/AGESA BIOS unlock that others have tried on 395 boxes; it fails, and recovery needs a hardware SPI programmer. The clocks were already pinned at max, not thermally throttled. Revive this only with different silicon, an 8533 SKU or an LPDDR6 next-gen part. On this box it’s closed.

rocWMMA flash-attention regresses past 32k

The community-tuned ROCm numbers looked great, so I chased a ROCm + rocWMMA flash-attention build. It helps at short context and, by community guidance and my own A/B against RADV, gives ground back at depth past ~32k, which is exactly the range this whole project cares about. The strixhalo.wiki maintainers now say it outright: as of ROCm 7.0.2+ the rocWMMA path shouldn’t be used with upstream llama.cpp, it’s slower than the plain ROCm/HIP path as context grows. RADV/Vulkan was the faster path at depth in my runs, so the ROCm detour was a net loss for the long-context case. The tuned 13.3 tok/s-at-130k number floating around comes from a patched branch (lhl’s Strix Halo benchmarking), not a stock build, which is worth knowing before you go replicate it.

MALL residency is too small to matter

gfx1151 has a 32MB L3/MALL that is genuinely strong when your working set fits, 3.4-9.6x DRAM bandwidth. A 35B MoE’s working set does not fit in 32MB. Measured, cache-residency bought about 1.1x on decode, and it’s conditional and it costs you recall to arrange. The cache is real and the win is not, for this model. This could be fun to revisit with smaller models, however! I really wanted to find a way to make this one work out.

KV-cache quant tax

q4_0 KV quant hurts prefill at mid context on this box (131k: 393 tok/s vs f16’s 473, a 0.83x loss), because the dequant overhead outweighs the bandwidth it saves at that depth. It flips to a win only at extreme context (256k: q4_0 240 vs f16 223, 1.08x). And once you’re running the sliding-window recipe, quant loses again (256k: q4_0 767 vs SWA-f16 855, 0.90x), because the window already bounded the KV to a size where there’s no bandwidth pressure left for quant to feed on. SWA and KV-quant are substitutes, not complements, you pick the bigger lever, you don’t stack them. I spent… a while trying to stack them.

Type 2: it silently did nothing

This is the long one, and the pile that cost me the most days, because a silent no-op is the only failure that doesn’t have the decency to announce itself. The run finishes, nothing errors, and the numbers come back looking exactly as plausible as a real result. You can burn a whole experiment on one and not know until you go digging.

The “content-aware” sparse mask was never content-aware

The tell, in hindsight, is embarrassingly clean. I had a tau threshold knob on the selector, and sweeping it across its entire range (0.01 to 0.99) moved neither prefill speed nor perplexity by a hair. At a 16k window every setting landed on ~753 tok/s and PPL 1.3512; at a 2k window every setting landed on ~903 tok/s. The absolute numbers move with the window, but tau itself was inert at both. A knob that changes nothing isn’t tuning anything, so my content selector was selecting no content. The reason is a softmax over all blocks peaked so hard on recent tokens that far-back salient blocks never clear any threshold worth setting, which quietly collapsed the whole thing into a plain sink-plus-sliding-window. I chased that for a week before I caught it, and the full autopsy (the mechanism, the max-pool swap that changed nothing, the DeltaNet layers that were actually carrying the needle) is in the 256k post. The Type 2 lesson is small and I won’t forget it again: if a feature has an off switch, run it. If “off” looks identical to “on,” it was never on.

The ggml step() bug: the mask that never masked

This one lived underneath the last one, and it came first: before my mask was even a lazy sliding window, it was nothing at all. It leaned on ggml’s step(), and step(0) disagrees with itself across backends: 0 on CPU, 1 on Vulkan. On the GPU path that meant my mask never dropped a tile, flash-attention never skipped anything, and I ran a whole recall curve that was silently dense. I chased it down, filed it as issue #25027 (accepted and fixed upstream), and wrote the whole saga up separately, so I’ll spare you the forensics here. It earns a mention because it’s the purest Type 2 specimen I have: my code was correct, the framework disagreed with itself at exactly one input value, and the only symptom was “the optimization does nothing,” with no error anywhere.

The additive-mask hack was O(N^2) in bookkeeping

This one ran, and ran correctly, which is exactly why it lands in this bucket: the mask masked, the outputs were right, and the speedup it was supposed to buy quietly never showed up. The tell was that it kept eroding the deeper I went. A sliding window applied as an additive -inf mask still makes flash-attention walk every query-tile / key-tile pair and pay a per-tile check on the all-masked tiles it then throws away: O(N) in real attention work, O(N^2) in tile bookkeeping, and at depth the bookkeeping is most of the bill. The 256k post has the ladder; the lesson for this list is that “it’s masked” and “it’s cheap” are separate claims, and a mask riding on top of a dense kernel only ever buys you the first one. The actual win needed native sliding-window attention with a bounded rolling KV cache (flat 850-890 tok/s from 32k to 1M, a 32x range that sags 4.8%), which is O(N) for real. The mask was a scaffold that looked like the answer.

Env-gated patches silently no-op on the wrong binary

This is the one I’d carve into the case. HOMER has two llama-server binaries: the system one at /usr/local/bin (no patches) and my build-vk/bin build (the SWA and per-layer-quant patches). My patches are environment-gated, so on the unpatched binary they don’t error, they just quietly do nothing, n_swa stays 0 and the model runs dense. I ran a windowing sweep against the system binary once and got a clean, plausible, completely dense curve, and nothing in the output said a word. The only defense is behavioral: an out-of-window needle must miss, and if it doesn’t, either you’re on the wrong binary or the flag never took. I hit this same wall on the 256k work too, which is why “it ran” stopped counting as “it ran patched” for me. Every grid now starts with a needle the feature is supposed to drop, and if it survives, I throw the grid out before I read a single number.

Kernel-level knobs that measured to nothing

A cluster of small ones, grouped because they share a shape: I turned them on, measured, and found no signal. Transparent hugepages, a measured no-op (the GART already uses large pages). amd_iommu=off, neutral, because iommu=pt overrides it and there was near-zero headroom anyway. Borrowing gfx1100 kernels via HSA_OVERRIDE_GFX_VERSION=11.0.0 looked incredible on a GEMM microbench (5.6x) and produced no end-to-end LLM win I could measure. Microbenchmark speedups that don’t survive contact with a real forward pass are their own kind of silent nothing.

Type 3: it worked, I read it wrong

The NIAH “5/5” that was measuring thinking budget

qwen3.6 is a reasoning model, and for a while I measured needle-in-a-haystack recall as “did it answer within the -n token budget,” not “could it retrieve.” Those aren’t the same thing. A too-short budget makes the model look like it failed to recall when it just ran out of room to think, and a generous one flatters recall for reasons that have nothing to do with the retrieval mechanism I thought I was testing. Several of my early clean 5/5 numbers were contaminated by this, including a “win4096 5/5 but +74% PPL” that I’d been treating as a real recall win. I now measure recall with greedy decode and a stop token, deterministic and exact, and I re-ran the suspect numbers.

The temp-0 degeneracy attractor that fooled me three times

The single most expensive misread of the project. At temp-0 greedy decode, deep-context generation on this model falls into a degeneracy attractor, it starts emitting bare “3” or single-character garbage, and it does this non-monotonically: a keep-budget sweep goes hit / miss / hit / miss with no pattern, and the baseline itself flips run to run. I read this as “eviction is dropping the needle” at least three separate times before pinning it. It is not an eviction failure, the scorer keeps the needle at global rank #1; it is a decode-side collapse from attention-sink deprivation, and bumping temperature to 0.3 partially stabilizes it. I burned days attributing a generation-side attractor to my byte-reduction code because the symptom (wrong answer at depth) looked identical to the thing I was afraid of.

The NPU that was the right answer to the wrong model

Not a mistake so much as a scope misread. The NPU’s flat-prefill win is real, I measured it, on a dense proxy model (Qwen3-8B), where dense attention collapses with context and the NPU’s steady prefill overtakes the iGPU around 16-32k. My actual hero model is an MoE whose iGPU prefill is robust and defers that collapse past the range I care about, so the crossover never appears and the NPU stops being a lever. The measurement was right; treating it as a general win for this model was the error. But turn that around, because it’s not a dead end so much as an open door I walked past. On a dense long-context model the crossover is real: the NPU genuinely overtakes the iGPU on prefill past ~16-32k, and the only reason I didn’t chase it further is that my hero model defers the collapse the NPU was going to rescue me from. The stack is already standing on HOMER. So if you’re running a dense model at depth on one of these boxes, this is the thread I’d pull, and I’d love to hear where it lands, because I think there’s a real win sitting there that I just wasn’t the right person to find.

The pattern

The Strix box is pinned at the bandwidth wall, so saving bytes is the only thing that helps, and I spent most of these dead-ends discovering that a thing I thought was saving bytes wasn’t doing anything at all.

  • A mask that masked nothing.
  • A patch on the wrong binary.
  • A kernel knob that microbenched beautifully and moved no real tokens.

The dangerous failures aren’t the slow ones, they’re the ones where the run completes, the numbers look plausible, and you’re one behavioral check away from finding out you measured a no-op. The single habit that came out of all of this: before trusting any result, prove the thing you think you turned on is the thing that ran. An out-of-window needle has to miss. Or you’re measuring dense and calling it a win.