# ~1,400 t/s prefill is real. 60 t/s decode is not. I graded 5 Strix Halo forks with Qwen3.8 Flash-Next — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/ac0b3085c045295e

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-09-09 17:46 ingested 2026-09-10 02:42

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1wbsnqo/1400_ts_prefill_is_real_60_ts_decode_is_not_i/)

## retained text



```
# Strix Halo LLM Fork Comparison & Analysis

**TL;DR:** Most community debates focus on **decode speed** (token generation rate), but the true bottleneck on Strix Halo is **prefill speed** (processing large context). Additionally, **speculative decoding alters greedy outputs (`temp=0`)** across all tested GGUF builds due to GPU kernel batching variations.

---

## What to Run

| Goal | Recommended Engine / Stack | Why |
| :--- | :--- | :--- |
| **Daily Coding Agent** | [Nathan's strix-halo toolbox](https://github.com/Nathanw1014/strix-halo-llamacpp) (Vulkan v0.7.4.1) | Best agent benchmark scores ([neon-ladder](https://www.reddit.com/r/LocalLLaMA/comments/1w6cjm5/neon_ladder_a_playtestgraded_benchmark_for_local/): 15–18/19). Maintained, flexible, open GGUF build. *Turn off the drafter for exact reruns.* |
| **Huge Prompts / Long Reads** | [halogen 0.5.3](https://github.com/peonist-ai/halogen-flash-server) | Extreme prefill speed (1,424 tok/s at 32k vs ~433 tok/s on Nathan's). Passable quality, but closed-source, requires custom 126 GB format, and lacks JSON mode. |
| **Exact Deterministic Reruns** | Any build with **spec off**, or *halogen* | Speculative drafting changes `temp=0` token outputs across all tested GGUF builds. |
| **Dual GPU / R9700 Setup** | `sixvolts/llama-halo-hybrid` | Honest, verified numbers for dual-GPU hardware setups requiring a riser. |

---

## The neon-ladder Scoreboard

The [neon-ladder](https://github.com/aic0d3r/neon-ladder) benchmark is playtest-graded; a build only scores for contract features it actually ships on identical test setups:

| Stack | Static Score (of 19) | Laser + Obstacle Pad | Serve Speed | Daily Driver? |
| :--- | :--- | :--- | :--- | :--- |
| **Nathan's Toolbox** (Flash-Next) | **15–18** (medium band) | **Shipped** | **420 px/s** | **Yes** |
| **halogen** (4 builds tested) | 14, 14, 15, 15 | 0/4 each | Slow 4/4 (320 px/s) | **No** (Fast lane only) |

### Why Nathan's Toolbox Wins for Agents
- **Contract Feature Delivery:** Ships both required game features (laser powerup & obstacle pad) every run.
- **Flexibility:** Supports standard open GGUF weights, switchable speculative drafters, CPU-expert memory tuning, and desktop coexistence.
- **Why halogen Fails as a Daily Driver:** Across 4 builds, *halogen* never shipped either required feature, always defaulted to slow serve speed, lacks JSON mode (`response_format`), requires 126 GB custom weights, and offers no middle-memory tuning mode.

---

## Performance & Claims Graded

Tested on ROG Flow Z13 (Ryzen AI Max+ 395, 128 GB RAM, gfx1151, 70W power limit):

| Stack | Claimed Speed | Measured Decode (Serial / Drafted) | 32k Prefill Speed | Verdict / Status |
| :--- | :--- | :--- | :--- | :--- |
| **[halogen 0.5.3](https://github.com/peonist-ai/halogen-flash-server)** | ~50 decode / 1200 prefill | 33.2 / 43.1 tok/s | **1,350 tok/s** | Prefill reproduces; ~50 t/s was the drafted row. |
| **myhacsint** | ~60 decode / 600 prefill | 25.7 / 35.6 tok/s | 584 tok/s | Does not reproduce; matches toolbox performance at parity. |
| **strix-llama** | ~30 decode / 800 prefill | 14.0 / — tok/s *(thread)* | 397 tok/s | The 800 prefill claim used unverified draft-skip hacks. |
| **llama-halo-hybrid** | 45–60 decode (sustained 46) | *(Requires secondary GPU)* | — | Holds on dual-GPU setups; byte-identical output where checked. |
| **[Nathan's Toolbox](https://github.com/Nathanw1014/strix-halo-llamacpp)** | *(Untuned 2x baseline)* | 26.0 / 34.3 tok/s | 433 tok/s | **Wins agent bench** (15–18/19), but fails spec-identity gate. |

---

## Key Technical Discoveries

1. **Decode Speeds Are Misleading:** Variations in decode rates (e.g., 20 t/s vs 50 t/s) are driven by prompt content (prose vs. highly repetitive JSON) or whether drafting is enabled. Serial decode across all builds sits strictly between 14–34 t/s.
2. **Speculative Decoding Breaks Output Identity:** Enabling speculative drafting on GGUF builds fails deterministic `temp=0` identity gates (fails 3 to 9 out of 10 prompts). Loading the drafter alters Vulkan GPU batch execution and floating-point precision, causing coin-flip word choices (e.g., bolding different words in a sentence). Only *halogen* passed the exact-identity test (10/10).
3. **Prefill is the True Hardware Gap:** *halogen* achieves ~3.5x faster real-world prefill than standard llama.cpp forks (1,424 tok/s vs 433 tok/s at 32k context) due to hand-tuned ROCm GEMM kernels specifically compiled for Strix Halo. Standard llama.cpp Vulkan MoE paths remain shader-bound and leave matrix throughput unused.

---

## Verification Request for Fork Authors

Developers publishing speed claims are encouraged to report **Perplexity (PPL)** or **KL-divergence** scores alongside benchmark numbers:

```bash
# Get test dataset
./scripts/get-wikitext-2.sh

# Run PPL on GPU
llama-perplexity -m your-model.gguf -f wikitext-2-raw/wiki.test.raw -ngl 99

# Test deterministic output identity (serial vs spec)
llama-cli -m your-model.gguf -p "The capital of France is" -n 200 --temp 0 --seed 1 > serial.txt
llama-cli -m your-model.gguf -md your-drafter.gguf -p "The capital of France is" -n 200 --temp 0 --seed 1 > spec.txt
diff serial.txt spec.txt
```

---

## Reference Links & Resources

* **Benchmark Harness:** [neon-ladder Repo](https://github.com/aic0d3r/neon-ladder) | [Reddit Post](https://www.reddit.com/r/LocalLLaMA/comments/1w6cjm5/neon_ladder_a_playtestgraded_benchmark_for_local/)
* **Build Guides:** [Qwen3.8-27B Build Guide](https://www.reddit.com/r/LocalLLaMA/comments/1vsw6nz/qwen3827b_q5_k_xl_on_strix_halo_at_31_ts_decode/) | [Qwen3.8-Flash-Next Setup](https://www.reddit.com/r/StrixHalo/comments/1w6cf5t/qwen38flashnext_on_strix_halo_40_ts_sustained/)
* **Workflow & Testing:** [pi + llama.cpp Agent Setup](https://www.reddit.com/r/StrixHalo/comments/1w6c5nz/running_a_local_coding_agent_on_strix_halo_with/) | [DeepSeek V4 Flash](https://www.reddit.com/r/LocalLLaMA/comments/1vlmh0b/deepseek_v4_flash_0731_at_27_ts_decode_on_strix/) | [Benchmarking Methodology](https://www.reddit.com/r/LocalLLaMA/comments/1vduxth)
```



document fetched 2026-09-10 00:46 · canonical https://reddit.com/r/LocalLLaMA/comments/1wbsnqo/1400_ts_prefill_is_real_60_ts_decode_is_not_i
