← sources

I ran Qwen3.5-0.8B in pure Assembly

source: reddit (reddit:LocalLLaMA)published 2026-11-08 15:39ingested 2026-11-09 02:37

retained text

On an AMD EPYC 7B12, it generated 6.51 tokens/sec on one CPU thread. The Transformers reference reached 4.34.

All 32 generated tokens matched in both runs.

Code: https://github.com/suayptalha/qwen3\_5-assembly

X Post: https://x.com/suayptalha/status/2119254536283906205

Tokenization, model inference and sampling all run in Assembly. No libc or Python at runtime; Python prepares the tokenizer tables at build time.

Benchmark: 19 input tokens, greedy decoding, 2 runs. Assembly reads BF16 weights; Transformers loaded them as FP32 and used PyTorch fallback kernels.

document fetched 2026-11-09 00:36 · canonical https://reddit.com/r/LocalLLaMA/comments/1xr18t3/i_ran_qwen3508b_in_pure_assembly