← sources

M6 Mac Mini owners (32GB): now you can run Qwen3.8-27B in high quality on the Apple Neural Engine

source: reddit (reddit:LocalLLaMA)published 2026-10-24 21:05ingested 2026-10-25 02:39

retained text

ANEMLL Forge 0.2.1: Qwen3.8-27B on the M6 Neural Engine now runs up to 192K context (M6 32GB).

New: INT8 K/V cache (half of FP16) and a smaller drafter sharing the target's output head — 0.86 GB less ANE memory, same speed.

Decode depends on content: JSON 35–62 tok/s, code 18–32, prose 10–17. Prefill 260 tok/s at short context, 62 at 192K. 128K–192K use a smaller prefill batch (32 vs 64 rows) to stay within memory budget.

Heads up: full 192K first compile takes \~2h on M6 (one time, cached). \`--ctx\` exports only the entries you need, so compile is much faster: 16K is \~10 min from download to running server. IMO 96K is the sweet spot for size/speed on 32GB: 123 tok/s prefill, \~27 tok/s decode, \~21.4 GB total memory.

Release notes: https://github.com/Anemll/anemll-forge/blob/main/docs/RELEASE\_NOTES\_v0.2.1.md

Video (Tetris generated over \~150K tokens, no compaction): https://x.com/anemll/status/2111941341819408866

Paper: Three Bit MLP Vector Quantization in Apple Neural Engine — https://github.com/Anemll/anemll-forge/blob/main/docs/papers/anemll-forge-v02.pdf

Cheers,
Anemll 🤖

document fetched 2026-10-25 00:36 · canonical https://reddit.com/r/LocalLLaMA/comments/1xejv61/m6_mac_mini_owners_32gb_now_you_can_run_qwen3827b