I don't care what some ppl say, I like Strata!
retained text
I'm now getting 3x the speed than with llama.cpp (Unsloth fork): 10.52.637.962 I slot print\_timing: id 0 | task 0 | prompt eval time = 620987.80 ms / 38761 tokens ( 16.02 ms per token, 62.42 tokens per second) 10.52.637.966 I slot print\_timing: id 0 | task 0 | eval time = 15443.97 ms / 292 tokens ( 53.07 ms per token, 18.84 tokens per second) 10.52.637.967 I slot print\_timing: id 0 | task 0 | total time = 636431.76 ms / 39053 tokens Strata (same Hermes session, loaded with Strata and a turn after the previous one with llama.cpp): \[strata\] reading the prompt: 8,192 of 38,968 tokens, 5 s so far \[strata\] reading the prompt: 16,384 of 38,968 tokens, 9 s so far \[strata\] reading the prompt: 18,720 of 38,968 tokens, 11 s so far \[strata\] reading the prompt: 26,912 of 38,968 tokens, 16 s so far \[strata\] reading the prompt: 35,104 of 38,968 tokens, 20 s so far \[strata\] reading the prompt: 38,963 of 38,968 tokens, 23 s so far \[strata\] thinking: 2 of max 92096 tokens, 36.7 tok/s, 24 s \[strata\] thinking: 48 of max 92096 tokens, 44.8 tok/s, 25 s \[strata\] thinking: 94 of max 92096 tokens, 44.7 tok/s, 26 s \[strata\] thinking: 144 of max 92096 tokens, 46.1 tok/s, 27 s \[strata\] thinking: 212 of max 92096 tokens, 51.2 tok/s, 28 s \[strata\] thinking: 273 of max 92096 tokens, 52.8 tok/s, 29 s \[strata\] thinking: 342 of max 92096 tokens, 54.8 tok/s, 30 s \[strata\] thinking: 407 of max 92096 tokens, 56.0 tok/s, 31 s \[strata\] thinking: 463 of max 92096 tokens, 55.9 tok/s, 32 s \[strata\] thinking: 519 of max 92096 tokens, 55.8 tok/s, 33 s \[strata\] thinking: 578 of max 92096 tokens, 56.1 tok/s, 34 s \[strata\] answering: 651 of max 92096 tokens, 57.5 tok/s, 35 s \[strata\] answering: 716 of max 92096 tokens, 58.1 tok/s, 36 s \[strata\] answering: 769 of max 92096 tokens, 57.6 tok/s, 37 s \[strata\] done: 799 tokens in 37 s (57.4 tok/s) (stop, cancel=False), expert cache 94.5% hit (+1.3% of the routed experts over PCIe) For PP llama.cpp took 10 mins, Strata 23 seconds, and I get 3x the TG speed... I have no idea how is that possible, but I've been using it for about 10 days, every day and results are consistent (actually latest updates made it faster, because I was getting "only" about 45 t/s TG). And I load both with the very same files: qwen3.8-flash-next-unsloth-ud-q4\_k\_xl and full KV (fp16) and 130k context (110k with llama.cpp), and even with "--kv-resident", "32768" speed barely drops above that. I love llama.cpp and I'm starting to love Strata as well! Thank you Strata!
document fetched 2026-10-14 00:42 · canonical https://reddit.com/r/LocalLLaMA/comments/1x4qrxq/i_dont_care_what_some_ppl_say_i_like_strata