← sources

How can I use LFM 3 1.7B at high concurrency locally?

source: reddit (reddit:LocalLLaMA)published 2026-10-28 18:15ingested 2026-10-29 02:42

retained text

I'm trying to get a swarm of these agents to be able to do larger tasks when orchestrated so that I can try to use cloud models less, but I find that even with `--parallel 16`, after like 6 sequences running concurrently it just doesn't do any more. In addition, when I start another request, every other instance goes to about 1 tok/s at the same time until that request finishes prefill.

Hardware: 16 GB RAM + 8 GB VRAM (RTX 4060 Laptop)  
OS: Windows 11  
Command: `llama serve --hf-repo LiquidAI/LFM3-1.7B-GGUF --cache-type-k q8_0 --cache-type-v q8_0 --parallel 16 --kv-unified`

document fetched 2026-10-29 00:44 · canonical https://reddit.com/r/LocalLLaMA/comments/1xhvwzu/how_can_i_use_lfm_3_17b_at_high_concurrency