what's the best model/harness currently for 16GB vram?
retained text
The answer to these kind of qns changes frequently, but right now with so many qwen variants as well as new class of inference servers like strata/freetoken it seems we are seeing something special. there seem to be 3 different vectors for optimizing throughput 1. the model - DASLab's GSQ-RCO, Swift etc were to my knowledge the first models to run well enough on 16GB without noticeable degradation, according to multiple posts here. And good enough for agentic coding with real world context size 2. inference optimizations - things like dflash2, mtp, kvarn, kv streaming etc, and special chat templates 3. inference servers - Then came strata and now freetoken which showed that llama.cpp left gpu idle a lot, and used smarter caching etc. and there was ninfer, nvfp4 etc before that. Harness - Pi is I think the most popular, but there are many others like deepseek harness, less known ones like [Maki](https://github.com/tontinton/maki) So right now, for users with <= 16GB vram, esp CUDA, what is the best we have? I'm assuming the best in class local model for coding is still Qwen 3.8 27B or Flash Next.
document fetched 2026-10-12 00:37 · canonical https://reddit.com/r/LocalLLaMA/comments/1x3fwgd/whats_the_best_modelharness_currently_for_16gb