Smaller RAM DGX Spark alternative?
retained text
Howdy, I was wondering if anyone knew of any turnkey low-power draw solutions to host inference with 10-20GB of VRAM? I have an 4x3090 AI GPU cluster that I'm running big models on, but I am hosting my memory system LLMs (embeddings, rankings, etc) on my gaming 4090 right now. I'm out of PCIE lanes on my AI machine and really don't want to build a whole other system (cpu, ram, mobo, gpu, etc) just to run about 15-20GB of small models. I want to keep things local so a VPS is off the table, and I think just running these off the CPU would be too slow although speed isnt that important for Hindsight? I was wondering if anyone knew if there was anything like the spark that is considerably cheaper (\~1k or so?) for this type of use or other systems that required only a little tinkering to get to work like this? Thanks! For reference, this is what I'm running: Qwen 3 Embedding-8B-Q6\_K at 9216 context x 4 @ 6.21 GB for embeddings BGE-reranker-large-8\_0 at 512 context x 4 @ 604MB for rankings Qwen 3.5-9B-UD-Q3\_K\_XL at 64k context x 4 @ 5.98 GB for synthesis/consolidation
document fetched 2026-09-03 00:43 · canonical https://reddit.com/r/LocalLLaMA/comments/1w5drkz/smaller_ram_dgx_spark_alternative