← sources

Smaller RAM DGX Spark alternative?

source: reddit (reddit:LocalLLaMA)published 2026-09-02 15:34ingested 2026-09-03 02:43

retained text

Howdy,

I was wondering if anyone knew of any turnkey low-power draw solutions to host inference with 10-20GB of VRAM?

I have an 4x3090 AI GPU cluster that I'm running big models on, but I am hosting my memory system LLMs (embeddings, rankings, etc) on my gaming 4090 right now. I'm out of PCIE lanes on my AI machine and really don't want to build a whole other system (cpu, ram, mobo, gpu, etc) just to run about 15-20GB of small models.

I want to keep things local so a VPS is off the table, and I think just running these off the CPU would be too slow although speed isnt that important for Hindsight? I was wondering if anyone knew if there was anything like the spark that is considerably cheaper (\~1k or so?) for this type of use or other systems that required only a little tinkering to get to work like this? Thanks!

  
For reference, this is what I'm running:

Qwen 3 Embedding-8B-Q6\_K at 9216 context x 4 @ 6.21 GB for embeddings

BGE-reranker-large-8\_0 at 512 context x 4 @ 604MB  for rankings

Qwen 3.5-9B-UD-Q3\_K\_XL at 64k context x 4 @ 5.98 GB for synthesis/consolidation

document fetched 2026-09-03 00:43 · canonical https://reddit.com/r/LocalLLaMA/comments/1w5drkz/smaller_ram_dgx_spark_alternative