Built a local AI rig with 4× RTX 3060s—working with what I had
retained text
Built this setup using hardware I already owned and parts I could afford. It’s a bit unconventional, and I’ve gotten a few comments making fun of it, but I’m proud of getting it working. Figured some people here might appreciate the project. **Specs:** * Ryzen 7 3700X — 8 cores / 16 threads * 4× RTX 3060 12GB — 48GB total VRAM distributed across the cards * ASUS TUF Gaming X570-PLUS Wi-Fi * 16GB DDR4 * WD\_BLACK SN850X 2TB NVMe * Corsair RM1000e 1000W PSU * Open-air frame, two ribbon cables and two powered PCIe x1 risers * Ubuntu, llama.cpp, and a custom local AI interface I’ve been building called MRCV **What it actually runs:** * Qwen 14B Q4: around 35 tokens/sec * Qwen 27B Q4: around 18.4 tokens/sec without speculative decoding * Qwen 32B Q4: around 15.8 tokens/sec * Llama 3.3 70B Q4: 7.6 tokens/sec, tested at 4K context * Qwen 27B processed an 8K-token prompt at roughly 984 tokens/sec , that’s prompt processing, separate from generation speed. I also tested two versus four cards. Generation speed barely changed on the models that fit on two, but the extra memory let me run 70B. Beyond chatting, I’ve connected it to my Mac so it can inspect projects, edit files, look up documentation, and run tests. The 27B with speculative decoding completed one small coding task in about 14 seconds and passed all 12 checks. Bigger projects are still a work in progress—it makes mistakes, and getting the agent behavior reliable has been its own learning experience. This is a hobby project. I’ve learned a lot about Linux, CUDA, quantization, splitting models across GPUs, and figuring out where the bottlenecks actually are. There’s something satisfying about getting a model this large running on hardware you put together yourself. Anyone else running local AI on a collection of parts rather than a purpose built workstation?
document fetched 2026-10-29 00:38 · canonical https://reddit.com/r/homelab/comments/1xi1fvw/built_a_local_ai_rig_with_4_rtx_3060sworking_with