← sources

My Local ML Dev Lab

source: reddit (reddit:homelab)published 2026-10-30 00:19ingested 2026-10-31 02:39

retained text

The goal with this lab is to give me the ability to run AI/ML workloads locally so I can more readily play and experiment.

I'm a SWE in the AI space, so having something like this is super convenient. My company has massive cloud compute resources that I have access to, but for me, the convenience and hands on nature of a home lab is hard to beat.

Everything lives in a 12U TecMojo 10" rack, apart from the GPU tower next to it.

| Position | Hardware | What it does |  
|---|---|---|  
| On top | GL.iNet travel router | Repeater bridging the rack to the house network |  
| 1 | TP-Link 8-port gigabit switch | |  
| 2 | Blank panel | |  
| 3 | 12-port patch panel | |  
| 4 | Lenovo ThinkCentre Tiny | Future k3s worker, with two more on the way |  
| 5–6 | Gigabyte AI TOP ATOM (GB10) | Runs half of a large mixture-of-experts model, linked to the Spark below over a direct 200G QSFP cable |  
| 7–8 | NVIDIA DGX Spark (GB10) | The other half of that model |  
| 9–10 | 2x Intel NUC (i7-7567U), one behind the other | k3s control plane, plus a dev box and CI runner |  
| 11–12 | ElecVoztile 1U PDU | |  
| Beside the rack | Thermaltake tower with an RTX 3090 Ti | Runs a 27B model for the smaller jobs |  
| On the floor | 1500VA pure sine wave UPS | Feeds the PDU and the tower |

Everything runs on k3s. Argo CD syncs from a self-hosted Gitea, and Gitea Actions builds the images. LiteLLM gives all the models one API, Prometheus and Grafana handle monitoring, and a self-hosted AI agent does a lot of the day-to-day operations.

document fetched 2026-10-31 00:39 · canonical https://reddit.com/r/homelab/comments/1xj0vvw/my_local_ml_dev_lab