My Local ML Dev Lab
retained text
The goal with this lab is to give me the ability to run AI/ML workloads locally so I can more readily play and experiment. I'm a SWE in the AI space, so having something like this is super convenient. My company has massive cloud compute resources that I have access to, but for me, the convenience and hands on nature of a home lab is hard to beat. Everything lives in a 12U TecMojo 10" rack, apart from the GPU tower next to it. | Position | Hardware | What it does | |---|---|---| | On top | GL.iNet travel router | Repeater bridging the rack to the house network | | 1 | TP-Link 8-port gigabit switch | | | 2 | Blank panel | | | 3 | 12-port patch panel | | | 4 | Lenovo ThinkCentre Tiny | Future k3s worker, with two more on the way | | 5–6 | Gigabyte AI TOP ATOM (GB10) | Runs half of a large mixture-of-experts model, linked to the Spark below over a direct 200G QSFP cable | | 7–8 | NVIDIA DGX Spark (GB10) | The other half of that model | | 9–10 | 2x Intel NUC (i7-7567U), one behind the other | k3s control plane, plus a dev box and CI runner | | 11–12 | ElecVoztile 1U PDU | | | Beside the rack | Thermaltake tower with an RTX 3090 Ti | Runs a 27B model for the smaller jobs | | On the floor | 1500VA pure sine wave UPS | Feeds the PDU and the tower | Everything runs on k3s. Argo CD syncs from a self-hosted Gitea, and Gitea Actions builds the images. LiteLLM gives all the models one API, Prometheus and Grafana handle monitoring, and a self-hosted AI agent does a lot of the day-to-day operations.
document fetched 2026-10-31 00:39 · canonical https://reddit.com/r/homelab/comments/1xj0vvw/my_local_ml_dev_lab