← sources

is it worth dedicating a gpu to an llm that sits idle 95% of the time?

source: reddit (reddit:homelab)published 2026-09-23 09:08ingested 2026-09-24 02:39

retained text

got a 3060 12gb  in the main box, next to the nas, home assistant and a few docker services. card was half for running a local model on the small stuff. log and alert summaries, a notes chat endpoint, home assistant voice. all light.

answers in a second when i hit it, but that's maybe twenty times a day. rest of the time it sits there warm, burning watts. doesn't justify a gpu idling round the clock.

three ways i see it: always loaded and eat the idle draw, load on demand and eat the cold-start hit, or drop the card and run small models on cpu per service.

been poking at that last one. model running in-process through nobodywho, loads a gguf, no inference daemon.

really curious how do you run yours?

document fetched 2026-09-24 00:40 · canonical https://reddit.com/r/homelab/comments/1wo0tfs/is_it_worth_dedicating_a_gpu_to_an_llm_that_sits