is it worth dedicating a gpu to an llm that sits idle 95% of the time?
retained text
got a 3060 12gb in the main box, next to the nas, home assistant and a few docker services. card was half for running a local model on the small stuff. log and alert summaries, a notes chat endpoint, home assistant voice. all light. answers in a second when i hit it, but that's maybe twenty times a day. rest of the time it sits there warm, burning watts. doesn't justify a gpu idling round the clock. three ways i see it: always loaded and eat the idle draw, load on demand and eat the cold-start hit, or drop the card and run small models on cpu per service. been poking at that last one. model running in-process through nobodywho, loads a gguf, no inference daemon. really curious how do you run yours?
document fetched 2026-09-24 00:40 · canonical https://reddit.com/r/homelab/comments/1wo0tfs/is_it_worth_dedicating_a_gpu_to_an_llm_that_sits