Two GPUs, multiple models, which ones should get a dedicated card?
retained text
We’re a team of 3 sharing one box with two GPUs, like 24GB card and a 16GB card and we need to run several models: a couple of 7B ones plus a few smaller low-traffic ones. I’m considering to try GPUStack for unified deployment and GPU allocation, and real question is how to split the two cards. heres my two option. Option A: pin one GPU per model,each model gets its own card, clean isolation, no contention. Option B: pack the low-traffic small models together onto one card and leave the other card for the main model. That stretches the hardware further,but those small models are VRAM and compete when they all get hit at once. So just curious what do people actually do here? if u’ve got only two cards but the team needs several models,how do you split them?
document fetched 2026-11-08 00:26 · canonical https://reddit.com/r/selfhosted/comments/1xpyyfq/two_gpus_multiple_models_which_ones_should_get_a