24GB is plenty if your engines take turns
retained text
System service that watches your card, swaps and loads whatever you want, whenever you want. Don’t have to swap anything manually, collisions avoided, VRAM fully dedicated to each function while it’s running (no VRAM squatters). I had issues everyday (Linux) where a node process would orphan, comfyUI auto starts and holds 500MB that strata can’t claim. I also wanted to be able to switch natively and not have to think about starting each engine, making sure strata or llama wasn’t stuck loaded, or flux GGUF’s still claimed in comfyUI from an old workflow. I swap between Ninfer-4090 (llama-swap), strata, comfyUI, OpenWebUI, and I do it between multiple devices all using the same RTX 4090. Maybe you want to have your box run Hermes at night and you take the card back in the day, you can do it natively without thinking about and have full visibility on what was done and when. Berth will also serve your engines as an API so when you need to switch natively you load berth as the API endpoint and you can load strata, polystrata, and any other custom engine. Works for gaming as well, you don’t have to worry about a cron job, a request, or anything grabbing the card while you’re gaming or doing custom workflows. REPO: [https://github.com/T-Crypt/berth](https://github.com/T-Crypt/berth) EDIT: *Berth v0.3 is out, and I need testers with big RAM and more than one GPU* *v0.3 parks engines in system RAM instead of unloading them. A switch back to a parked engine skips the reload from disk. My workstation has 62 GB, and Strata alone pins 40 GB, so I can't measure parking at the sizes you run.* *I'm looking for:* *- 64, 96 or 128 GB of RAM with a 24 GB card. You'd park a vLLM or ComfyUI engine, switch away, switch back, and send me the times.* *- Two or more GPUs, any mix. Multi-GPU leases are the next release, and I want real topologies before I write them.* *- Windows with an NVIDIA card. One of you already runs it there. contrib/windows/check.ps1 compares Berth's per-process VRAM with Task Manager and writes a report.* *- AMD on Linux, for the release after that.*
document fetched 2026-10-21 00:41 · canonical https://reddit.com/r/LocalLLaMA/comments/1xahi82/24gb_is_plenty_if_your_engines_take_turns