Local Ai Pc 7663 Dual Epyc / Dual 9709
retained text
CPU Information Name AMD EPYC 7663 Topology 2 Processors, 112 Cores, 224 Threads Memory Information RDiMM 2933Mhz Size 1007.61 GB System Information Operating System Ubuntu 24.04.5 LTS Motherboard Giga Computing MZ72-HB2-00 GPUs 2x ASUS Turbo R9700 AI Pro 32 GB newest BIOS low Fan Profile (throttled to 210w currently) ROCm version: 7.2.3 Beside that baby I have a Strix Halo M5 128Gb Now wanted to setup that big boy for local llm For myself want to use it for coding . But I am also looking for something where I can also easily switch Model within the UI . Can Openwebui reload the model and what I read is that vllm is best for tensor split formte two cards . Also want to use it for family for image creation within the ui and also Image checking kind of Allrounder as chatgpt Is that possible with vLLM? I am also looking for docker setups so i can keep my host clean Thank you for you suggestions
document fetched 2026-10-04 00:36 · canonical https://reddit.com/r/LocalLLaMA/comments/1wwjrmw/local_ai_pc_7663_dual_epyc_dual_9709