Built an assistant with 5080, 5060ti and 2x3090?
retained text
Slow news day.. Core: **Hardware** * **Home PC (WSL2 + Docker Desktop):** RTX 5080 16 GB and RTX 5060 Ti 16 GB * **3090 server:** 2× RTX 3090 24 GB * **VPS:** hosts the web chat (`ivy-chat`) (docs) **IVY's brain (the language models)** * **Local:** Qwen3.8-27B (Unsloth UD-Q8\_K\_XL) on both 3090s, served by llama-swap * **Cloud:** DeepSeek (the default for retries and side calls), plus Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5 * **Engine (Harness):** the`scripts` bash CLI tool-loop. The web gateway (`ivy-web-gateway.mjs`, Node) connects the web chat and the Voice tab to it. **Memory** * **CAGRA** (NVIDIA cuVS GPU vector index) in `cagra-builder`, with SQLite `rag.db` as the store. The index and reranker sit on the 5080. * **Embedder:** Qwen3-Embedding-8B Q8\_0 on a patched llama.cpp server, on the 5060 Ti * **Reranker:** Qwen3-Reranker-4B Q8\_0 on llama.cpp, on the 5080 * **Size:** about 198,700 facts, 73,300 of them in the active index **Voice and hearing (PC)** * **Voice:** Breeze TTS 2, int4 (`ivy-tts`), on the 5060 Ti * **Ears:** Qwen3-ASR (`ivy-asr`) * **Speaker ID:** NVIDIA TitaNet (`ivy-speaker`, CPU), which tells who is talking **Face (3090)** * **Motion:** voice-to-motion model `p4_v7`, built on Ditto, turns her voice into face motion * **Head:** 3D Gaussian-splatting head `c2_breath_c`, about 338,000 Gaussians * **Painter:** R3 (a U-Net plus the full-resolution stage), redrawing every frame in H3's painted style * **Delivery:** H.264 video stream to the Voice tab **Creative and teacher models on the 3090 (llama-swap, loaded on demand)** * MiniMax H3: the teacher videos the face learns from * MOSS-Image, Qwen-Image, P-JiT 21K: images * MOSS-SoundEffect, MiniMax Music: sound effects and music * DreamX-Creator: video * SoMark OXR: turning documents into text * Older or idle voice lanes: VibeVoice, MOSS-TTS, Breeze lanes, plus Gemma and Qwen-Next
document fetched 2026-11-18 00:46 · canonical https://reddit.com/r/LocalLLaMA/comments/1xynh14/built_an_assistant_with_5080_5060ti_and_2x3090