Liked Muse, so I cut the 30B model in half by width, distilled it back, and it does 57 of 60 tool tasks its parent does 60 of
retained text
I've liked how Muse-Glimmer worked, so I wanted to see if I could produce a smaller "kid" out of it. Ornith's sharp decisions on when to think and which tool to call were the other thing I liked, so Ornith-1.0-9B got to be the policy teacher while the parent wrote the words. No RL anywhere, distillation only. I present to you **Xyntetik-Kvist-14B**. **What it is good for** * Smaller than the Muse parent but still manages most tool tasks: 57 of 60 held-out closed-loop tasks (contacts, weather, flights, currency, dates, units, stock), scored by re-executing the calls against ground truth, where the parent does 60. * Fits a 24 GB card whole at Q8_0 (15.4 GB) or the Q5_0 mix (10.3 GB) and serves an OpenAI-, Anthropic- and Responses-compatible API through Xyntetik Runner, so it drops into an agent loop you already have. * Every failed attempt is published beside it: 12 gated runs, 2 full passes, one shipped. The training record has the preregistrations, the amendments and the defects, so you can see exactly where it breaks before you build on it. * Give it a calculator tool for arithmetic. Without one it gets "17% of 2,340" wrong, and the card says so. **Numbers, from the card** | claim | number | |---|---| | parent | Muse-Glimmer-30B, cut by width (hidden 6,656 to 5,760, FFN 19,968 to 10,240, heads 32 to 24), all 52 layers kept | | size | 14.44 B parameters; BF16 28.9 GB, Q8_0 15.4 GB, Q5_0 mix 10.3 GB | | distillation | 6,000 steps, 98.3 M tokens, 162 hours, then 1,440 steps on agentic trajectories | | fidelity to parent | KLD 0.762, margin-qualified top-1 84.0% on 45,056 held-out positions (a student's row, not the quant bar) | | tool tasks | 57 of 60 held-out, re-executed against ground truth; parent 60, untrained control 0 | | format and calls | 199 of 200 first turns well formed; 99 of 99 tool calls valid | | attempts | 12 gated attempts, 2 full passes, attempt 12 shipped | | weak spot | calc tasks 12 of 15 over 160; 7 of 160 runs end in a reasoning loop | | serving | Runner v0.5.7 or later | | licence | Apache-2.0 | **Links** * Model: https://huggingface.co/Joakimpalm-Zen/Xyntetik-Kvist-14B * GGUF: https://huggingface.co/Joakimpalm-Zen/Xyntetik-Kvist-14B-GGUF * Training record: https://huggingface.co/datasets/Joakimpalm-Zen/Xyntetik-Kvist-14B-training-record * Collection: https://huggingface.co/collections/Joakimpalm-Zen/xyntetik-kvist-distilled-agents-6ab9f930af19ca84df9ebf6b * Runner v0.5.7: https://github.com/Joakimpalm-Zen/xyntetik-runner/releases/tag/v0.5.7 EDIT: reading the comments, i should have said this first. this is not a general drop-in for Muse or a gemma4 replacement, and it was never going to be on my compute (98M distillation tokens vs the trillion a real distill wants, i simply lack the compute). the purpose was more on getting the tool calling right. IQ4_NL mix (7.6 GB) is up now too, it scores the same 57/60 on the tool tasks but does not fit an 8 GB card whole (50/52 layers on a 3070, ~5 tok/s).
document fetched 2026-09-29 00:43 · canonical https://reddit.com/r/LocalLLaMA/comments/1ws6bmh/liked_muse_so_i_cut_the_30b_model_in_half_by