Caphlon-Model: I pruned and vocabulary-shrunk Qwen 3.8 27B into a 6.87 GiB GGUF. Runs fully on 8GB RAM
retained text
I wanted to share **Caphlon-Model**, a compact build of **Qwen 3.8 27B** that fits into a single **6.87 GiB GGUF file** and runs smoothly on an 8 GB RAM laptop. Instead of traditional student-teacher distillation, I cut representation redundancy directly from the 27B topology to keep its reasoning core intact. **How it was made:** * **Vocabulary Surgery:** Reduced the merge table to a 100k target. This saved \~649 MiB but made it strictly an **English-only** model. * **Layer Pruning:** Removed 4 redundant layers (layers 12–15), saving \~302 MiB. * **Quantization:** Formatted in Unsloth Dynamic **IQ2\_S** GGUF. **Benchmarks (Tesla T4, llama.cpp):** * **HumanEval:** 29/30 (96.7% pass@1 retention) * **MMLU Subset:** 344/502 (92.9% retention) * **English Perplexity:** 6.6088 (vs 6.3638 base) and Github : https://github.com/demiralpdev/Caphlon-Model WebSite : [https://caphlon.dev/](https://caphlon.dev/) huggingface : https://huggingface.co/ssuasd/Caphlon-Model
document fetched 2026-10-30 00:44 · canonical https://reddit.com/r/ollama/comments/1xim7lj/caphlonmodel_i_pruned_and_vocabularyshrunk_qwen