[PR] Xing4.0-29B-A4B GGUF via ik_llama.cpp — Full native mHC/MLA runtime + HF converter
retained text
Hey everyone, China Telecom (TeleAI) recently released Xing4.0-29B-A4B (a 29.5B MoE with \~4B active parameters, combining DeepSeek-style MLA and 4-stream mHC residual mixing): Model: [https://huggingface.co/XingChen-AGI/Xing4.0-29B-A4B](https://www.google.com/url?sa=E&q=https%3A%2F%2Fhuggingface.co%2FXingChen-AGI%2FXing4.0-29B-A4B) Upstream llama.cpp PRs are currently stalled or closed due to debates around adding model-specific GGML opcodes. To get the model running cleanly without adding technical debt to the engine, I've opened PR #2808 on ik\_llama.cpp providing full native runtime support and an HF-to-GGUF converter: PR: [https://github.com/ikawrakow/ik\_llama.cpp/pull/2808](https://www.google.com/url?sa=E&q=https%3A%2F%2Fgithub.com%2Fikawrakow%2Fik_llama.cpp%2Fpull%2F2808) What's implemented in this PR: Full Architecture Runtime (LLM\_ARCH\_XING4\_0): * 4-stream mHC: built entirely using standard generic GGML primitives (pre/post blocks, affine transforms, clamp \[-30, 30\], Sinkhorn iterations). Zero new ggml\_op added, so it runs out of the box on CPU, CUDA, Metal, and Vulkan. * Stream pooling: unweighted average across the 4 streams before output norm. * MLA attention: DeepSeek-V2 style latent attention with dynamic head sizing. * MoE FFN: dense lead layers + routed experts + shared expert. * Node budget: bumped max\_nodes (n\_tokens \* 160) to prevent graph overflow with large ubatches (ubatch >= 2048). HF to GGUF Converter: * Adds Xing4Model to convert\_hf\_to\_gguf.py with native SentencePiece extraction from tokenizer.model. * Direct tensor mapping for mHC weights. Quantization: * Automatically protects hc\_attn\_fn and hc\_ffn\_fn from quantization in llama-quantize, and forces 1D scales/bases to F32. * Quantization floor: keeps MLA latent projections at source precision for Q6\_K and Q8\_0 to avoid quality loss. Verification: * Perplexity: 12.4190, generation is coherent with no NaNs. * Full chat template included with thinking tags (<think>...</think>) and tool calling. Quick test from the branch: git clone [https://github.com/ikawrakow/ik\_llama.cpp](https://www.google.com/url?sa=E&q=https%3A%2F%2Fgithub.com%2Fikawrakow%2Fik_llama.cpp) cd ik\_llama.cpp git fetch origin pull/2808/head:xing4 && git checkout xing4 cmake -B build -DGGML\_NATIVE=ON cmake --build build --config Release -j$(nproc) python3 convert\_hf\_to\_gguf.py path/to/Xing4.0-29B-A4B --outtype f16 --outfile xing4-f16.gguf ./build/bin/llama-cli -m xing4-f16.gguf -p "Hello!" -n 64 Feel free to test it out and report back if you get benchmarks on different setups!
document fetched 2026-10-31 00:44 · canonical https://reddit.com/r/LocalLLaMA/comments/1xjqsb4/pr_xing4029ba4b_gguf_via_ik_llamacpp_full_native