← sources

HuggingFaceBio/Carbon-X (DNA understanding and generation, Qwen 3 finetune)

source: reddit (reddit:LocalLLaMA)published 2026-10-22 18:17ingested 2026-10-23 02:42

retained text

**Carbon-X** is a 14.8B-parameter decoder-only language model trained jointly on DNA sequences and natural-language text. You give it raw DNA inside an ordinary chat conversation and it answers in English: it predicts whether mutations are benign or pathogenic, identifies regions that control gene activity, classifies sequences by organism group, predicts the next bases of a sequence, and explains its reasoning. Responses can contain both DNA and natural language.

DNA and text share a single token stream in a single decoder. There is no separate DNA encoder. A hybrid tokenizer keeps a BPE vocabulary for text and encodes DNA as non-overlapping 6-mers, so one DNA token covers 6 base pairs.

Across five DNA benchmarks, which cover variant-effect prediction, sequence understanding and next-base prediction, Carbon-X averages **66.7%** with thinking disabled. That is 27 points above Qwen3-14B, the model it starts from, and ahead of every other open model we evaluated. It is close to GPT-6.1 Sol (67.5%) at about 1/1,800 of the inference cost per problem, and it leads every model we tested on coding-variant effects and next-base prediction ([Evaluation](https://huggingface.co/HuggingFaceBio/Carbon-X#evaluation)). Carbon-X answers directly, without extended reasoning, so each problem costs few output tokens. We estimate that analysing the 2.4 million variants of uncertain significance in ClinVar would cost about $300 with Carbon-X, compared with $32,000 to $4 million with the frontier models we evaluated. The weights fit on a single 80 GB GPU in bfloat16, so researchers can keep sensitive or unpublished sequences on their own infrastructure.

# Key features

* **14.8B parameters**: 40 layers, hidden size 5120, GQA (40 query heads, 8 KV heads), SwiGLU, RMSNorm with QK-norm.
* **Hybrid tokenizer**: the Qwen3 BPE vocabulary for text plus 5,515 DNA tokens. These are 4,096 6-mers, 1,364 tail tokens for sequences whose length is not a multiple of 6, and the `<dna>`, `</dna>` and `<oov>` markers. DNA in a prompt is detected and tagged automatically.
* **Chat model**: post-trained and evaluated with thinking disabled.
* **Context window**: 40,960 tokens.
* **Training**: 302B tokens of mid-training on DNA, biology text, English and instruction data, then supervised fine-tuning and reinforcement learning with verifiable rewards on DNA tasks.
* **Efficient inference**: serves with vLLM on a single GPU.

[https://huggingface.co/HuggingFaceBio/Carbon-X-GGUF](https://huggingface.co/HuggingFaceBio/Carbon-X-GGUF)

document fetched 2026-10-23 00:46 · canonical https://reddit.com/r/LocalLLaMA/comments/1xcqbsn/huggingfacebiocarbonx_dna_understanding_and