# KRAFTON/Raon-1-nano · Hugging Face — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/1dd7ee29edb78371

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-11-11 16:31 ingested 2026-11-12 02:42

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1xtmbkn/kraftonraon1nano_hugging_face/)

## retained text



```
Raon-1 nano is a dense language model with 1.5B effective parameters (3.1B total, including a 1.5B per-layer embedding table), developed by [KRAFTON AI](https://krafton.ai/en/). It is designed to **follow instructions, maintain conversational context, and call the right tools in both Korean and English**. As part of the Raon open model family, it is released under the **Apache 2.0 license**.

# [](https://huggingface.co/KRAFTON/Raon-1-nano#highlights)Highlights

* **Faithful, natural conversation**: Leads the comparison models on the instruction-following, context-following, and Korean conversation benchmarks shown below.
* **Korean and English**: Trained on Korean and English, with solid English capabilities alongside its Korean strengths.
* **Tool calling**: Tool calling within conversations, for everyday tasks such as office work and mobile control.
* **On-device deployment**: Available in GGUF, MLX, and LiteRT-LM formats for local inference, including on Apple silicon.
* **Long context**: Natively supports up to 262,144 tokens of context.

# Model Overview

Raon-1 nano uses a hybrid dense architecture, consisting of 18 linear attention layers and 6 full attention layers.

* **Raon Delta Attention (RDA):** Gated delta-rule linear attention with separate erase and write gates and a fixed-size recurrent state.
* **Raon-mHC:** Raon-mHC reduces the computational cost of DeepSeek's manifold-constrained hyper-connections (mHC), which speeds up training.
* **Per-layer embedding:** Each layer has its own small 256-dimensional embedding for every token. After its MLP, the layer uses it to adjust its output before passing it on.
* **Hybrid layout:** 18 of the 24 layers are Raon Delta Attention layers with a fixed-size state. Only the 6 MLA layers keep a KV cache, about 7 KiB per token in BF16.
* **Tied embeddings:** The LM head shares the input embedding.

|Field|Value|
|:-|:-|
|Architecture|Dense|
|Total Parameters|3.1B (including the 1.5B per-layer embedding table)|
|Layers|24|
|Hidden Size|1,536|
|Attention|Hybrid (RDA 3 : Gated MLA 1)|
|Position Encoding|NoPE (no positional encoding)|
|Number of Attention Heads|16 for RDA; 32 for Gated MLA|
|Attention Head Dimension|128 for RDA; 192 QK / 128 V, for Gated MLA|
|Query / KV LoRA Rank|512 / 512, for Gated MLA|
|FFN Intermediate Size|6,144|
|Activation Function|SwiGLU|
|Residual Connection|Raon-mHC w/ 4 streams|
|Per-Layer Embedding|256 per layer|
|Vocabulary Size|259,840|
|LM Head|Tied|
|MTP|1 layer, 55M|
|Context Length|262,144 tokens|

The checkpoint on HuggingFace contains 3.2B parameters: 3.1B for the main model and 55M for the Multi-Token Prediction (MTP) module.

# [](https://huggingface.co/KRAFTON/Raon-1-nano#training)Training

* **MoE-to-Dense:** Raon-1 nano was derived from Raon-1 mini preview (30B-A4B) through [MoE-to-Dense](https://arxiv.org/abs/2605.28207) pruning and knowledge distillation.
* **Continued pre-training:** 3T tokens.
* **Post-training:** supervised fine-tuning (SFT) and on-policy distillation (OPD) from Raon-1 mini preview, followed by reinforcement learning (RL).

# Quantizations:

* [https://huggingface.co/KRAFTON/Raon-1-nano-gguf](https://huggingface.co/KRAFTON/Raon-1-nano-gguf)
* [https://huggingface.co/KRAFTON/Raon-1-nano-mlx](https://huggingface.co/KRAFTON/Raon-1-nano-mlx)
* [https://huggingface.co/KRAFTON/Raon-1-nano-litert-lm](https://huggingface.co/KRAFTON/Raon-1-nano-litert-lm)
```



document fetched 2026-11-12 00:44 · canonical https://reddit.com/r/LocalLLaMA/comments/1xtmbkn/kraftonraon1nano_hugging_face
