← sources

Jev for speech

source: reddit (reddit:deeplearning)published 2026-11-17 11:04ingested 2026-11-18 02:37

retained text

Marimba is Jev for audio: a System One model for speech. You ask typed questions about the last 8 seconds of audio and get typed answers back: a yes/no, a choice, or a calibrated score.

- Is someone speaking? Did the turn end? Which language? Did someone say "refund"? Did the speaker change?
- One pass of a frozen Qwen3-ASR-0.6B encoder serves every question, with small readouts on top. Streams every 250 ms.
- Runs locally on CPU or GPU. Apache-2.0, `pip install "marimba-audio[torch]"`.
- Plugs into Pipecat, LiveKit and Vapi as a turn detector.

```python
from marimba import Marimba, Question

model = Marimba.from_pretrained(backend="torch", device="cpu", dtype="float32")
questions = [
    Question.score("Has the speaker finished?"),
    Question.choice("What language?", ["en", "pt", "es"]),
]
for d in model("clip.wav", questions):
    print(d.task, d.value, d.probability)
```

On 1,000 paired public questions against much bigger audio LLMs (Qwen3-Omni-30B-A3B, Qwen2-Audio-7B, FireRedAudio 10.6B), Marimba wins binary term detection on Common Voice and FLEURS, and turn end (0.935 AUROC vs 0.54–0.79). That last one is in-distribution for us, since we trained on Smart Turn data. It loses on Speech Commands, and Smart Turn v3.2 still beats it on its own test set (0.982 vs 0.930). Full table on the model card.

- Live demo: https://huggingface.co/spaces/charcot-ai/marimba
- Model + comparison table: https://huggingface.co/charcot-ai/marimba
- Code: https://github.com/Charcot-AI/marimba-api
- Paper: https://arxiv.org/abs/2611.18330

Happy to answer questions. Feedback on which questions you'd want it to answer is very welcome.

document fetched 2026-11-18 00:24 · canonical https://reddit.com/r/deeplearning/comments/1xyhnhn/jev_for_speech