loading
loading
Fish Audio's Realtime STT Claims Top Accuracy at $0.36 per Hour Fish Audio's new streaming speech-to-text claims the top spot on Cekura's leaderboard with 2.21% WER and conversation-aware turn-taking for voice agents. - Fish Audio launched a preview of Realtime STT, a streaming speech-to-text model built for voice agents. - Claims #1 of 20 on Cekura's Converse-STT benchmark with 2.21% overall word error rate. - Priced at $0.36 per session hour, billed per second of session duration. - Served as transcribe-stream-1 over a WebSocket endpoint at wss://api.fish.audio/v1/asr/realtime. - Pitch is conversation-aware turn-taking, deciding when the caller is done rather than relying on VAD timeouts. - Positions Fish Audio as a single-vendor voice stack alongside its existing S2 Pro and Drama 3 TTS models. Fish Audio has released a preview of Realtime STT, a streaming speech-to-text model designed for voice agents and live conversations. The system transcribes incoming audio and identifies when a speaker has finished a turn, giving an agent a signal to begin generating its response. The preview places Fish Audio alongside Deepgram, AssemblyAI, Soniox, and OpenAI in the market for real-time automatic speech recognition. Fish claims a 2.21% overall word error rate on Cekura’s STT leaderboard, which would rank first among the models cited in the benchmark. Inside the preview Fish exposes the model as transcribe-stream-1 through a WebSocket connection, according to its model documentation. Applications send live audio to the endpoint and receive transcription results as the conversation progresses. The existing transcribe-1-pro model continues to cover recordings of up to 60 minutes. Developers moving from file transcription to the streaming API will need to manage a persistent connection, incremental results, turn events, reconnects, and session state. How the price compares Provider pricing can cover different features, connection rules, and billing units, so the hourly figures require verification against current plan terms. Fish describes billing by session duration, making idle-connection charges and timeout behavior relevant to production cost estimates. Turn detection is the real pitch Streaming speech recognition usually produces partial text before finalizing an utterance. A separate voice activity detector, or VAD, watches for speech and silence. Mid-sentence pauses can trigger premature replies, while conservative silence thresholds add a noticeable delay after the caller finishes. Fish describes Realtime STT as conversation-aware, combining transcription with a decision about when the speaker has yielded the turn. The company has not documented the underlying method, so developers should evaluate the behavior as a product signal rather than assume a specific VAD or semantic-detection architecture. Fish says the combined signal can reduce response lag without depending on a long silence timeout. A useful evaluation should measure each claimed outcome separately: - Premature turn-end events during pauses, hesitations, and filler words - Delay between the end of speech and the final turn signal - Behavior during interruptions and overlapping speech - Accuracy and stability of final transcripts sent to the language model Putting the 2.21% WER in context Cekura calculates word error rate, or WER, from substitutions, deletions, and insertions relative to a reference transcript. Lower scores indicate fewer transcription errors, and a 2.21% aggregate WER corresponds to roughly 2.21 errors per 100 reference words in the evaluated set. The leaderboard uses 1,000 short public clips from the Pipecat dataset and eight licensed recordings drawn from four phone conversations. The public clips emphasize acoustic recognition across many samples; the licensed Ocular recordings add longer, conversational phone audio. That second group is useful for voice-agent testing, although its small size makes per-call results and error distribution important. Fish’s claimed lead over Zoom is 0.14 percentage points. Model updates, benchmark settings, language mix, audio quality, and endpoint configuration can move rankings by more than that margin, so production decisions need workload-specific tests. Latency data will determine whether the accuracy result translates into a responsive agent. Cekura reports several measurements that Fish has yet to provide for the preview: - Final-text delay: The interval before finalized text becomes available. The cited leaderboard reports a 38 ms median for Soniox and 123 ms for Cartesia Ink 2. - Time to first text: The delay before an application receives an initial partial transcript. AssemblyAI Universal 3.6 Pro is listed at 485 ms. - Ocular WER: Accuracy on the licensed phone-call subset, which may better represent support and sales calls. - Tail latency: P95 and P99 results reveal slow outliers that median measurements can hide. How it fits into a voice agent A conventional voice agent sends audio through three sequential systems, with each stage adding processing and network latency: - The STT service converts live speech into partial and final text. - A language model generates a response from the finalized user turn. - A text-to-speech service synthesizes the response for playback. Turn finalization gates the second step in many implementations. Faster detection can shorten the silent gap before a reply, provided the model does not increase false turn endings and force the agent to interrupt callers. Likely workloads - Customer-support and sales agents built with frameworks such as LiveKit or Pipecat - Meeting transcription systems that need reliable turn boundaries - Phone-based interactive voice systems with low response-latency targets - Multilingual assistants, subject to confirmation of Realtime STT’s language coverage Production details remain open The preview materials leave several implementation questions unanswered, including language coverage, streaming diarization, and the client-side representation of a completed turn. Fish’s file-based transcribe-1-pro supports speaker diarization and inline emotion tags, but the company has not documented those features for transcribe-stream-1. Teams evaluating the API should confirm the following details against the current preview documentation: - Supported codecs, sample rates, channel layouts, and audio chunk sizes - Event schemas for partial text, final text, and turn completion - Configurable confidence thresholds and silence settings - Maximum session length, concurrency limits, and rate limits - Reconnect, retry, backpressure, and transcript-resumption behavior - Language detection, custom vocabulary, diarization, and emotion metadata - Billing treatment for silence, idle connections, and failed sessions - Data retention, regional processing, and enterprise privacy controls A practical benchmark candidate Fish Audio is primarily known for text-to-speech products. The company reports that S2 Pro ranked first in its own blind TTS comparison with a Bradley-Terry score of 3.07. Adding streaming STT gives Fish a broader voice-agent offering and may simplify vendor management for teams already using its speech synthesis API. A representative bake-off should replay consented production-like audio and compare Fish with the incumbent provider using overall WER, domain-specific term accuracy, false turn endings, missed interruptions, P50 and P95 finalization latency, connection failures, and total session cost. Shadow traffic can also expose behavior under packet loss, background noise, accents, and overlapping speech before the model handles live conversations. The 2.21% benchmark claim and $0.36 hourly rate make transcribe-stream-1 a plausible evaluation candidate. Production adoption will depend on its latency distribution, turn-event reliability, language support, streaming feature set, and performance on each team’s actual call audio.
document fetched 2026-10-28 02:52