Concepts
Choose the right speech workflow by input state, result timing, and output type.
Start with two questions: is the input audio or text, and—if it is audio—is the whole unit already available? Those answers choose the processing model. Transport and SDK client come afterward.
Choose the processing model
| Starting point | Choose | Behavior | Decision boundary |
|---|---|---|---|
| A complete long-form or large recording | Batch transcription | Upload once, receive a jobId, and poll a job | Use for meetings, podcasts, archives, and other work where asynchronous completion is acceptable. |
| A complete audio unit whose latency matters | Fast transcription | Send the whole encoded audio unit over Socket.IO, then receive result events | Use for bounded full-audio work such as one agentic or conversational turn. It is not live audio streaming and is not the long-recording path. |
| Audio that is still arriving | Realtime transcription | Send PCM16 chunks and receive provisional and final text while the stream is open | Use for calls, captions, microphones, and live media pipelines. |
| A need to know who spoke when | Diarization | Produce speaker time segments alongside batch audio or from a live PCM stream | Add it to the matching batch or live workflow; it does not produce the transcript itself. |
| Text that must become audio | Text-to-speech (TTS) | Send text and receive streamed raw PCM samples | Use when speech is the output rather than the input. |
The decisive distinction is the input boundary. Batch and fast transcription both start with complete audio; fast result events do not make the input realtime. If audio is still being produced, choose realtime.
Batch is a job
Batch submission returns jobId. Poll the result route until a terminal state
or until the app's overall deadline expires.
| Status | State | App decision |
|---|---|---|
queued | Non-terminal | Wait, then poll again within the deadline. |
processing | Non-terminal | Keep waiting within the same deadline. |
done | Terminal success | Read the completed result. |
failed | Terminal failure | Stop polling and surface the job failure. |
cleared | Terminal without a stored result | Stop polling and treat the result as unavailable. |
The published API does not define a retention duration, so do not design around
a guaranteed result-availability window. SDK 0.18.0 succeeds on done, raises
on failed, and otherwise stops at its configured timeout; a direct poller must
handle cleared itself.
Result retrieval defaults to save_result=false. A terminal done or failed
read can clear stored fields after building its response, so a lost response
can be followed by cleared. Set save_result=true before polling when
terminal delivery must be repeatable; that setting still defines no retention
duration.
Event results are state, not a transcript log
Fast, realtime, and diarization responses evolve. Reconcile them by request or stream ID instead of appending every event.
| Surface | Result markers | State rule |
|---|---|---|
| Fast transcription | id, seq, is_final | Treat non-final responses as provisional and commit the final response once. |
| Realtime transcription | id, seq, is_final, is_speech_final | Replace provisional text while both final flags are false; commit when either becomes true. |
| Live diarization | id, final_segments, active_segments, is_final | Accumulate unseen final additions and replace the revisable active tail. |
Fast and Realtime responses contain seq, but their current public contracts
do not define ordering or uniqueness semantics for it. Route events by id,
process them in observed arrival order, and treat each event's text and words as
that event's state. Finish only on the capability's final signal or an
error/deadline.
RealtimeSubtitles intentionally ignores provisional responses and
deduplicates finalized responses by stream ID and seq. Because the current
Realtime wire contract does not promise distinct seq values, do not use that
helper to collect multiple final events. Collect finalized words in observed
arrival order and render them with Subtitles instead.
End a live input with the documented final frame, wait only to an app deadline, and always clean up the client. A close wait expiring does not prove a final result arrived; inspect the state recorded by callbacks.
Build a speaker timeline
Diarization produces relative speaker labels over time, not real-world speaker
identity. Live SDK updates expose the reconciled timeline as update.segments
and new final additions as update.newlyFinalized /
update.newly_finalized. Closing the SDK stream waits up to five seconds and
returns the best-known timeline if no final update arrives in that time.
In the batch V2 response, final_word_segments and diarization_segments are
separate. If the app needs speaker-attributed words, document an explicit
overlap rule rather than assuming every word already contains speaker.
Audio and transport contracts
| Surface | Audio contract | SDK 0.18.0 transport |
|---|---|---|
| Batch transcription | Complete supported audio-file container | REST |
| Fast transcription | One complete AAC, FLAC, MP3, MP4, or WAV file | Socket.IO |
| Realtime transcription and live diarization | PCM16 little-endian, 16 kHz, mono | Socket.IO |
| TTS through the SDK | Raw PCM16 little-endian, 24 kHz, mono output | Socket.IO |
| Direct HTTP TTS | Undelimited protocol capture with conceptual 16 kHz PCM16 service frames; not generically decodable audio | No public SDK wrapper |
Raw PCM is not a media-file container. A typical player needs a WAV header with
the matching sample rate. Direct integrations can also use the validated
Realtime HTTP operations; no public SDK 0.18.0 client wraps them.
Next steps
Concepts explain what to choose. The SDK guides define released client behavior; recipes assemble complete tasks; OpenAPI and AsyncAPI define direct wire contracts.
Make a first request
Run a tested batch or realtime workflow after choosing the processing model.
Use an SDK client
Review exact methods, options, response fields, timeouts, and cleanup behavior.
Complete a task
Build speaker-attributed transcripts, finalized captions, or playable TTS output.
Integrate directly
Use transport guides, then confirm exact fields and frames in the generated API references.