Concepts

Pick the HUMAIN Voice workflow and audio format for your input, and learn how batch jobs, streaming results, speaker timelines, and speech output behave.

Choose a workflow

WorkflowUse it whenInputHow results arrive
Batch transcriptionThe recording already exists: meetings, calls, podcasts, archives.One complete audio file over REST.Submission returns a jobId; poll until the job reaches a terminal status.
Fast transcriptionYou have one short, complete clip and need the text quickly, such as a finished voice command or conversation turn.One complete audio file over Socket.IO or HTTP streaming.Zero or more provisional results, then one result with is_final: true.
Realtime transcriptionAudio is still arriving: a microphone, a phone call, live captions.Raw PCM chunks over Socket.IO or HTTP streaming.Provisional text that is replaced as you speak, a final result at each end of speech, and is_final: true when the stream ends.
Live diarizationYou need to know who is speaking while audio is still arriving. It returns speaker segments, not text.Raw PCM chunks over Socket.IO or HTTP streaming.A growing list of finalized speaker segments plus a revisable active tail, until is_final: true.
Text to speech (TTS)Speech is the output and text is the input.Text, a model, and a voice over Socket.IO, HTTP streaming, or one buffered POST (OpenAI-compatible speech).Audio chunks; the last chunk is marked final. The buffered POST returns one complete file.
Voice AgentYou want a spoken conversation: the user talks and a hosted agent answers.The user's microphone, streamed over WebRTC from a browser or app.Spoken replies, live transcripts, and tool calls until the call ends; see Voice Agent.

If the audio already exists, use batch; use fast only for one short complete clip where latency matters (fast accepts up to 30 minutes, but use it for a single turn; see Limits). Per-workflow size and duration limits are in Limits, and the language, model, and voice values for each workflow are in Models and voices.

For live text and speakers together, set the diarization flag (bit 2 of the frame flags byte) on a direct Socket.IO realtime stream; SDK 0.18.1 doesn't expose this flag (see Socket.IO API). Or run realtime transcription and live diarization on the same audio and align them by time.

Audio formats by workflow

WorkflowAudio format
Batch transcriptionOne complete audio file in any format ffmpeg can decode. Unsupported, corrupt, or empty files return 422.
Fast transcriptionOne complete AAC (ADTS), FLAC, MP3, MP4, or WAV file. The service detects the format from the bytes, not the filename. Put the MP4 moov atom at the start of the file.
Realtime transcription and live diarizationRaw PCM16 little-endian, 16 kHz, mono, with no WAV header.
TTS output from the SDKRaw PCM16 little-endian, 24 kHz, mono, with no WAV header.
TTS output over HTTP streaming16 kHz PCM16 wrapped in service frames that a client can't reliably split. Use the SDK, or POST /v1/audio/speech for one complete file, when you need playable audio.

To convert a recording to realtime PCM, including 8 kHz phone audio that must become 16 kHz, use the ffmpeg command in Realtime transcription. To play or share TTS output, add a WAV header with the matching sample rate; see Text to speech: save or stream.

Batch is a job

Submitting a file creates a job. Poll it with a timeout on each request and one overall deadline for the whole job.

StatusTerminalWhat to do
queuedNoWait, then poll again.
processingNoKeep polling within the same deadline.
doneYesRead the transcript and any speaker results.
failedYesStop polling and report the failure.
clearedYesStop polling. The stored result is gone and can't be read again.

All five statuses return HTTP 200; a non-2xx response is a request, authentication, or server error, covered in Error codes. By default, a terminal read can delete the stored result, so a second read returns cleared; set save_result=true if you may need to read it again. See Batch REST API.

Streaming results are state, not a log

Fast transcription, realtime transcription, and live diarization send results that change. Treat each one as the current state of a request or stream, and replace earlier state instead of appending every event.

FlagMeaningWhat to do
is_final false (realtime: and is_speech_final false)Provisional. Text can still change.Replace the previous provisional text for that id.
is_speech_final: true, is_final falseA speaker paused; this segment's text is final. Realtime only.Commit the text and keep the stream open.
is_final: trueThe last result for the request or stream.Commit any remaining text and stop waiting for that id.

Live diarization: see Speaker timelines.

Follow these rules for every streaming workflow:

  • Route each result by its id. One connection can carry several requests or streams.
  • Apply results in the order they arrive.
  • Don't use seq to order results or remove duplicates.
  • Finish when is_final: true arrives, an error arrives, or your deadline expires. Ending your input doesn't mean the final result has arrived; how to close a realtime stream and wait is in Realtime transcription.

Speaker timelines

Batch diarization (an option on a batch job) and live diarization both return speaker labels with start and end times. A label marks a speaker relative to one result; it doesn't identify a person.

  • Live diarization sends final_segments, which won't change, and active_segments, which can still change. Add each new final segment to your timeline once, and replace the active segments on every result. The SDK already does this merge: update.segments is the current timeline and newlyFinalized / newly_finalized holds the new final segments, so don't merge them again.
  • Labels are opaque, result-relative strings. Batch formats them like SPEAKER_1 and streaming like SPEAKER_01, so don't parse them or compare them across results.
  • Live diarization can split one person into two labels. Merge labels yourself if your app needs one label per person.

How the batch response attributes words to speakers is in Transcribe a recording.

Speech output

TTS returns audio in chunks. The SDK offers two ways to consume them:

ModeSDK methodUse it when
Bufferedsynthesize()You want the whole clip at once: writing a file, attaching audio to a message, caching a prompt.
StreamingsynthesizeStream() / synthesize_stream()You want playback to start before synthesis finishes, such as a voice agent's reply.

synthesize() returns only after the final chunk; with streaming, the clip is complete at the chunk marked final. If the request fails partway, discard the partial audio. Both modes are shown in Text to speech: save or stream.

Next steps

On this page