Realtime HTTP API

Build a speaker timeline from arriving audio

Use this operation to build a speaker timeline while PCM audio is still arriving.

POST/http/diarization-stream

Use this operation to build a speaker timeline while PCM audio is still arriving. It does not transcribe speech or identify real people. JavaScript and Python SDK 0.18.0 use Socket.IO; they do not issue this HTTP request.

From a trusted backend, send one complete binary frame per POST with X-Api-Key and the diarization capability. Reuse one fresh nonzero UUID until the stream ends, and keep at most one POST in flight for that UUID. Concurrent requests for the same UUID can overwrite response ownership. Capture audio into a bounded queue and have one sender drain it, closing each response before sending the next frame. Distinct UUID streams can run concurrently.

Every frame has an 18-byte header followed by nonempty raw mono PCM16 little-endian audio at 16 kHz. Bytes 0..15 are the UUID. In byte 16, bit 0 is is_start and bit 1 is is_final; reserved bits 2..7 MUST be zero and a frame that sets any of them is rejected with 400 VALIDATION_INVALID_FORMAT. Byte 17 must be 0 (Arabic), 1 (English), 2 (code-switch), or 255 (auto), but is discarded after validation and does not change diarization. Set start only on the first frame, final on the last real audio frame, and both (0x03) for a one-frame stream. Every request must include a nonempty even-length PCM payload; there is no empty terminator.

Each 200 can contain zero or more NDJSON records. Buffer network reads and split only on newline. Aggregate records across every response for the UUID. Accumulate unseen final_segments deltas, replace the previous active_segments snapshot, and sort the reconciled timeline by start_time. Speaker labels are relative to one stream, not identities. Segment times are seconds from the stream start. Retain a nonempty active tail in a final record as provisional; do not relabel it finalized.

Only an observed record with is_final: true completes the stream. A final request bit, an empty 200, response EOF, or timeout does not. If output has started, a later failure ends the partial 200 stream without an appended JSON error. A normal non-final response window ending after two seconds preserves the session. Aborting a POST or timing out the final response cancels the session, and the session expires after 60 seconds without client or inference activity. There is no HTTP chunk replay, resume, or idempotency contract. After an ambiguous failure, stop the producer, close every response, preserve the timeline as incomplete, and recover with a fresh UUID instead of replaying an old chunk.

The service selects the realtime diarization model; clients have no model selector. Missing model configuration returns 400 DIARIZATION_MODEL_NOT_FOUND. Backend capacity and inference errors currently collapse to retryable 500 DIARIZATION_FAILED; a production gateway can independently return 429 with deployment-specific details.

Authorization

ApiKeyAuth
X-Api-Key<token>

In: header

Request Body

application/octet-stream

Response Body

application/x-ndjson

application/json

application/json

application/json

application/json

application/json

application/json

application/json

application/json

application/json

application/json

curl -sS --fail-with-body --connect-timeout 10 --max-time 120 -X POST \  "https://example.com/http/diarization-stream" \  -H "Origin: https://example.com" \  -H "Content-Type: application/octet-stream" \  --data-binary @request.bin

Zero or more diarization NDJSON records. Once output has started, a later failure ends the partial stream without an appended JSON error. Completion requires an observed record with is_final: true.

application/x-ndjson

Final-segment delta plus current active snapshot

{
  "id": "7f51f2c2-e7bc-41c8-a850-f848df2ddfc8",
  "final_segments": [
    {
      "start_time": 0,
      "end_time": 1.5,
      "speaker": "SPEAKER_01"
    }
  ],
  "active_segments": [
    {
      "start_time": 1.5,
      "end_time": 3,
      "speaker": "SPEAKER_02"
    }
  ],
  "is_final": false
}

Later final delta with a best-known provisional tail

{
  "id": "7f51f2c2-e7bc-41c8-a850-f848df2ddfc8",
  "final_segments": [
    {
      "start_time": 1.5,
      "end_time": 3,
      "speaker": "SPEAKER_02"
    }
  ],
  "active_segments": [
    {
      "start_time": 3,
      "end_time": 3.4,
      "speaker": "SPEAKER_01"
    }
  ],
  "is_final": true
}

Invalid diarization frame, stream start, or service model configuration

application/json

Header is short or UUID is zero

{
  "error": "invalid audio upload",
  "code": "VALIDATION_FILE_CORRUPT",
  "detail": "invalid audio upload",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z"
}

Language byte is not 0, 1, 2, or 255

{
  "error": "invalid language",
  "code": "VALIDATION_INVALID_LANGUAGE",
  "detail": "invalid language",
  "job_id": "7f51f2c2-e7bc-41c8-a850-f848df2ddfc8",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z"
}

PCM payload is empty

{
  "error": "audio upload is empty",
  "code": "VALIDATION_FILE_CORRUPT",
  "detail": "audio upload is empty",
  "job_id": "7f51f2c2-e7bc-41c8-a850-f848df2ddfc8",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z"
}

PCM payload has an odd byte count

{
  "error": "audio must contain int16 samples",
  "code": "VALIDATION_INVALID_FORMAT",
  "detail": "audio must contain int16 samples",
  "job_id": "7f51f2c2-e7bc-41c8-a850-f848df2ddfc8",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z"
}

First frame does not set the start bit

{
  "error": "missing is_start flag",
  "code": "VALIDATION_REQUIRED_FIELD",
  "detail": "missing is_start flag",
  "job_id": "7f51f2c2-e7bc-41c8-a850-f848df2ddfc8",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z"
}

Realtime diarization is not configured

{
  "error": "diarization model not found",
  "code": "DIARIZATION_MODEL_NOT_FOUND",
  "detail": "diarization model not found",
  "job_id": "7f51f2c2-e7bc-41c8-a850-f848df2ddfc8",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z"
}

Unauthorized

application/json

Example missing_key

{
  "error": "Invalid authentication",
  "code": "AUTH_UNAUTHORIZED",
  "detail": "Invalid authentication",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z"
}

The API key does not grant access to the requested voice capability

application/json

Example scope_denied

{
  "error": "scope not permitted",
  "code": "AUTH_FORBIDDEN",
  "detail": "scope not permitted",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z"
}

Method not allowed

application/json

Example wrong_method

{
  "error": "method not allowed",
  "code": "METHOD_NOT_ALLOWED",
  "detail": "method not allowed",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z"
}

No audio arrived on this pseudo-session within its idle window, so the session was retired (RFC 9110 15.5.9). Not retryable against the same session id, which is now tombstoned: start a new session with a new id and is_start.

application/json

Example session_went_idle

{
  "error": "session idle timeout exceeded; start a new session",
  "code": "SESSION_IDLE_TIMEOUT",
  "detail": "session idle timeout exceeded; start a new session",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z",
  "data": {
    "limit": 900,
    "observed": 901,
    "unit": "seconds",
    "bound": "session_idle"
  }
}

The chunk conflicts with the state of its pseudo-session (RFC 9110 15.5.10): the id was never started, has already been retired, or an is_start arrived for an id that is already live. One answer covers all of them, so timing cannot change the contract. Start a new session with a new id.

application/json

Example not_live

{
  "error": "session is not live; start a new session with is_start and a new id",
  "code": "SESSION_EXPIRED",
  "detail": "session is not live; start a new session with is_start and a new id",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z"
}

The request body exceeded the configured byte limit for this audio route: 64 MiB for Fast uploads, 16 MiB for Realtime ASR frames, 16 MiB for Realtime diarization frames. Non-retryable at the same size; resend a smaller unit or chunk.

data.bound names which byte limit was hit - fast_audio_bytes, realtime_asr_frame_bytes, or realtime_diarization_frame_bytes. data.observed is the exact request size when the client declared a Content-Length, and otherwise a MINIMUM (the limit plus one byte), because a body with no declared length is cut off mid-read and its true size is never learned.

This status is only ever reached from a BYTE count. A request whose bytes are acceptable but whose decoded audio is too long is 422 with AUDIO_DURATION_EXCEEDED instead.

application/json

Content-Length was declared, so observed is exact

{
  "error": "request body too large",
  "code": "PAYLOAD_TOO_LARGE",
  "detail": "request body too large",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z",
  "data": {
    "limit": 16777216,
    "observed": 20971520,
    "unit": "bytes",
    "bound": "realtime_asr_frame_bytes"
  }
}

No declared length, so observed is the limit plus one byte

{
  "error": "request body too large",
  "code": "PAYLOAD_TOO_LARGE",
  "detail": "request body too large",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z",
  "data": {
    "limit": 67108864,
    "observed": 67108865,
    "unit": "bytes",
    "bound": "fast_audio_bytes"
  }
}

The request parsed correctly and its bytes were acceptable, but the amount of AUDIO it asks the service to process exceeds this endpoint's ceiling (RFC 9110 15.5.21). A small compressed upload that decodes to many hours is exactly this case, which is why it is not 413.

data.bound names which audio ceiling was hit:

  • fast_audio_duration - one Fast submission decoded to more than 1800 seconds. Split the recording or use the batch transcription API.
  • session_audio_duration - a realtime session has now sent more total audio content than its 14400-second (4 hour) allowance. The session is retired; start a new one.

data.observed is in whole seconds, rounded up. Where the service stopped decoding at the ceiling it never learned the true total length, so the observed value is a MINIMUM rather than an exact measurement.

Not retryable: resending the identical audio cannot succeed. Shorten the unit, or move to the batch API.

application/json

Example fast_decoded_audio_too_long

{
  "error": "decoded audio duration exceeds the maximum for this endpoint; split the recording or use the batch transcription API",
  "code": "AUDIO_DURATION_EXCEEDED",
  "detail": "decoded audio duration exceeds the maximum for this endpoint; split the recording or use the batch transcription API",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z",
  "data": {
    "limit": 1800,
    "observed": 3601,
    "unit": "seconds",
    "bound": "fast_audio_duration"
  }
}

Example session_audio_allowance_spent

{
  "error": "session maximum audio duration exceeded; start a new session",
  "code": "AUDIO_DURATION_EXCEEDED",
  "detail": "session maximum audio duration exceeded; start a new session",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z",
  "data": {
    "limit": 14400,
    "observed": 14401,
    "unit": "seconds",
    "bound": "session_audio_duration"
  }
}

A realtime request was throttled. Three distinct sources share this status on these routes and the code field distinguishes them:

  • SESSION_BYTE_RATE_EXCEEDED - audio is arriving faster than the session's sustained rate allows (four times real time, with a 16 MiB burst). Honour Retry-After; the identical payload then succeeds. The session stays live. data.bound is session_audio_rate_burst.
  • CONCURRENCY_LIMIT_EXCEEDED - the billable account already has as many concurrent operations of this kind in flight as its plan allows. data.bound is account_concurrency_<workload>.
  • SESSION_SLOTS_EXHAUSTED - the account holds as many concurrent HTTP pseudo-sessions as this process allows.

The gateway's per-key request-RATE limit also answers 429, reports RATE_LIMIT_EXCEEDED, and has a deployment-specific body shape. All of these are retryable and none consumes credit or quota.

application/json

Example audio_arriving_too_fast

{
  "error": "audio is arriving faster than this session allows; slow down to real time and retry",
  "code": "SESSION_BYTE_RATE_EXCEEDED",
  "detail": "audio is arriving faster than this session allows; slow down to real time and retry",
  "retryable": true,
  "timestamp": "2026-01-15T10:30:00Z",
  "data": {
    "limit": 16777216,
    "observed": 33554432,
    "unit": "bytes",
    "bound": "session_audio_rate_burst"
  }
}

Example account_concurrency_exhausted

{
  "error": "too many concurrent operations for this account",
  "code": "CONCURRENCY_LIMIT_EXCEEDED",
  "detail": "too many concurrent operations for this account",
  "retryable": true,
  "timestamp": "2026-01-15T10:30:00Z",
  "data": {
    "limit": 8,
    "observed": 8,
    "unit": "operations",
    "bound": "account_concurrency_realtime_asr"
  }
}

Diarization failed before a final record was emitted. Backend capacity and inference failures currently collapse to this retryable response.

application/json

Example diarization_failed

{
  "error": "realtime diarization failed",
  "code": "DIARIZATION_FAILED",
  "detail": "realtime diarization failed",
  "job_id": "7f51f2c2-e7bc-41c8-a850-f848df2ddfc8",
  "retryable": true,
  "timestamp": "2026-01-15T10:30:00Z"
}

Next steps

Complete the live stream lifecycle before consuming speaker labels: keep one UUID across framed chunks, serialize same-UUID POSTs, reconcile final deltas with the latest active snapshot, and clean up on a final result, structured error, or deadline.

Implement HTTP diarization

Apply the binary frame contract and incremental-result lifecycle.

Compare the SDK transport

Use the released Socket.IO client when it better matches your application.

Stream Realtime ASR from arriving audio

Canonical direct HTTP operation for Realtime ASR.

Capture an HTTP TTS service-frame stream

This is a direct platform HTTP operation.

On this page

Next steps