Stream Realtime ASR from arriving audio

POST/http/stt-stream

Canonical direct HTTP operation for Realtime ASR. POST /http/realtime-asr is a compatibility alias; new clients should use this path. JavaScript and Python SDK 0.18.0 use Socket.IO and do not call either HTTP route.

Send one POST for each audio chunk as it arrives. Prefix every body with the same 18-byte control header: bytes 0..15 are a fresh nonzero UUID in raw binary form; byte 16 contains is_start in bit 0 and is_final in bit 1; byte 17 is the language (0=ar, 1=en, 2=codeswitch, 255=auto). Keep reserved flag bits zero. Append nonempty raw mono PCM16 little-endian audio at 16 kHz, without a WAV header. Set is_start only on the first chunk, neither flag on intermediate chunks, and is_final on the last chunk with audio; set both for a single-chunk stream.

A 200 response contains zero or more newline-delimited JSON records. Buffer across network reads and parse complete lines. Treat is_speech_final as a speech boundary. Complete the stream only after observing is_final: true; the request final bit and an HTTP response ending, including an empty 200, are not completion signals. Treat seq as opaque and process records in observed arrival order. If output has started, a later failure ends the partial 200 stream without an appended JSON error.

Use X-Api-Key with the realtime ASR capability from a trusted backend. Bound every POST/read and the whole stream. Chunk replay and session resume are not defined; after an ambiguous failure, stop the old stream, discard provisional state, and restart with a fresh UUID. The public contract does not define whether chunk POSTs should overlap or be serialized; use the coordination pattern provisioned for your environment. A normal non-final response window ending after two seconds preserves the session. Aborting a POST or timing out the final response cancels the session. The session also expires after 60 seconds without accepted client audio or an inference response.

Authorization

ApiKeyAuth

X-Api-Key<token>

In: header

Request Body

application/octet-stream

Response Body

`application/x-ndjson`

`application/json`

curl -sS --fail-with-body --connect-timeout 10 --max-time 120 -X POST \  "https://example.com/http/stt-stream" \  -H "Origin: https://example.com" \  -H "Content-Type: application/octet-stream" \  --data-binary @request.bin

Zero or more Realtime ASR NDJSON records. Once output has started, a later failure ends the partial stream without an appended JSON error. Completion requires an observed record with is_final: true.

application/x-ndjson

Example partial

{
  "id": "7f51f2c2-e7bc-41c8-a850-f848df2ddfc8",
  "seq": 0,
  "transcription": "hello wor",
  "words": [
    {
      "start_time": 0,
      "end_time": 0.45,
      "word": "hello"
    }
  ],
  "is_speech_final": false,
  "is_final": false
}

Example final

{
  "id": "7f51f2c2-e7bc-41c8-a850-f848df2ddfc8",
  "seq": 0,
  "transcription": "hello world",
  "words": [
    {
      "start_time": 0,
      "end_time": 0.45,
      "word": "hello"
    },
    {
      "start_time": 0.46,
      "end_time": 0.9,
      "word": "world"
    }
  ],
  "is_speech_final": true,
  "is_final": true
}

Invalid control header, UUID, language byte, or PCM payload

application/json

Header is short or UUID is zero

{
  "error": "invalid audio upload",
  "code": "VALIDATION_FILE_CORRUPT",
  "detail": "invalid audio upload",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z"
}

Invalid language byte

{
  "error": "invalid language",
  "code": "VALIDATION_INVALID_LANGUAGE",
  "detail": "invalid language",
  "job_id": "7f51f2c2-e7bc-41c8-a850-f848df2ddfc8",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z"
}

PCM payload is empty

{
  "error": "audio upload is empty",
  "code": "VALIDATION_FILE_CORRUPT",
  "detail": "audio upload is empty",
  "job_id": "7f51f2c2-e7bc-41c8-a850-f848df2ddfc8",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z"
}

PCM payload has an odd byte count

{
  "error": "audio must contain int16 samples",
  "code": "VALIDATION_INVALID_FORMAT",
  "detail": "audio must contain int16 samples",
  "job_id": "7f51f2c2-e7bc-41c8-a850-f848df2ddfc8",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z"
}

Unauthorized

application/json

Example missing_key

{
  "error": "Invalid authentication",
  "code": "AUTH_UNAUTHORIZED",
  "detail": "Invalid authentication",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z"
}

The API key does not grant access to the requested voice capability

application/json

Example scope_denied

{
  "error": "scope not permitted",
  "code": "AUTH_FORBIDDEN",
  "detail": "scope not permitted",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z"
}

Method not allowed

application/json

Example wrong_method

{
  "error": "method not allowed",
  "code": "METHOD_NOT_ALLOWED",
  "detail": "method not allowed",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z"
}

No audio arrived on this pseudo-session within its idle window, so the session was retired (RFC 9110 15.5.9). Not retryable against the same session id, which is now tombstoned: start a new session with a new id and is_start.

application/json

Example session_went_idle

{
  "error": "session idle timeout exceeded; start a new session",
  "code": "SESSION_IDLE_TIMEOUT",
  "detail": "session idle timeout exceeded; start a new session",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z",
  "data": {
    "limit": 900,
    "observed": 901,
    "unit": "seconds",
    "bound": "session_idle"
  }
}

The chunk conflicts with the state of its pseudo-session (RFC 9110 15.5.10): the id was never started, has already been retired, or an is_start arrived for an id that is already live. One answer covers all of them, so timing cannot change the contract. Start a new session with a new id.

application/json

Example not_live

{
  "error": "session is not live; start a new session with is_start and a new id",
  "code": "SESSION_EXPIRED",
  "detail": "session is not live; start a new session with is_start and a new id",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z"
}

The request body exceeded the configured byte limit for this audio route: 64 MiB for Fast uploads, 16 MiB for Realtime ASR frames, 16 MiB for Realtime diarization frames. Non-retryable at the same size; resend a smaller unit or chunk.

data.bound names which byte limit was hit - fast_audio_bytes, realtime_asr_frame_bytes, or realtime_diarization_frame_bytes. data.observed is the exact request size when the client declared a Content-Length, and otherwise a MINIMUM (the limit plus one byte), because a body with no declared length is cut off mid-read and its true size is never learned.

This status is only ever reached from a BYTE count. A request whose bytes are acceptable but whose decoded audio is too long is 422 with AUDIO_DURATION_EXCEEDED instead.

application/json

Content-Length was declared, so observed is exact

{
  "error": "request body too large",
  "code": "PAYLOAD_TOO_LARGE",
  "detail": "request body too large",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z",
  "data": {
    "limit": 16777216,
    "observed": 20971520,
    "unit": "bytes",
    "bound": "realtime_asr_frame_bytes"
  }
}

No declared length, so observed is the limit plus one byte

{
  "error": "request body too large",
  "code": "PAYLOAD_TOO_LARGE",
  "detail": "request body too large",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z",
  "data": {
    "limit": 67108864,
    "observed": 67108865,
    "unit": "bytes",
    "bound": "fast_audio_bytes"
  }
}

The request parsed correctly and its bytes were acceptable, but the amount of AUDIO it asks the service to process exceeds this endpoint's ceiling (RFC 9110 15.5.21). A small compressed upload that decodes to many hours is exactly this case, which is why it is not 413.

data.bound names which audio ceiling was hit:

fast_audio_duration - one Fast submission decoded to more than 1800 seconds. Split the recording or use the batch transcription API.
session_audio_duration - a realtime session has now sent more total audio content than its 14400-second (4 hour) allowance. The session is retired; start a new one.

data.observed is in whole seconds, rounded up. Where the service stopped decoding at the ceiling it never learned the true total length, so the observed value is a MINIMUM rather than an exact measurement.

Not retryable: resending the identical audio cannot succeed. Shorten the unit, or move to the batch API.

application/json

Example fast_decoded_audio_too_long

{
  "error": "decoded audio duration exceeds the maximum for this endpoint; split the recording or use the batch transcription API",
  "code": "AUDIO_DURATION_EXCEEDED",
  "detail": "decoded audio duration exceeds the maximum for this endpoint; split the recording or use the batch transcription API",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z",
  "data": {
    "limit": 1800,
    "observed": 3601,
    "unit": "seconds",
    "bound": "fast_audio_duration"
  }
}

Example session_audio_allowance_spent

{
  "error": "session maximum audio duration exceeded; start a new session",
  "code": "AUDIO_DURATION_EXCEEDED",
  "detail": "session maximum audio duration exceeded; start a new session",
  "retryable": false,
  "timestamp": "2026-01-15T10:30:00Z",
  "data": {
    "limit": 14400,
    "observed": 14401,
    "unit": "seconds",
    "bound": "session_audio_duration"
  }
}

A realtime request was throttled. Three distinct sources share this status on these routes and the code field distinguishes them:

SESSION_BYTE_RATE_EXCEEDED - audio is arriving faster than the session's sustained rate allows (four times real time, with a 16 MiB burst). Honour Retry-After; the identical payload then succeeds. The session stays live. data.bound is session_audio_rate_burst.
CONCURRENCY_LIMIT_EXCEEDED - the billable account already has as many concurrent operations of this kind in flight as its plan allows. data.bound is account_concurrency_<workload>.
SESSION_SLOTS_EXHAUSTED - the account holds as many concurrent HTTP pseudo-sessions as this process allows.

The gateway's per-key request-RATE limit also answers 429, reports RATE_LIMIT_EXCEEDED, and has a deployment-specific body shape. All of these are retryable and none consumes credit or quota.

application/json

Example audio_arriving_too_fast

{
  "error": "audio is arriving faster than this session allows; slow down to real time and retry",
  "code": "SESSION_BYTE_RATE_EXCEEDED",
  "detail": "audio is arriving faster than this session allows; slow down to real time and retry",
  "retryable": true,
  "timestamp": "2026-01-15T10:30:00Z",
  "data": {
    "limit": 16777216,
    "observed": 33554432,
    "unit": "bytes",
    "bound": "session_audio_rate_burst"
  }
}

Example account_concurrency_exhausted

{
  "error": "too many concurrent operations for this account",
  "code": "CONCURRENCY_LIMIT_EXCEEDED",
  "detail": "too many concurrent operations for this account",
  "retryable": true,
  "timestamp": "2026-01-15T10:30:00Z",
  "data": {
    "limit": 8,
    "observed": 8,
    "unit": "operations",
    "bound": "account_concurrency_realtime_asr"
  }
}

Realtime ASR failed before a final record was emitted

application/json

Example transcription_failed

{
  "error": "realtime ASR transcription failed",
  "code": "ASR_TRANSCRIPTION_FAILED",
  "detail": "realtime ASR transcription failed",
  "job_id": "7f51f2c2-e7bc-41c8-a850-f848df2ddfc8",
  "retryable": true,
  "timestamp": "2026-01-15T10:30:00Z"
}

Next steps

Build the live lifecycle next: reuse one stream UUID across framed chunks, replace partial display state, commit only final results, and end every wait with an application deadline.

Frame live HTTP audio

Construct request frames and reconcile zero or more NDJSON records per response.

Use the Socket.IO SDK

Follow the released SDK lifecycle when you do not need direct HTTP.

Stream Realtime ASR from arriving audio

Authorization

Request Body

Response Body

200application/x-ndjson

400application/json

401application/json

403application/json

405application/json

408application/json

409application/json

413application/json

422application/json

429application/json

500application/json

Example partial

Example final

Header is short or UUID is zero

Invalid language byte

PCM payload is empty

PCM payload has an odd byte count

Example missing_key

Example scope_denied

Example wrong_method

Example session_went_idle

Example not_live

Content-Length was declared, so observed is exact

No declared length, so observed is the limit plus one byte

Example fast_decoded_audio_too_long

Example session_audio_allowance_spent

Example audio_arriving_too_fast

Example account_concurrency_exhausted

Example transcription_failed

Next steps

Frame live HTTP audio

Use the Socket.IO SDK

On this page

`application/x-ndjson`

`application/json`

`application/json`

`application/json`

`application/json`

`application/json`

`application/json`

`application/json`

`application/json`

`application/json`

`application/json`