Transcribe Live Audio

This recipe builds one production-shaped live transcription session. It starts while audio is still arriving, keeps provisional UI state separate from committed text, writes captions from finalized words, and closes every resource at a defined boundary.

When to use this recipe

Choose by the state of the audio:

Fast transcription receives a complete audio unit. Batch owns complete long-form recordings. Neither replaces Realtime ASR when the producer must send audio before the utterance has ended.

Before you start

You need:

• JavaScript @humain-voice/sdk@0.18.0 or Python humain-voice==0.18.0 in a trusted server runtime.
• The provisioned API_URL and API_KEY. Both released RealtimeClient implementations default to /socket.io.
• A source that can provide raw PCM16 little-endian, 16 kHz, mono audio.
• An app-level session deadline and a place to keep provisional, committed, and error state separately.

export API_URL="https://api.voice.humain.com"
export API_KEY="YOUR_API_KEY"

Keep the API key out of browser and mobile code. See Authentication for the organization's credential flow.

1. Prepare raw PCM input

The Realtime ASR audio contract is exact:

For a repeatable walkthrough, convert a recording to the same raw format that a microphone or call pipeline must produce:

ffmpeg -i input.wav -f s16le -acodec pcm_s16le -ar 16000 -ac 1 speech.pcm

Send the raw bytes, not a WAV header or compressed container. The 3,200-byte chunk and 100 ms pace are the tested fixture's framing choice; the input format is the service contract.

2. Run the tested happy path

The programs read speech.pcm, pace it like a live producer, label every response, collect final-event words in arrival order, wait up to five seconds for final state, render speech.vtt with Subtitles, and clean up the client.

import { readFile, writeFile } from 'node:fs/promises';

import {
  type ErrorResponse,
  Language,
  RealtimeClient,
  Subtitles,
  type WordSegment,
} from '@humain-voice/sdk';

const CHUNK_BYTES = 3_200; // 100 ms of PCM16LE, 16 kHz, mono audio.

function requiredEnv(name: 'API_KEY' | 'API_PATH' | 'API_URL'): string {
  const value = process.env[name];
  if (!value) throw new Error(`${name} is required`);
  return value;
}

async function pause(milliseconds: number): Promise<void> {
  await new Promise<void>((resolve) => setTimeout(resolve, milliseconds));
}

async function main(): Promise<void> {
  const inputPath = process.argv[2] ?? 'speech.pcm';
  const outputPath = process.argv[3] ?? 'speech.vtt';
  const client = new RealtimeClient({
    api_url: requiredEnv('API_URL'),
    api_path: requiredEnv('API_PATH'),
    api_key: requiredEnv('API_KEY'),
  });
  const finalizedWords: WordSegment[] = [];
  let serverError: ErrorResponse | undefined;
  let protocolFinalObserved = false;

  try {
    const stream = await client.startStream(Language.ArEn, {
      onResponse: (response) => {
        const kind = response.is_final
          ? 'final'
          : response.is_speech_final
            ? 'speech-final'
            : 'partial';
        console.info(`${kind}:`, response.transcription);
        if (response.is_final) protocolFinalObserved = true;
        if (response.is_final || response.is_speech_final) {
          // The current public Realtime contract does not guarantee increasing
          // seq values, so collect final words in arrival order instead of
          // asking RealtimeSubtitles to deduplicate by id:seq.
          finalizedWords.push(...response.words);
        }
      },
      // The released SDK can invoke a stream handler more than once for one
      // routed error, so keep this callback idempotent.
      onError: (error) => {
        serverError = error;
      },
    });

    const pcm = await readFile(inputPath);
    for (let offset = 0; offset < pcm.length; offset += CHUNK_BYTES) {
      await stream.send(pcm.subarray(offset, offset + CHUNK_BYTES));
      await pause(100);
    }

    // close() sends the last frame and waits for protocol is_final, a routed
    // error, or this timeout. It resolves rather than throwing on timeout.
    await stream.close(5);
    if (serverError) {
      throw new Error(serverError.message ?? serverError.code ?? 'Realtime stream failed');
    }
    if (!protocolFinalObserved) {
      throw new Error('Realtime stream ended before protocol is_final');
    }
    await writeFile(outputPath, Subtitles.fromWords(finalizedWords).toVtt(), 'utf8');
  } finally {
    await client.disconnect();
  }
}

void main().catch((error: unknown) => {
  console.error(error);
  process.exitCode = 1;
});

A successful run has these observable outcomes:

1. Each response is printed as partial, final, or speech-final according to its flags.
2. No recorded server error remains when the stream finishes.
3. speech.vtt contains cues built only from finalized word timing.
4. The JavaScript client disconnects in finally; the Python async context closes its Socket.IO and underlying HTTP resources.

The saved PCM file makes the walkthrough repeatable. In production, replace the file read and timer with the microphone or call source, while keeping the same state, termination, and cleanup boundaries.

3. Reconcile provisional and final text

Realtime text is evolving state, not one append-only string. Route responses by stream id, assign an app-local arrival number, and apply this transition table. Keep server seq only as diagnostic data because its ordering and uniqueness are not part of the current public contract.

Keep the error callback idempotent. SDK 0.18.0 can invoke a stream error handler more than once for one routed error.

Do not render a late provisional response over committed text. Preserve the last committed transcript independently from the mutable provisional line so a disconnect cannot erase stable results.

4. Generate captions from finalized words

In the response callback, ignore words while both final flags are false. Append the words from final or speech-final events to an app-owned array in observed arrival order. After termination, pass that array to Subtitles.fromWords() / Subtitles.from_words() and render WebVTT.

SDK 0.18.0 also exposes RealtimeSubtitles, which ignores provisional events and deduplicates final events by id:seq. Do not use it to collect multiple final events against the current wire contract, because distinct seq values are not guaranteed. The fixtures use the app-owned collection path and write the caption file only after the stream closes and recorded errors are checked.

5. Terminate on a deadline and clean up

When the producer has no more audio, call close(5) in JavaScript or close(timeout_seconds=5.0) in Python. In release 0.18.0, close:

1. sends the final stream frame;
2. waits for protocol-level is_final, a routed error, or the supplied timeout; and
3. resolves when the wait expires instead of raising a timeout error.

The SDK default final-result wait is one second; the fixtures deliberately pass five. This timeout bounds only the close wait. Keep a separate app deadline for connection, audio production, sending, and the complete session.

A resolved close does not prove is_final arrived. is_speech_final is an utterance boundary and does not release the close wait. Check the protocol-final and error state recorded by callbacks. If the stream must be abandoned, stop() / stop_sync() removes its context without the final wait.

Always perform client-level cleanup after stream-level termination. A routed error can remove the stream context before close runs, but JavaScript must still call disconnect() in finally, and Python must still exit the client context.

6. Recover only across a new stream boundary

The public contract does not define transparent session resume or what server state survives a dropped connection. On a routed error or disconnect:

1. Stop feeding the old stream and preserve only committed final results.
2. Discard unresolved provisional text and clean up the old client.
3. If the app deadline and retry policy allow, reconnect with bounded backoff and jitter and create a new stream with a fresh ID.
4. Keep the new stream's results separate until the app explicitly joins the two committed timelines.

Do not assume that replaying earlier chunks is safe; the contract provides no resume position or per-chunk acknowledgement. Audio captured around the drop may need to be retained and processed separately. See the Realtime lifecycle guide for the full recovery boundary.

Next steps