Voice Agent

Client tools

Let the voice agent run functions in your user's browser, such as looking up an order or opening a page, and speak the result.

A client tool is a function in your page that the agent can call during the conversation. You declare it in the session request and handle it in the browser. The agent decides when to call it from its description, waits for the result, and uses the result in its reply.

Declare a tool

Add the tool to agent.client_tools when you start the session:

{
  "agent": {
    "prompt": "You help customers track their orders.",
    "client_tools": [
      {
        "name": "lookup_order",
        "description": "Look up the delivery status of an order by its number.",
        "parameters": {
          "type": "object",
          "properties": {
            "order": { "type": "string", "description": "Order number, such as T-1182." }
          },
          "required": ["order"]
        }
      }
    ]
  }
}
FieldRequiredMeaning
nameYes1–64 characters: letters, digits, _, -. Unique within the session. client_tool_call and end_call are reserved.
descriptionYesWhat the tool does and when to use it, up to 4,096 characters. The agent relies on it to decide when to call the tool.
parametersNoA JSON Schema object describing the arguments. Omit it for a tool without arguments.

A session can declare up to 16 tools; see Limits.

Handle the call in the browser

Pass one handler per declared tool to AgentSession.start(). The handler receives the arguments the agent chose and returns the result:

import { AgentSession } from '@humain-voice/agent';

const call = await AgentSession.start({
  token: () => fetch('/agent-session', { method: 'POST' }).then((res) => res.json()),
  clientTools: {
    lookup_order: async ({ order }) => {
      const res = await fetch(`/orders/${encodeURIComponent(String(order))}`);
      if (!res.ok) throw new Error('order not found');
      const body = await res.json();
      return { status: body.status, eta: body.eta };
    },
  },
});
RuleDetail
ResultAny JSON value. The agent reads it and answers in its own words.
Suggested replyReturn { "response": "…" } to give the agent the sentence to say.
FailureThrow an error. The agent tells the user it couldn't finish the action.
Time limit120 seconds; after that the call to the tool fails.
Result size15 KiB of JSON. Return only what the agent needs.
Missing handlerCalls to that tool fail, and onError reports CLIENT_TOOL_NOT_HANDLED.

While the tool runs, the agent can fill the silence; set this with Tool acknowledgments.

Let the agent end the call

Every session has a built-in end_call tool. The agent uses it when the conversation is over, for example after the user says goodbye; the call ends with agent_ended. Tell the agent when to hang up in the prompt, such as "When the user has no more questions, say goodbye and end the call."

Handle tools without the SDK

The agent calls tools over LiveKit RPC. Register the method client_tool_call on the user's participant. The request payload is JSON:

{ "tool": "lookup_order", "arguments": { "order": "T-1182" } }

Respond with the result as a JSON string, or with { "ok": false, "error": "…" } on failure:

room.registerRpcMethod('client_tool_call', async ({ payload }) => {
  const { tool, arguments: args } = JSON.parse(payload);
  if (tool !== 'lookup_order') return JSON.stringify({ ok: false, error: `unknown tool ${tool}` });
  const res = await fetch(`/orders/${encodeURIComponent(String(args.order))}`);
  if (!res.ok) return JSON.stringify({ ok: false, error: 'order not found' });
  return JSON.stringify(await res.json());
});

The rest of the raw contract is in Without the SDK.

On this page