# Kheelona Voice: the whole documentation

> Talk to Kheelona voices from a web page or from your own server. Voice conversations for toys and apps, in 11 languages (four supported, five beta, two alpha).

Every page of the documentation as Markdown, in the order of the sidebar. Each page starts with its title and a Source line. The index is [llms.txt](https://docs.voice.kheelona.com/llms.txt).

---

# Kheelona Voice

> Talk to Kheelona voices from a web page or from your own server. Voice conversations for toys and apps, in 11 languages (four supported, five beta, two alpha).

Source: https://docs.voice.kheelona.com/

Kheelona Voice lets your app or toy hold a spoken conversation with a **Kheelona voice**. The person speaks, the voice
listens and answers out loud, usually within about three seconds, in the language the person speaks. The person can
interrupt the voice at any time.

You pick a voice from Kheelona's catalogue, such as `lumi-dragon`. Each voice is a complete character with its own sound,
personality and safety level. You never choose or see the technology underneath: Kheelona runs and improves it.

## What you get

- **Voices.** The voices Kheelona has enabled for your account. See [Voices and versions](https://docs.voice.kheelona.com/concepts/voices.md).
- **Sessions.** One session is one conversation between one person and one voice. Your server starts it, and a browser
  or your server connects to it. See [Sessions](https://docs.voice.kheelona.com/concepts/sessions.md).
- **11 languages.** English, Hindi, Marathi, Bengali, Assamese, Gujarati, Punjabi, Tamil, Telugu, Kannada and Malayalam.
  Four are supported, five are beta and two are alpha. You choose the languages of each session. See
  [Languages](https://docs.voice.kheelona.com/concepts/languages.md#supported-beta-and-alpha).
- **Live text.** The text of what was said, as it happens, for captions or your own records. Kheelona does not keep it.
  See [Live text](https://docs.voice.kheelona.com/concepts/live-text.md).
- **Status events.** Listening, thinking, speaking and interrupted, to drive a character or a talk button.
- **Your own instructions.** Free text for one session, such as a child's first name. It is never stored in readable form.
- **Safety.** Every voice has a safety level, and Kheelona's safety rules check what is said and what the voice is about to say. See [Safety levels](https://docs.voice.kheelona.com/concepts/safety-levels.md).
- **Clear limits.** You have a maximum number of sessions at once and a maximum session length. When Kheelona is busy a
  new session is refused straight away with a wait time. See [Limits](https://docs.voice.kheelona.com/concepts/limits.md).

## Two ways to connect

| | From a web page | From your server |
|---|---|---|
| For | Web apps and pages | Toys, devices and backends that relay audio |
| How | The [browser SDK](https://docs.voice.kheelona.com/guides/browser-sdk.md), over WebRTC with a WebSocket fallback | A [WebSocket](https://docs.voice.kheelona.com/guides/server-to-server.md) |
| Start here | [Quickstart for browsers](https://docs.voice.kheelona.com/get-started/quickstart-browser.md) | [Quickstart for servers](https://docs.voice.kheelona.com/get-started/quickstart-server.md) |

In both, **your server creates the session with your API key**. The key stays on your server. The page or device gets only
a client secret that works once and for about a minute.

## What to expect

- **Best effort during the partner pilot.** The launch is limited and partners are hand-picked. Planned maintenance is
  announced in advance.
- **No sandbox.** There is no separate test mode. You integrate against the live service, within your session limit.
  Read [Going live](https://docs.voice.kheelona.com/guides/going-live.md) before real users arrive.
- **Response time is a target.** Typically under 3 seconds from the end of speech to the first sound. See
  [Latency](https://docs.voice.kheelona.com/concepts/latency.md).
- **Kheelona keeps no conversation.** No audio and no conversation text is stored.

## Where to go next

- [Quickstart for browsers](https://docs.voice.kheelona.com/get-started/quickstart-browser.md): a page that talks, in about 10 minutes.
- [Quickstart for servers](https://docs.voice.kheelona.com/get-started/quickstart-server.md): a relay that talks, in a few minutes.
- [Errors and retries](https://docs.voice.kheelona.com/guides/errors-and-retries.md): every error code and what to do.
- [Security](https://docs.voice.kheelona.com/guides/security.md): keys, origins and a Content Security Policy.
- [Examples](https://docs.voice.kheelona.com/examples/browser-basic.md): the code, in Node, Python, browser and curl.

## Use these docs with an AI tool

Every page has a Markdown version: use **Download as Markdown** at the bottom of the page. To get the whole documentation
as one file, use **Download all docs (Markdown)** at the bottom of any page, or open [the single file](https://docs.voice.kheelona.com/llms-full.txt).
[llms.txt](https://docs.voice.kheelona.com/llms.txt) is an index of the Markdown pages.

---

# Quickstart for servers

> Run the example relay against Kheelona Voice in a few minutes, first with typed text, then with your own recording.

Source: https://docs.voice.kheelona.com/get-started/quickstart-server/

In a few minutes you will have a voice answering from your own machine. You need a Kheelona API key, and the example
folder for your language: the examples you received from the Kheelona team, with the SDK package. They are proprietary and
shared privately; the commands below run inside that folder, so `cd examples/node-server` means the `node-server` folder
of what you received.

- Node: Node 24 or newer and pnpm.
- Python: Python 3.12 or newer and [uv](https://docs.astral.sh/uv/).

## 1. Get an API key

Sign in to your Kheelona console. From the public launch, the first time anyone signs in to your console, they set up a second sign-in step with an authenticator app on their phone (before the launch it may be optional). Have the phone at hand if asked. Then open **API keys** and create a key. Copy it when it is shown: it is shown once. It
starts with `kv_live_`.

The key belongs on your server only. Never put it in a toy, an app or a web page.

## 2. Set it

```sh
export KHEELONA_API_KEY=kv_live_…
```

The examples read the key only from this variable. They never print it.

## 3. Install

**Node**

```sh
cd examples/node-server
pnpm install
```

**Python**

```sh
cd examples/python-server
uv sync
```

## 4. Run the relay with text

A typed turn needs no microphone and no recording. The relay creates a session, connects, sends your text, saves the
reply audio and ends the session.

**Node**

```sh
pnpm relay --text "Hello! Tell me a short story."
```

**Python**

```sh
uv run python relay.py --text "Hello! Tell me a short story."
```

## 5. What you should see

The relay prints each event on one line, and the text of what the voice says under it. The middle lines vary with the
reply. The last line is always the summary.

```text
event session.ready
event state
…
event reply.sentence
  text: Once upon a time, a little dragon found a shiny stone.
…
event reply.done
event state
event session.ended
session ses_… ended client_ended replies 1 audio 8.4s saved reply.wav
```

Open `reply.wav` to hear it (24 kHz, 16-bit, mono).

A typed turn gives one reply, as above. A recording can give more: the voice may hear it as more than one turn, and you
may see `input.speech_started` and `interrupted` events when the person, or the clip, keeps talking while the voice is
replying. Then there is more than one `reply.done`, and the count in the last line is higher. That is normal. See
[Turn-taking](https://docs.voice.kheelona.com/concepts/turn-taking.md).

If something is wrong, the relay prints one line to standard error and exits with 1:

```text
error unauthenticated request_id req_…
```

Quote that line if you ask us for help: a request ID comes with a refused request, and the session ID with an error during a session. An exit code of 2 means a usage mistake, such as a missing key or no
`--text` and no `--wav`. See [Errors and retries](https://docs.voice.kheelona.com/guides/errors-and-retries.md).

## 6. Send your own recording

The relay sends a WAV file the way a toy would: 20 ms frames, at real time. The file must be 16 kHz, 16-bit, mono. Convert
any recording with ffmpeg:

```sh
ffmpeg -i in.wav -ac 1 -ar 16000 -sample_fmt s16 question.wav
```

Then run the relay with it:

**Node**

```sh
pnpm relay --wav question.wav --out reply.wav
```

**Python**

```sh
uv run python relay.py --wav question.wav --out reply.wav
```

You now see `transcript.final` with what was heard, and the reply as before. The relay keeps sending silence after
your recording ends, as a live microphone does, until the reply is finished. Set the languages the session listens for with
`--languages`, for example `--languages en,hi`.

You can give both `--text` and `--wav`. The typed turn goes first and the recording follows once its reply is done. A
recording can be heard as more than one turn, so expect more than one `reply.done` and, at times, `interrupted` events.

## 7. Read the session back

Every session leaves a record: when it ran, how long, how many turns, why it ended. It never holds what was said. Read
the one you just ran with `GET /v1/sessions/{id}` (the ID is in the relay's last line). The call, with its fields and
errors, is in the [REST reference](https://docs.voice.kheelona.com/reference/generated/api/getsession.md). Your API key needs the `sessions:read` scope.

To see the call that creates a session, here is the same thing with curl (it needs `uuidgen`; the key goes to curl on
standard input so it never shows in a process list):

```sh title="create-session.sh"
API_URL="${KHEELONA_API_URL:-https://api.voice.kheelona.com}"
VOICE_ID="${KHEELONA_VOICE_ID:-lumi-dragon}"

# One Idempotency-Key per logical session: repeating the request with the same key returns the same session.
printf 'Authorization: Bearer %s\n' "$KHEELONA_API_KEY" | curl -sS --fail-with-body \
  -H @- \
  -H 'Content-Type: application/json' \
  -H "Idempotency-Key: $(uuidgen)" \
  -X POST "$API_URL/v1/sessions" \
  -d "{\"voice_id\": \"$VOICE_ID\", \"languages\": [\"en\", \"hi\"]}"
```

The answer holds a client secret. It works once and for about 60 seconds. Don't paste it into chat or logs.

## Next

- [Server-to-server](https://docs.voice.kheelona.com/guides/server-to-server.md): how the connection works, step by step.
- [Sessions](https://docs.voice.kheelona.com/concepts/sessions.md): client secrets, idempotency and ending.
- [Quickstart for browsers](https://docs.voice.kheelona.com/get-started/quickstart-browser.md): the same, from a web page with the browser SDK.
- [Limits](https://docs.voice.kheelona.com/concepts/limits.md): what is capped, and what "busy" means.

---

# Quickstart for browsers

> Talk to a Kheelona voice from a web page in about 10 minutes, with the example page and its small backend.

Source: https://docs.voice.kheelona.com/get-started/quickstart-browser/

In about 10 minutes you will talk to a voice from a web page on your own machine. The page uses the Kheelona browser
SDK. A small backend of yours creates the session, so your API key never reaches the page.

You need:

- Node 24 or newer;
- a Kheelona API key;
- the `browser-basic` example folder and the browser SDK package: the examples you received from the Kheelona team, with
  the SDK package. They are proprietary and shared privately, and not on a public package registry. The commands below
  run inside that folder, so `cd examples/browser-basic` means the `browser-basic` folder of what you received. Contact
  the Kheelona team for access.

## 1. Get an API key and allow your page

Sign in to your Kheelona console. From the public launch, the first time anyone signs in to your console, they set up a second sign-in step with an authenticator app on their phone (before the launch it may be optional). Have the phone at hand if asked. Then open **API keys** and create a key. Copy it when it is shown: it is shown once. It
starts with `kv_live_`.

Then tell Kheelona which page may talk to your voices. Open **Allowed origins** and add the address of your page. An
origin is the scheme, the host and the port, for example `https://app.example.com`.

For development you can run the page on `http://localhost:5180`. Kheelona has to turn on **development access** for
your account first: it lets any `http://localhost` address connect, on any port. To ask for it, write to the Kheelona
team and name the account it is for. Until it is on, `http://localhost` is refused like any origin that is not in your
list. Without it, or without
your origin in the list, the session is refused with `origin_not_allowed`.

## 2. Install the SDK

The SDK is the package file you received from the Kheelona team. In the example folder you received, the package file
is already beside `package.json`, which points at it, so a plain install is enough:

```sh
cd examples/browser-basic
npm install
```

To add the SDK to your own app instead, install it from the package file:
`npm install ./kheelona-voice-<version>.tgz`.

The example's backend serves the SDK files to the page from this install, so the page needs no bundler.

## 3. Run it

```sh
export KHEELONA_API_KEY=kv_live_…
npm start
```

Open `http://localhost:5180`, click **Start talking** and allow the microphone when the browser asks. Say hello. The
state line moves from `listening` to `thinking` to `speaking`, and your words and the voice's words appear as captions.
You can also type a message instead of speaking.

The example has four more controls. **End** ends the session. **Mute** switches the microphone off and on.
**Interrupt** stops the voice at once. If the browser will not play the voice, **Tap to allow sound** appears.

## 4. Settings

The backend reads these variables. Only the key is required.

| Variable | Default | Meaning |
|---|---|---|
| `KHEELONA_API_KEY` | none (exit 2 with a hint when missing) | Your API key. Server side only. |
| `KHEELONA_API_URL` | `https://api.voice.kheelona.com` | The API |
| `KHEELONA_VOICE_ID` | `lumi-dragon` | The voice |
| `HOST` / `PORT` | `127.0.0.1` / `5180` | Where the page and the backend listen |
| `ALLOWED_ORIGIN` | `http://localhost:5180` | The one page origin allowed to call the backend |
| `KHEELONA_REALTIME_ORIGINS` | `https://api.voice.kheelona.com wss://api.voice.kheelona.com` | The origins of the `connect` URLs, separated by spaces. The page may connect to these and to nothing else. |

`KHEELONA_REALTIME_ORIGINS` is the origins (scheme, host and port, no path) of the two `connect` URLs in the answer to
`POST /v1/sessions`: `websocket_url` and `webrtc_url`. The page's Content Security Policy lets it connect to these and to
nothing else. The default fits the hosted service, so leave it alone if your `connect` URLs are on
`api.voice.kheelona.com`. If they are on another host, read the origins from a session you created (the
[curl example](https://docs.voice.kheelona.com/examples/curl.md) creates one) and join them with a space. The curl example has a one-line helper, `realtime-origins.sh`, that reads the answer on standard input
(`./create-session.sh | ./realtime-origins.sh`, needs `jq`). It is this:

```bash title="realtime-origins.sh"
# $ANSWER holds the JSON answer of POST /v1/sessions
echo "$ANSWER" | jq -r '.connect | [.websocket_url, .webrtc_url] | map(select(. != null) | capture("^(?<o>[a-z]+://[^/]+)").o) | unique | join(" ")'
```

Export what it prints as `KHEELONA_REALTIME_ORIGINS`. A `ws://` or `wss://` origin and its `http://` or `https://`
twin are separate origins: list each one that appears in a `connect` URL.

## 5. The three files

These are the files that make it work. All of them, with the rest of the folder, are on the
[browser-basic example](https://docs.voice.kheelona.com/examples/browser-basic.md) page.

### The backend: `server.ts`

It serves the page, the SDK files and one route, `POST /api/voice-session`. The route creates a session with your key
and returns only what the page needs: the session ID, the client secret and the connection addresses.

```ts title="server.ts"
// The example's whole backend: it serves the page, the SDK files and the mint route from ONE origin (so no CORS), and
// it holds your API key. Run it with `npm start`, then open http://localhost:5180 (see Settings on the example's docs page).
import { readFile, realpath, stat } from 'node:fs/promises';
import { createServer as createHttpServer, type IncomingMessage, type Server, type ServerResponse } from 'node:http';
import { dirname, extname, resolve, sep } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';

const HERE = dirname(fileURLToPath(import.meta.url));
const REQUEST_TIMEOUT_MS = 15_000;

/** The settings (see Settings on the example's docs page). */
export interface BrowserEnv {
  apiKey: string;
  apiUrl: string;
  voiceId: string;
  /** The one page origin that may call the mint route. */
  allowedOrigin: string;
  /** Origins the page may connect to (the `connect` URLs the API returns), space separated, for the CSP. */
  realtimeOrigins: string;
  host: string;
  port: number;
}

/** Where the files come from. Tests pass their own folders. */
export interface ServerOptions {
  publicDir?: string;
  sdkDir?: string;
}

/** Reads the settings; throws an Error with a one-line hint when something is missing or wrong (exit code 2). */
export function readEnv(env: Record<string, string | undefined>): BrowserEnv {
  const apiKey = env['KHEELONA_API_KEY'];
  if (!apiKey) throw new Error('KHEELONA_API_KEY is not set. Create an API key in the Kheelona console and export it.');
  const allowedOrigin = (env['ALLOWED_ORIGIN'] ?? 'http://localhost:5180').trim();
  if (allowedOrigin === '' || allowedOrigin.toLowerCase() === 'null') {
    throw new Error("ALLOWED_ORIGIN must be your page's origin, for example https://app.example.com");
  }
  const realtimeOrigins = (env['KHEELONA_REALTIME_ORIGINS'] ?? 'https://api.voice.kheelona.com wss://api.voice.kheelona.com').trim();
  // The value goes into a response header: plain origins only.
  if (!/^[A-Za-z][A-Za-z0-9+.-]*:\/\/[^\s;,'"]+(?: [A-Za-z][A-Za-z0-9+.-]*:\/\/[^\s;,'"]+)*$/.test(realtimeOrigins)) {
    throw new Error('KHEELONA_REALTIME_ORIGINS must be origins separated by spaces, for example https://host wss://host');
  }
  const port = Number(env['PORT'] ?? '5180');
  if (!Number.isInteger(port) || port < 0 || port > 65535) throw new Error('PORT must be a port number.');
  return {
    apiKey,
    apiUrl: env['KHEELONA_API_URL'] || 'https://api.voice.kheelona.com',
    voiceId: env['KHEELONA_VOICE_ID'] || 'lumi-dragon',
    allowedOrigin,
    realtimeOrigins,
    host: env['HOST'] || '127.0.0.1',
    port,
  };
}

/** The strict policy the page runs under: this origin's files, and the Kheelona realtime origins to connect to. */
export function contentSecurityPolicy(realtimeOrigins: string): string {
  return [
    "default-src 'self'",
    "script-src 'self'",
    "style-src 'self'",
    `connect-src 'self' ${realtimeOrigins}`,
    "img-src 'self'",
    "media-src 'self'",
    "object-src 'none'",
    "base-uri 'self'",
    "form-action 'self'",
    "frame-ancestors 'none'",
  ].join('; ');
}

const TYPES: Readonly<Record<string, string>> = {
  '.html': 'text/html; charset=utf-8',
  '.js': 'text/javascript; charset=utf-8',
  '.css': 'text/css; charset=utf-8',
  '.map': 'application/json; charset=utf-8',
};

function send(res: ServerResponse, csp: string, status: number, type: string, body: string | Buffer, headers: Record<string, string> = {}): void {
  res.writeHead(status, {
    'Content-Type': type,
    'Content-Security-Policy': csp,
    'X-Content-Type-Options': 'nosniff',
    'Referrer-Policy': 'no-referrer',
    'Cache-Control': 'no-store',
    ...headers,
  });
  res.end(body);
}

const sendJson = (res: ServerResponse, csp: string, status: number, body: unknown, headers: Record<string, string> = {}): void => {
  send(res, csp, status, 'application/json', JSON.stringify(body), headers);
};

/**
 * Maps a request path to a file inside `root`, or `undefined`. The path is decoded first, then resolved, and refused
 * unless the result is still inside `root`: `..`, encoded dots and slashes, backslashes and NUL never get out.
 */
async function fileInside(root: string, requestPath: string): Promise<string | undefined> {
  let decoded: string;
  try {
    decoded = decodeURIComponent(requestPath);
  } catch {
    return undefined;
  }
  if (decoded.includes('\0') || decoded.includes('\\')) return undefined;
  const base = resolve(root);
  const file = resolve(base, `.${decoded}`);
  if (!file.startsWith(base + sep)) return undefined;
  if (!(extname(file) in TYPES)) return undefined;
  try {
    // A symlink must not lead out either: compare real paths. The root itself may be a link (pnpm links packages),
    // so it is resolved first and is the one explicitly trusted folder.
    const realFile = await realpath(file);
    if (!realFile.startsWith((await realpath(base)) + sep)) return undefined;
    return (await stat(realFile)).isFile() ? realFile : undefined;
  } catch {
    return undefined;
  }
}

async function serveStatic(req: IncomingMessage, res: ServerResponse, csp: string, dirs: Required<ServerOptions>): Promise<void> {
  const path = (req.url ?? '/').split('?')[0] ?? '/';
  const isSdk = path.startsWith('/sdk/');
  const file = await fileInside(isSdk ? dirs.sdkDir : dirs.publicDir, isSdk ? path.slice('/sdk'.length) : path === '/' ? '/index.html' : path);
  if (file === undefined) {
    sendJson(res, csp, 404, { error: { code: 'not_found' } });
    return;
  }
  send(res, csp, 200, TYPES[extname(file)] ?? 'application/octet-stream', req.method === 'HEAD' ? '' : await readFile(file));
}

/** The mint route: create the session with your API key, hand the page only what it needs. */
async function mintSession(env: BrowserEnv, req: IncomingMessage, res: ServerResponse, csp: string): Promise<void> {
  // Same-origin by design (no CORS headers). Only your own page may call this route: a request with no Origin, or
  // another one, never reaches Kheelona. JSON only, so a cross-site HTML form cannot spend your key.
  if (req.headers.origin !== env.allowedOrigin) {
    sendJson(res, csp, 403, { error: { code: 'origin_not_allowed' } });
    return;
  }
  if (!req.headers['content-type']?.toLowerCase().startsWith('application/json')) {
    sendJson(res, csp, 415, { error: { code: 'unsupported_media_type' } });
    return;
  }
  // Authenticate your user here: only signed-in users may start a session.
  // The request body is never read (this route takes no input), so its size and Content-Length don't matter.
  // One attempt only: to retry on 429/503, reuse one Idempotency-Key as examples/node-server does.
  const upstream = await fetch(`${env.apiUrl.replace(/\/+$/, '')}/v1/sessions`, {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${env.apiKey}`,
      'Content-Type': 'application/json',
      'Idempotency-Key': globalThis.crypto.randomUUID(),
    },
    body: JSON.stringify({
      voice_id: env.voiceId,
      languages: ['en', 'hi'],
      end_user_ref: 'demo-user', // replace with the signed-in user's opaque ID
    }),
    signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS),
  });
  if (!upstream.ok) {
    const body = (await upstream.json().catch(() => ({}))) as { error?: { code?: unknown; request_id?: unknown } };
    const code = plain(body.error?.code) ?? 'unknown_error';
    const requestId = plain(body.error?.request_id);
    console.error(`error ${code} request_id ${requestId ?? '-'}`);
    const retryable = upstream.status === 429 || upstream.status === 503;
    const retryAfter = upstream.headers.get('retry-after');
    sendJson(res, csp, retryable ? upstream.status : 502, { error: { code, ...(requestId ? { requestId } : {}) } }, retryable && retryAfter && /^\d+$/.test(retryAfter) ? { 'Retry-After': retryAfter } : {});
    return;
  }
  const created = (await upstream.json()) as {
    session_id: string;
    client_secret: { value: string };
    connect: { webrtc_url: string; websocket_url: string };
  };
  // Return only what the page needs: never the API key, and nothing else from the answer.
  sendJson(res, csp, 201, {
    sessionId: created.session_id,
    clientSecret: created.client_secret.value,
    endpoints: { webrtcUrl: created.connect.webrtc_url, websocketUrl: created.connect.websocket_url },
  });
}

/** A short identifier made of plain characters, or undefined: nothing else from an answer is repeated. */
function plain(value: unknown): string | undefined {
  return typeof value === 'string' && /^[A-Za-z0-9_.:-]{1,64}$/.test(value) ? value : undefined;
}

/** Creates the server (not listening yet): the page, the SDK files and the mint route on one origin. */
export function createServer(env: BrowserEnv, options: ServerOptions = {}): Server {
  const dirs: Required<ServerOptions> = {
    publicDir: options.publicDir ?? resolve(HERE, 'public'),
    sdkDir: options.sdkDir ?? resolve(HERE, 'node_modules/@kheelona/voice/dist'),
  };
  const csp = contentSecurityPolicy(env.realtimeOrigins);
  return createHttpServer((req, res) => {
    void (async () => {
      const path = (req.url ?? '/').split('?')[0];
      if (path === '/api/voice-session') {
        if (req.method !== 'POST') {
          sendJson(res, csp, 405, { error: { code: 'method_not_allowed' } }, { Allow: 'POST' });
          return;
        }
        await mintSession(env, req, res, csp);
      } else if (req.method === 'GET' || req.method === 'HEAD') {
        await serveStatic(req, res, csp, dirs);
      } else {
        sendJson(res, csp, 405, { error: { code: 'method_not_allowed' } }, { Allow: 'GET, HEAD' });
      }
    })().catch((error: unknown) => {
      // Only the kind of failure: an error message can echo a request.
      console.error(`error ${error instanceof Error ? error.name : 'unexpected'} request_id -`);
      if (!res.headersSent) sendJson(res, csp, 502, { error: { code: 'internal' } });
      else res.end();
    });
  });
}

if (process.argv[1] && pathToFileURL(process.argv[1]).href === import.meta.url) {
  let env: BrowserEnv;
  try {
    env = readEnv(process.env);
  } catch (error) {
    console.error((error as Error).message);
    process.exit(2);
  }
  createServer(env).listen(env.port, env.host, () => {
    console.log(`page on http://${env.host === '127.0.0.1' ? 'localhost' : env.host}:${String(env.port)} (page origin ${env.allowedOrigin})`);
  });
}
```

### The page: `public/index.html`

```html title="public/index.html"
<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <title>Kheelona Voice: browser example</title>
    <link rel="stylesheet" href="/style.css" />
  </head>
  <body>
    <main>
      <h1>Talk to a Kheelona voice</h1>
      <p class="status">State: <strong id="state">not connected</strong></p>
      <div class="row">
        <button id="start" type="button">Start talking</button>
        <button id="end" type="button" disabled>End</button>
        <button id="allow-sound" type="button" hidden>Tap to allow sound</button>
      </div>
      <div class="row">
        <button id="mute" type="button" aria-pressed="false" disabled>Mute</button>
        <button id="interrupt" type="button" disabled>Interrupt</button>
      </div>
      <p id="error" role="alert" hidden></p>
      <div id="captions" aria-live="polite"></div>
      <div class="row">
        <label for="say">Or type a message</label>
        <input id="say" type="text" maxlength="2000" autocomplete="off" disabled />
        <button id="send" type="button" disabled>Send</button>
      </div>
    </main>
    <script type="module" src="/main.js"></script>
  </body>
</html>
```

### The script: `public/main.js`

It asks your backend for a session, connects with the SDK, and listens. The listeners are attached **after**
`await connect()`. The [browser SDK guide](https://docs.voice.kheelona.com/guides/browser-sdk.md) explains each step.

```js title="public/main.js"
/* global document, fetch */
// The page: mint a session on your server, connect with the SDK, show what is said.
// A plain ES module, no build step. The CSP allows only this origin: no inline scripts or styles.
import { KheelonaVoice } from '/sdk/index.js';
import { describeError, MintError } from '/errors.js';
import { wireControls } from '/controls.js';

const $ = (id) => document.getElementById(id);
let session = null;

async function connectToVoice() {
  // Your server creates the session with your API key. The page only ever gets a single-use client secret.
  const res = await fetch('/api/voice-session', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: '{}',
  });
  if (!res.ok) {
    const body = await res.json().catch(() => ({}));
    const retryAfter = Number(res.headers.get('Retry-After'));
    throw new MintError(body.error?.code ?? 'mint_failed', res.status, retryAfter > 0 ? retryAfter * 1000 : undefined);
  }
  const { sessionId, clientSecret, endpoints } = await res.json();
  // Call this from a click: the browser starts the audio and the microphone only after a user gesture.
  // A failed connect needs a new session (a new secret), so every attempt mints again.
  return KheelonaVoice.connect({ sessionId, clientSecret, endpoints, workletBaseUrl: '/sdk/' });
}

function listen(voice) {
  // Attach after `await connect()`: nothing is missed, and `state` first gives the current state.
  voice.on('state', (state) => {
    $('state').textContent = state;
  });
  voice.on('transcript', ({ text }) => addCaption('You', text));
  voice.on('replySentence', ({ text }) => addCaption('Voice', text));
  voice.on('error', (e) => {
    if (e.code === 'audio_unavailable') {
      // The browser refused to play the voice: ask for a tap (see the "Tap to allow sound" button).
      $('allow-sound').hidden = false;
    } else {
      showError(e);
    }
  });
  voice.on('ended', () => finish());
}

// Captions are set with textContent only: what people and the voice say is never read as HTML.
function addCaption(who, text) {
  const line = document.createElement('p');
  line.className = who === 'You' ? 'you' : 'voice';
  line.textContent = `${who}: ${text}`;
  $('captions').append(line);
}

function showError(e) {
  // Show the code (and when to try again); never log error.cause, it can quote connection details.
  $('error').textContent = describeError(e);
  $('error').hidden = false;
}

function setRunning(running) {
  $('start').disabled = running;
  $('end').disabled = !running;
  $('say').disabled = !running;
  $('send').disabled = !running;
  $('mute').disabled = !running;
  $('interrupt').disabled = !running;
}

function finish() {
  session = null;
  setRunning(false);
  $('allow-sound').hidden = true;
  // The next session starts unmuted.
  $('mute').setAttribute('aria-pressed', 'false');
  $('mute').textContent = 'Mute';
  $('state').textContent = 'ended';
}

$('start').addEventListener('click', async () => {
  $('error').hidden = true;
  setRunning(true);
  $('state').textContent = 'connecting';
  try {
    session = await connectToVoice();
    listen(session);
  } catch (e) {
    showError(e);
    finish();
    $('state').textContent = 'not connected';
  }
});

wireControls(() => session, { muteButton: $('mute'), interruptButton: $('interrupt') });

$('end').addEventListener('click', async () => {
  if (session) await session.end();
});

function sendTyped() {
  const text = $('say').value.trim();
  if (!session || text === '') return;
  // The caption comes back as a `transcript` event, like a spoken turn's.
  session.sendText(text);
  $('say').value = '';
}
$('send').addEventListener('click', sendTyped);
$('say').addEventListener('keydown', (e) => {
  if (e.key === 'Enter') sendTyped();
});

// A tap is the gesture the browser needs: call resumeAudio() directly in the handler, with no await before it.
$('allow-sound').addEventListener('click', () => {
  session?.resumeAudio();
  $('allow-sound').hidden = true;
});
```

## Troubleshooting

**`origin_not_allowed`.** The origin of your page is not in your allowed origins. Add it in the console, with the exact
scheme, host and port. For `http://localhost`, ask the Kheelona team to turn on development access. A page opened from a
file (`file://`) has no usable origin and is always refused. Open it through the backend instead.

**The microphone is denied.** The page shows `microphone_unavailable`. Allow the microphone for the page in the browser's
site settings, then click Start again: a failed start needs a new session, and the page makes one each time. Browsers
give a page the microphone only on `https` or on `localhost`. Check that no other program is using it.

**You hear nothing.** Browsers play sound only after a click or a tap. Start the session from a click, as the example
does. If the browser still refuses, the SDK sends an `audio_unavailable` error and the example shows **Tap to allow
sound**: tap it. In your own page, call `session.resumeAudio()` directly in the click handler. See the
[browser SDK guide](https://docs.voice.kheelona.com/guides/browser-sdk.md#no-sound).

**It connected, but through the WebSocket.** The SDK tries WebRTC first and moves to a WebSocket if WebRTC is not
connected within 3 seconds, for example on a network that blocks it. Everything works the same. Check
`session.transport` to see which one you got.

**Something else went wrong.** The page shows an error code. Look it up in [Errors and retries](https://docs.voice.kheelona.com/guides/errors-and-retries.md).
Keep the code, and the request ID or the session ID when your page has one, if you ask us for help.

## Next

- [Browser SDK guide](https://docs.voice.kheelona.com/guides/browser-sdk.md): every option, event and call.
- [Security](https://docs.voice.kheelona.com/guides/security.md): before this page goes to real users.
- [Going live](https://docs.voice.kheelona.com/guides/going-live.md): the checklist.

---

# Sessions and client secrets

> What a session is, how a one-time client secret reaches the client, and how a session ends.

Source: https://docs.voice.kheelona.com/concepts/sessions/

A **session** is one conversation between one person and one voice. Your server creates it, a client connects to it,
and it ends. Everything else is built on that.

## Create a session on your server

`POST /v1/sessions` with your API key. You choose the voice and, if you like, the languages. The answer holds:

- `session_id`;
- `client_secret`: a `value` (starts with `kvcs_`) and its `expires_at`;
- `connect`: the `websocket_url` (for servers and toys) and the `webrtc_url` (for browsers and apps, used by the [browser SDK](https://docs.voice.kheelona.com/guides/browser-sdk.md));
- the pinned `voice`, the `languages`, the `limits`, and `warnings`.

The request and answer are in the [REST reference](https://docs.voice.kheelona.com/reference/generated/api/createsession.md).

**Node**

```ts title="kheelona.ts"
const REQUEST_TIMEOUT_MS = 15_000;
/** The longest wait for one retry, whatever `Retry-After` says. */
const MAX_WAIT_S = 30;

export interface CreateSessionOptions {
  apiUrl: string;
  /** Your API key. Keep it on the server: read it from the environment, never ship it to a browser or a toy. */
  apiKey: string;
  voiceId: string;
  /** Default `["en", "hi"]`. */
  languages?: string[];
  /** An opaque ID for your end user (not a name, e-mail address or other personal data). */
  endUserRef?: string;
  /** `false` = the voice is never cut off (for devices without echo cancellation). */
  interruptions?: boolean;
}

export interface CreatedSession {
  sessionId: string;
  /** The single-use client secret (`kvcs_…`). It is valid for 60 seconds. */
  clientSecret: string;
  websocketUrl: string;
  webrtcUrl: string;
}

/** An error answer from the API: the HTTP status, the error code and the request ID to quote to support. */
export class KheelonaApiError extends Error {
  readonly status: number;
  readonly code: string;
  readonly requestId: string | undefined;
  /** Seconds from the `Retry-After` header, if the API sent one. */
  readonly retryAfterSeconds: number | undefined;
  /** For status 0 (no answer): the name of the failure, e.g. `TimeoutError`. Never its message. */
  readonly causeName: string | undefined;

  constructor(status: number, code: string, requestId?: string, retryAfterSeconds?: number, causeName?: string) {
    super(`Kheelona API error ${String(status)} ${code}${requestId ? ` (request_id ${requestId})` : ''}`);
    this.name = 'KheelonaApiError';
    this.status = status;
    this.code = code;
    this.requestId = requestId;
    this.retryAfterSeconds = retryAfterSeconds;
    this.causeName = causeName;
  }
}

/** Creates a session with `POST /v1/sessions`. Retries 429 and 503 (up to 3 attempts) with one idempotency key. */
export async function createSession(opts: CreateSessionOptions): Promise<CreatedSession> {
  const idempotencyKey = globalThis.crypto.randomUUID(); // one per logical session: a retry never creates a second one
  const body = JSON.stringify({
    voice_id: opts.voiceId,
    languages: opts.languages ?? ['en', 'hi'],
    ...(opts.endUserRef === undefined ? {} : { end_user_ref: opts.endUserRef }),
    ...(opts.interruptions === undefined ? {} : { interruptions: opts.interruptions }),
  });
  for (let attempt = 1; ; attempt += 1) {
    let res: Response;
    try {
      res = await fetch(`${opts.apiUrl.replace(/\/+$/, '')}/v1/sessions`, {
        method: 'POST',
        headers: { Authorization: `Bearer ${opts.apiKey}`, 'Content-Type': 'application/json', 'Idempotency-Key': idempotencyKey },
        body,
        signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), // never wait forever for the API
      });
    } catch (error) {
      // Keep only the kind of failure: its message can echo the request.
      throw new KheelonaApiError(0, 'upstream_unreachable', undefined, undefined, error instanceof Error ? error.name : 'Error');
    }
    if (res.ok) {
      const created = (await res.json()) as {
        session_id: string;
        client_secret: { value: string };
        connect: { websocket_url: string; webrtc_url: string };
      };
      return {
        sessionId: created.session_id,
        clientSecret: created.client_secret.value,
        websocketUrl: created.connect.websocket_url,
        webrtcUrl: created.connect.webrtc_url,
      };
    }
    const error = await readError(res);
    const retryable = res.status === 429 || res.status === 503;
    if (!retryable || attempt >= 3) throw error;
    await new Promise((resolve) => setTimeout(resolve, Math.min(error.retryAfterSeconds ?? 1, MAX_WAIT_S) * 1000));
  }
}

/** Keeps only the error code and request ID from the answer (never the whole body). */
async function readError(res: Response): Promise<KheelonaApiError> {
  const header = res.headers.get('retry-after');
  const retryAfter = header !== null && /^\d+$/.test(header) ? Math.min(Number(header), Number.MAX_SAFE_INTEGER) : undefined; // a huge value stays a number the cap handles
  try {
    const parsed = (await res.json()) as { error?: { code?: unknown; request_id?: unknown } };
    const code = plain(parsed.error?.code) ?? 'unknown_error';
    const requestId = plain(parsed.error?.request_id);
    return new KheelonaApiError(res.status, code, requestId, retryAfter);
  } catch {
    return new KheelonaApiError(res.status, 'unknown_error', undefined, retryAfter);
  }
}

/** A short identifier made of plain characters, or undefined: nothing else from an answer is repeated. */
function plain(value: unknown): string | undefined {
  return typeof value === 'string' && /^[A-Za-z0-9_.:-]{1,64}$/.test(value) ? value : undefined;
}
```

**Python**

```python title="kheelona.py"
import asyncio
import re
import uuid
from collections.abc import Sequence

import httpx

MAX_ATTEMPTS = 3
MAX_WAIT_S = 30


class KheelonaApiError(Exception):
    """The API refused the request. Carries the status, error code and request id, never the key."""

    def __init__(self, status: int, code: str, request_id: str | None, retry_after: int | None = None):
        super().__init__(f"Kheelona API error {status} {code} (request_id {request_id or '-'})")
        self.status = status
        self.code = code
        self.request_id = request_id
        self.retry_after = retry_after


async def create_session(
    client: httpx.AsyncClient,
    *,
    api_url: str,
    api_key: str,
    voice_id: str,
    languages: Sequence[str] | None = None,
    end_user_ref: str | None = None,
    interruptions: bool = True,
) -> dict:
    """Create a session and return the API's answer: session_id, client_secret, connect, ...

    One Idempotency-Key covers every attempt, so a retry never creates a second session. A 429 or 503 is
    retried up to 3 times in all, after the seconds in its Retry-After header (1 if it has none).
    """
    body: dict = {"voice_id": voice_id, "languages": list(languages or ["en", "hi"])}
    if end_user_ref:
        body["end_user_ref"] = end_user_ref  # an opaque id of yours, never a name or an email
    if not interruptions:
        body["interruptions"] = False
    headers = {"Authorization": f"Bearer {api_key}", "Idempotency-Key": str(uuid.uuid4())}
    for attempt in range(1, MAX_ATTEMPTS + 1):
        response = await client.post(f"{api_url}/v1/sessions", json=body, headers=headers)
        if response.is_success:
            return response.json()
        error = _api_error(response)
        if response.status_code not in (429, 503) or attempt == MAX_ATTEMPTS:
            raise error
        await asyncio.sleep(min(error.retry_after or 1, MAX_WAIT_S))
    raise AssertionError("unreachable")


def _api_error(response: httpx.Response) -> KheelonaApiError:
    try:
        detail = response.json()["error"]
        code, request_id = _plain(detail["code"]) or "unknown_error", _plain(detail.get("request_id"))
    except (ValueError, KeyError, TypeError):
        code, request_id = "unknown_error", None
    retry_after = response.headers.get("Retry-After", "")
    return KheelonaApiError(response.status_code, code, request_id, int(retry_after) if retry_after.isdigit() else None)


def _plain(value: object) -> str | None:
    """A short identifier made of plain characters, or None: nothing else from an answer is repeated."""
    return value if isinstance(value, str) and re.fullmatch(r"[A-Za-z0-9_.:-]{1,64}", value) else None
```

## The client secret

The client secret is what lets one client connect to one session. It is not your API key.

- **It lives about 60 seconds** and works **once**. It is used up when the connection reaches `session.ready`.
- **Mint it on your server.** Your API key stays there. A browser, an app or a toy gets only the client secret and the
  session ID.
- Treat it like a password for those 60 seconds: don't log it and don't put it in a URL you share.
- If the connection fails for any reason, create a **new session**. A failed `connect()` needs a new secret.

### A mint route for your own pages

A web page or an app cannot hold your key, so it asks your server for a session (the
[browser quickstart](https://docs.voice.kheelona.com/get-started/quickstart-browser.md) shows the whole path). The example mint route is
`POST /api/voice-session`. It returns only `sessionId`, `clientSecret` and `endpoints`.

**Node**

```ts title="server.ts"
export function createApp(env: ExampleEnv): express.Express {
  const app = express();
  app.disable('x-powered-by');

  // Same-origin by design (no CORS headers): serve your page and this route from one origin.
  // Only your own page may call this route: a request with no Origin, or another one, never reaches Kheelona.
  const guard: express.RequestHandler = (req, res, next) => {
    const origin = req.get('origin');
    if (origin !== env.allowedOrigin) {
      res.status(403).json({ error: { code: 'origin_not_allowed' } });
      return;
    }
    res.set({ Vary: 'Origin', 'Cache-Control': 'no-store' });
    // JSON only: a cross-site HTML form cannot send it, so it cannot spend your key.
    if (!req.is('application/json')) {
      res.status(415).json({ error: { code: 'unsupported_media_type' } });
      return;
    }
    next();
  };

  app.all('/api/voice-session', guard);
  // express.json (body-parser) checks Content-Length against the limit before it reads the body.
  app.post('/api/voice-session', express.json({ limit: '2kb', type: 'application/json' }), async (req, res) => {
    const body = req.body as Record<string, unknown> | undefined;
    // Only `languages` is accepted. (Language codes here are 2-3 letters, as the API's are; the API validates too.)
    const languages = body?.['languages'];
    const knownKeys = typeof body === 'object' && !Array.isArray(body) && Object.keys(body).every((key) => key === 'languages');
    const validLanguages =
      languages === undefined ||
      (Array.isArray(languages) && languages.length >= 1 && languages.length <= 11 && languages.every((l) => typeof l === 'string' && /^[a-z]{2,3}$/.test(l)));
    if (!knownKeys || !validLanguages) {
      res.status(400).json({ error: { code: 'invalid_request' } });
      return;
    }
    try {
      // Authenticate your user here: only signed-in users may start a session.
      const session = await createSession({
        apiUrl: env.apiUrl,
        apiKey: env.apiKey,
        voiceId: env.voiceId,
        ...(languages === undefined ? {} : { languages: languages as string[] }),
        endUserRef: 'demo-user', // replace with the signed-in user's opaque ID
      });
      // Return only what the page needs: never the API key, and nothing else from the answer.
      res.status(201).json({
        sessionId: session.sessionId,
        clientSecret: session.clientSecret,
        endpoints: { webrtcUrl: session.webrtcUrl, websocketUrl: session.websocketUrl },
      });
    } catch (error) {
      const api = error instanceof KheelonaApiError ? error : undefined;
      console.error(`mint failed: ${api ? `${String(api.status)} ${api.code} request_id ${api.requestId ?? '-'}` : 'unexpected error'}`);
      if (api && (api.status === 429 || api.status === 503)) {
        if (api.retryAfterSeconds !== undefined) res.set('Retry-After', String(api.retryAfterSeconds));
        res.status(api.status);
      } else {
        res.status(502);
      }
      res.json({ error: { code: api?.code ?? 'internal', ...(api?.requestId ? { requestId: api.requestId } : {}) } });
    }
  });

  // A malformed JSON body is the caller's mistake, not ours.
  // Express recognises an error handler by its four parameters, so `_next` stays.
  // eslint-disable-next-line @typescript-eslint/no-unused-vars
  app.use((error: unknown, _req: express.Request, res: express.Response, _next: express.NextFunction) => {
    const tooLarge = (error as { status?: number }).status === 413;
    res.status(tooLarge ? 413 : 400).json({ error: { code: tooLarge ? 'payload_too_large' : 'invalid_request' } });
  });
  return app;
}
```

**Python**

```python title="app.py"
@app.post("/api/voice-session")
async def mint(request: Request) -> JSONResponse:
    # Only your own page may call this: a missing or foreign Origin is refused before anything else.
    if request.headers.get("origin") != allowed_origin:
        return _error(403, "origin_not_allowed")
    # JSON only, so a cross-site HTML form cannot start a session.
    media_type, _, _ = request.headers.get("content-type", "").partition(";")
    if media_type.strip().lower() != "application/json":
        return _error(415, "unsupported_media_type", NO_STORE)
    # Read the body in pieces and stop as soon as it passes the limit, even with no Content-Length.
    raw = bytearray()
    async for chunk in request.stream():
        raw += chunk
        if len(raw) > MAX_BODY_BYTES:
            return _error(413, "payload_too_large", NO_STORE)
    try:
        languages = _languages(json.loads(raw))
    except ValueError:
        return _error(400, "invalid_request", NO_STORE)

    # Authenticate your user here: only signed-in users may start a session.
    # The end_user_ref is that user's opaque id (never a name or an email); this demo uses a fixed one.
    try:
        session = await create_session(
            client,
            api_url=api_url,
            api_key=api_key,
            voice_id=voice_id,
            languages=languages,
            end_user_ref="demo-user",
        )
    except KheelonaApiError as err:
        log.warning("create session failed: status=%s code=%s request_id=%s", err.status, err.code, err.request_id)
        body = {"code": err.code, **({"requestId": err.request_id} if err.request_id else {})}
        if err.status in (429, 503):
            headers = {**NO_STORE, **({"Retry-After": str(err.retry_after)} if err.retry_after else {})}
            return JSONResponse({"error": body}, status_code=err.status, headers=headers)
        return JSONResponse({"error": body}, status_code=502, headers=NO_STORE)
    except httpx.HTTPError as err:
        log.warning("create session failed: %s", type(err).__name__)
        return _error(502, "upstream_unreachable", NO_STORE)

    # Return only what the page needs, nothing else from the API's answer.
    return JSONResponse(
        {
            "sessionId": session["session_id"],
            "clientSecret": session["client_secret"]["value"],
            "endpoints": {
                "webrtcUrl": session["connect"]["webrtc_url"],
                "websocketUrl": session["connect"]["websocket_url"],
            },
        },
        status_code=201,
        headers=NO_STORE,
    )
```

Before you copy it into production:

- **Authenticate your user** where the example says so. Without that, anyone who can reach the route can create
  sessions on your key and use up your limits.
- The route listens on loopback (`127.0.0.1`) by default and accepts only one page origin, only as JSON, and sends no
  CORS headers: it is meant to be same-origin with your page. Keep it that way, or put it behind your own gateway.
- Pass your user's **opaque ID** as `end_user_ref`: an ID of yours such as `u_8f3a2c`. Never a name, an email address or
  anything else that identifies a child. It is kept with the session record.

## Idempotency-Key

Send an `Idempotency-Key` header with `POST /v1/sessions`: one new UUID per logical session. If the request is sent
twice, for example after a timeout, the second answer is the **same session** with a **new client secret** and the old
secret stops working. You never end up with two sessions for one tap on a toy.

- Keys are remembered for 10 minutes.
- Use the same key when you retry the same request. Use a new key for a new session.
- Replaying a key for a session that is already live, or for a different request, answers `409` `invalid_request`.

## A session that is never connected

Creating a session reserves a place for it. If nothing connects within 60 seconds, the reservation expires and the place
is freed. A reserved session counts towards your concurrent-session limit until it connects, expires or is ended, so
don't create sessions far ahead of use. See [Limits](https://docs.voice.kheelona.com/concepts/limits.md).

## Ending a session

A session ends when:

- the client sends `session.end` (`client_ended`);
- your server calls `POST /v1/sessions/{id}/end` (`partner_ended`). It also cancels a reserved session that was never
  connected. Ending a session that is already over does nothing;
- it reaches its maximum length (`max_duration`), or nobody speaks and nothing plays for long enough
  (`idle_timeout`; see [Quiet sessions](#quiet-sessions) below);
- the connection drops (`disconnected`), or Kheelona stops the session (`server_shutdown`, `provider_failure`,
  `suspended`, `service_stopped`, `error`). `service_stopped` means the voice service was switched off; start a new
  session once it is back.

`session.ended` tells the client the reason. After that the session cannot be used again. For another conversation, create a
new session.

## Quiet sessions

When nobody speaks, the voice does not just wait for the idle timeout. This happens by default:

1. After about 20 seconds of silence, the voice gently asks if the person is still there.
2. If it is still quiet about 40 seconds after that, the voice says a friendly goodbye, and the session ends with the
   reason `idle_timeout`.

So, once the voice has spoken, a quiet session ends after about a minute, even if the idle timeout you were given is
longer. The idle timeout is now the longest a session can stay silent. Before the voice has spoken (the greeting is off,
the default, and the person has not said anything yet) there is nothing to check in on: that session lasts until the idle
timeout, and then ends with the goodbye. If your idle timeout is short (30 seconds, say), the check-in still comes
first and the goodbye follows at the limit.

What your client sees: a `session.warning` with the code `idle_soon` at least 20 seconds before the end (the goodbye takes a few seconds more), two short replies
with no new `transcript.final` before them (like a greeting), then `reply.done` and `session.ended` with the reason
`idle_timeout`. Any speech from the person, or a typed turn, starts the count again. Muting the microphone pauses the check-in and goodbye counts, but not the idle timeout: a muted session that nobody comes back to still ends at the idle timeout, with the goodbye first.
Kheelona can switch this off per deployment, and then a quiet session simply ends at the idle timeout, in silence.
## What Kheelona stores

**No audio and no conversation text is ever stored or logged.** That covers what the person says, what the voice says and
the `instructions` you send for a session, in readable form. Instructions reach the voice encrypted and are held in a
short-lived encrypted record until the connection starts (deleted within about two minutes).

What is kept is the record of the session, with counts instead of words:

- who and how: your partner and API key, your `end_user_ref` and `metadata`, the client kind, the voice, the declared
  languages and the languages heard;
- when and how long: created, connected and ended times, and the duration;
- how it went: the number of turns and interruptions, response times, audio seconds, token counts, and counts of
  safety actions (by category) and errors (by code);
- why it ended: the reason;
- the connection: the IP address, the user agent and the web origin, if any, and the country and region the IP address comes from (never an exact location). IP addresses are shortened after 30 days; the country and region stay with the record.

Session records are kept for 12 months (see the [data and privacy policy](https://docs.voice.kheelona.com/policies/data-and-privacy.md)). You can read them with `GET /v1/sessions` and `GET /v1/sessions/{id}`, and in your
console. Because `end_user_ref` and `metadata` are stored, keep personal data out of them.

---

# Languages

> Choose the languages of a session, what the default is, and what beta and alpha mean.

Source: https://docs.voice.kheelona.com/concepts/languages/

A session declares the languages it expects. Kheelona listens for those languages, and the voice replies in whichever
of them the person speaks.

| Code | Language | Status | Notes |
|---|---|---|---|
| `en` | English | Supported |  |
| `hi` | Hindi | Supported |  |
| `mr` | Marathi | Beta | Beta: recognition makes more mistakes than in Hindi. Test it with your users. |
| `bn` | Bengali | Beta | Beta: recognition makes more mistakes. Test it with your users. |
| `as` | Assamese | Alpha | Alpha: spoken Assamese is not recognised today: use typed turns. The voice's Assamese pronunciation is limited. |
| `gu` | Gujarati | Supported |  |
| `pa` | Punjabi | Beta | Beta: expect mistakes in recognition and pronunciation. A short turn can be taken for English if the session also declares English. |
| `ta` | Tamil | Supported |  |
| `te` | Telugu | Beta | Beta: recognition makes more mistakes, and the safety checks stop a few harmless sentences, so a child may get the gentle "talk to a grown-up" reply instead of an answer. |
| `kn` | Kannada | Beta | Beta: expect mistakes in recognition and pronunciation, and the safety checks stop a few harmless sentences. |
| `ml` | Malayalam | Alpha | Alpha: spoken Malayalam is often not recognised, so a turn may come back with no transcript. Typed turns work. The safety checks stop a few harmless sentences. |

Sessions default to `en, hi` when you declare no languages.

The same list is on the [supported languages page](https://docs.voice.kheelona.com/reference/languages.md) and comes from `GET /v1/languages`.

## Declare the languages

Send `languages` when you create the session: 1 to 11 distinct codes from the table, for example `["en", "ta"]`.
Without it a session uses `["en", "hi"]`, English and Hindi.

**We recommend English plus one Indian language.** Recognition listens only for the languages you declare, so a shorter
list is more accurate. Declare what the child really speaks.

**For the most reliable recognition, declare only the language the child speaks**, for example `["pa"]`. When a
session declares English and an Indian language, Kheelona decides for every turn which of the two it hears. On a clear
microphone that works well. With short turns in a noisy room, or a phone or laptop on loudspeaker, it can pick the
wrong one, more often in beta and alpha languages. If the child also speaks English, declare English too.

## Supported, beta and alpha

Every language has a stage. The table above shows it, and so does `GET /v1/languages` (`stage`).

- **Supported** (English, Hindi, Gujarati, Tamil): tested end to end. Speech is recognised reliably, and Kheelona tells
  the language apart from English.
- **Beta** (Telugu, Marathi, Bengali, Punjabi, Kannada): works, but speech recognition makes more mistakes. Some words
  are misheard, and a short turn can be taken for English when the session also declares English. Test with your users
  before you launch.
- **Alpha** (Malayalam, Assamese): early. Spoken turns are often not recognised, or come back wrong. Use typed turns,
  or use these languages for trials only, not as a child's main language.

In Telugu, Kannada and Malayalam the safety checks are also less precise: some harmless sentences may get the gentle
"talk to a grown-up you trust" reply instead of an answer. See [Safety levels](https://docs.voice.kheelona.com/concepts/safety-levels.md).

Beta and alpha languages are accepted, not refused. The answer to `POST /v1/sessions` carries a `warnings` list that
says so:

- `beta_languages`: lists the beta languages of this session;
- `alpha_languages`: lists the alpha languages of this session;
- `multiple_indian_languages`: you declared more than one Indian language, which makes recognition less certain.

`warnings` is empty when there is nothing to say. Show the warnings to whoever configures the product. They are advice
for you, not an error.

## Steer the reply language

The voice follows the person: it answers in the language it hears. To set a preference for a session, say so in
the `instructions` you send when you create it, for example "Answer in simple Hindi, even if the child speaks English."
Instructions are text only the voice sees. They are never stored in readable form. A language you steer towards should be one you
declared.

Typed turns work in every language. The `transcript.final` event tells you the language that was heard and its
stage (`language_stage`: `supported`, `beta` or `alpha`; `language_beta` is true for the last two).

---

# Limits, capacity and busy

> The limits on sessions, audio and API calls, and what to do when Kheelona is busy.

Source: https://docs.voice.kheelona.com/concepts/limits/

Some limits are fixed. Some are set for your account by Kheelona: ask the Kheelona team if one gets in your way.

## Sessions

| Limit | Default | Notes |
|---|---|---|
| Session length | 1,800 s | Set per partner by Kheelona, 60 to 3,600 s. A session can ask for less with `max_duration_s`, never more. |
| Idle timeout | 120 s | The longest a session stays open with nobody speaking and nothing playing. Set per partner, 30 to 600 s. Muting pauses the check-in and goodbye counts, not this timeout. A quiet session usually ends sooner: see [Quiet sessions](https://docs.voice.kheelona.com/concepts/sessions.md#quiet-sessions). |
| Client secret | 60 s, single use | See [Sessions](https://docs.voice.kheelona.com/concepts/sessions.md). |
| Reservation | 60 s | A session created but not connected. |
| Concurrent sessions | set per partner | Reserved and live sessions both count. |
| Monthly minutes | set per partner, optional | When it is used up, new sessions are refused with `forbidden`. |
| Languages | 1 to 11 | See [Languages](https://docs.voice.kheelona.com/concepts/languages.md). |
| `instructions` | 4,000 characters | |
| `metadata` | 10 keys, 200 characters per value, 2 KB in all | Stored: no personal data. |
| `end_user_ref` | 128 characters | Letters, digits and `. _ : -` only. |

## Audio and events

| Limit | Value |
|---|---|
| Input audio speed | at most 2 times real time, and at most 1 s ahead of real time |
| Input frame | 10 to 500 ms |
| One WebSocket message | 64 KB; a larger one closes the connection |
| One client event | 8 KB |
| One typed turn (`input.text`) | 2,000 characters |
| Typed turns per session | 5 at once, then 1 per second. A turn beyond that is dropped and the session gets a non-fatal `error` with `rate_limited` (`retryable: true`). Send it again after a second. |
| One spoken turn | 60 s of speech; a longer turn is cut and processed |

## API calls

Limits are per API key.

| Calls | Rate | Burst |
|---|---|---|
| `POST /v1/sessions` | 10 per second | 20 |
| Every other `/v1` call | 20 per second | 40 |

Responses carry `RateLimit-Limit`, `RateLimit-Remaining` and `RateLimit-Reset`, so you can slow down before you are
refused. Over the limit you get `429` with `rate_limited` and a `Retry-After`. More than 20 failed authentications a minute from one IP
address are refused for 10 minutes.

A relay normally creates one session per conversation, far below these rates. If a launch will create many sessions at
once, tell us beforehand.

## Capacity and "busy"

Kheelona runs a fixed amount of speaking capacity: people talking and getting replies at the same moment. Kheelona sets it for
the whole service, and sets how many sessions your account may run together.

A new session is checked when you create it, in this order: your account status, your monthly minutes, your concurrent
sessions, then the capacity of the service. You get one of these answers:

- `201`: go ahead.
- `429` `partner_limit_reached`: your account is at its concurrent-session limit. Wait for a session to end, then try
  again. If you need more, ask the Kheelona team.
- `503` `capacity_exhausted`: the service is full right now. The answer has a `Retry-After` of 5 to 30 seconds. Wait that
  long and try again. For a toy, say something like "I'm busy, try again in a moment" from the device itself.
- `503` `capacity_exhausted` with `details.reason` set: the voice service is not running right now.
  `voice_service_starting` means it is waking up: retry after 60 seconds (the answer's `Retry-After`).
  `voice_service_asleep` means it is switched off to save cost: retry after 300 seconds, and tell the Kheelona team if
  it stays asleep when you need it.
- `403` `forbidden`: the account is suspended or the monthly minutes are used up. Retrying will not help.

`GET /v1/health` needs no key and says `capacity: available` or `busy`. It is cached for 5 seconds, so use it to
choose between two messages for the user, not to decide whether a session will succeed: only the create call decides.

See [Errors and retries](https://docs.voice.kheelona.com/guides/errors-and-retries.md) for how to retry.

---

# Voices and versions

> What a Kheelona voice is, where to find the voices you can use, and why a session always keeps one version.

Source: https://docs.voice.kheelona.com/concepts/voices/

A **Kheelona voice** is a complete character: how it sounds, its personality, how it speaks and how strict its safety
rules are. You pick a voice. You never pick or see what runs underneath.

Each voice has an ID, such as `lumi-dragon`, which you pass as `voice_id` when you create a session.

## Find your voices

You only see the voices Kheelona has enabled for your account, and that list can differ from another partner's. There is
no public catalogue. Look in two places:

- the **Voices** page of your console: each voice with its name, its safety level and a sample clip to listen to;
- `GET /v1/voices` with your API key (the `voices:read` scope): the same list as data. See
  [List voices](https://docs.voice.kheelona.com/reference/generated/api/listvoices.md).

The docs use `lumi-dragon` as the example voice. Yours may be different.

## Versions

A voice changes over time as Kheelona improves it. Each change is published as a new **version**, and a published version
never changes afterwards.

- **A new session uses the current version** and keeps exactly that version until it ends. If Kheelona publishes an
  update in the middle of a conversation, that conversation is not affected.
- `session.ready` and the answer of `POST /v1/sessions` tell you which version a session has pinned
  (`voice.id`, `voice.version`, `voice.name`).
- A voice behaves the same for as long as its version is the same. If you need to know when it changes, check the
  version in `GET /v1/voices`.

## Languages

Every voice can be used in all 11 languages; see the stages: Malayalam and Assamese are alpha, and their spoken use is early. You choose which languages a
session uses when you create it. See [Languages](https://docs.voice.kheelona.com/concepts/languages.md) for how to declare them and
[Supported languages](https://docs.voice.kheelona.com/reference/languages.md) for the notes on each language and its stage.

## Safety level

Each voice has a safety level, `child` or `general`, set by Kheelona. You cannot change it, and your `instructions` cannot
switch it off. See [Safety levels](https://docs.voice.kheelona.com/concepts/safety-levels.md).

## Instructions

You can add `instructions` to a session, for example the child's first name. They shape the voice for that session. They
are never stored in readable form. See [Sessions](https://docs.voice.kheelona.com/concepts/sessions.md).

---

# Turn-taking

> How the voice knows when the person has started and stopped talking, what happens when they carry on or interrupt, and the order of events.

Source: https://docs.voice.kheelona.com/concepts/turn-taking/

The person does not press a button. The voice listens all the time, notices when they start to speak and when they have
finished, and answers.

## States

A session is always in one of three states, sent in `state` events.

| State | Meaning |
|---|---|
| `listening` | Waiting for the person to speak. |
| `thinking` | The person finished and the reply is being prepared. No voice yet. |
| `speaking` | The reply is playing. |

A session also has the state `ended` in the browser SDK, at the end.

## How the end of a turn is found

The voice listens for speech, and when the person goes quiet it decides whether they have finished or are only pausing.
A short pause in the middle of a sentence is usually not the end of a turn. If the person only paused and carries on, see
"When the person carries on" below. After about 3 seconds of silence a
turn ends in any case. One spoken turn is at most 60 seconds of speech: a longer one is cut and processed.

You get `input.speech_started` when speech is detected and `input.speech_stopped` when the turn ends. A turn that turns out to hold no words goes from `thinking` straight back to `listening` with no transcript and no reply.

## When the person pauses in the middle of a thought

Sometimes a person stops in the middle of an idea ("I want to go to the…") or says "wait, let me think". Where the
voice supports it, it notices this from the words, not only from the sound, and does **not** answer yet. What you see:
`input.speech_stopped`, `state: thinking`, `transcript.final`, then `state: listening`, with no reply. If the person goes
on, the voice understands both parts together as one turn. If they stay quiet, about 5 seconds after they stopped the
voice says a short line in character, like "Go on, I'm listening!", with no new transcript before it. Treat that line
like a greeting.

A typed turn that comes after such a pause is understood together with the first part too. The voice says the line only
once per pause, and after it the next turn is a new one.

This is on by default where it is supported and can be switched off per deployment. A typed turn is never treated as
unfinished.

## When the words were not understood

If the person interrupts the voice and what they said could not be understood, the voice asks them in character to say it
again, and repeats its last question if they may have missed it. It also comes with no new transcript. It happens at
most once in a row.
## Early replies

Sometimes the voice is not sure the person has finished, and would wait a few seconds to be sure. After half a second of
quiet it can start preparing a reply in the background. Nothing is said and you see nothing: no event, no caption, no
audio. The turn's events come only when the turn is confirmed, in the usual order. If the person goes on, the prepared
reply is thrown away and the whole turn is understood again. If they stay quiet, the reply starts at once when the turn
is confirmed. The words are checked first, as always.

## The slow-reply line

If the answer is not ready about 2.5 seconds after the person finished, the voice says one short line so they are never
left in silence: "Hmm, let me think!", "One moment…", and similar, in the session's language. It arrives as an ordinary
first `reply.sentence` of the reply, with its audio. The answer follows as the next sentences of the same reply.

- It never comes before the turn is confirmed.
- It does not come if the answer is ready by then.
- It never comes after a sad or worried turn, or after a turn that the safety checks flagged. Then the voice answers
  when it is ready.
- It is not used for a greeting or a short in-character line, and not on two turns in a row.
- If the person speaks during the line, that reply ends with `interrupted` and `reply.done` with the reason
  `interrupted` (flush as usual), and their speech joins the same turn. In the last moment of the line, once the answer itself is about to start, speech is treated like speech over the answer (see [Interruptions and echo](https://docs.voice.kheelona.com/guides/interruptions-and-echo.md)): the voice pauses and the same rules apply.

Treat it like any other sentence. It can be switched off per deployment.

## When the person carries on

- **While the voice is thinking,** before any reply audio has played, new speech joins the same turn. The reply being
  prepared is dropped, and the whole turn is understood again as one. This is so that "Hi Lumi!" followed by a question
  gets one answer, not two.
- **While the voice is speaking,** new speech pauses it, and the words decide whether it stops. See
  [Interruptions and echo](https://docs.voice.kheelona.com/guides/interruptions-and-echo.md).
- **A typed turn** (`input.text`) is a turn of its own, and it ends whatever is in progress first: a reply being spoken
  is interrupted, and a reply being prepared is cancelled.

Replies never overlap: a new reply starts only after the previous one has ended.

## The order of events in one turn

1. `input.speech_started`
2. `input.speech_stopped`
3. `state: thinking`
4. `transcript.final`
5. `reply.started`
6. `state: speaking`, together with the first `reply.sentence` and the first audio
7. more `reply.sentence` events, one per sentence
8. `reply.done`
9. `state: listening`

When the person interrupts, `interrupted` and a `reply.done` with the reason `interrupted` take the place of the end,
and the next turn's `input.speech_started` follows. Every `reply.started` gets exactly one `reply.done`.

Some replies answer no turn, so no `transcript.final` comes before them: a greeting, the "Go on, I'm listening!" line, a
request to say it again, and the check-in and goodbye of a quiet session. Each is an ordinary reply, from
`reply.started` to `reply.done`.

## Sessions that end the turn for you

A session also ends on its own: at its time limit, or when it stays quiet. A quiet session gets a check-in and then a
goodbye before it ends. See [Quiet sessions](https://docs.voice.kheelona.com/concepts/sessions.md#quiet-sessions) and [Limits](https://docs.voice.kheelona.com/concepts/limits.md).

---

# Safety levels

> The two safety levels, what is checked at each, what the safety event tells you, and what a child hears when a check acts.

Source: https://docs.voice.kheelona.com/concepts/safety-levels/

Every voice has a **safety level**, set by Kheelona: `child` or `general`. You can see it on the **Voices** page of your
console and in `GET /v1/voices`. You cannot change it, and the `instructions` of a session cannot switch it off.

## The levels

| | `child` | `general` |
|---|---|---|
| What the person says | Checked in every category | Checked for the most serious content |
| What the voice is about to say | Every sentence, in every category | Checked for the most serious content |
| Meant for | Voices for children | Voices for older users |

The `child` level is the strictest. It checks each sentence of the voice's reply before it is spoken, not only what the
person says. Use a `child` voice for a child.

A typed turn (`input.text`) goes through the same checks. On a `child` voice, what is said aloud is checked a little more strictly than typed text, because speech recognition can change the words.

## Enforced outside the prompt

Safety does not depend on asking the voice nicely. The checks run **outside** the voice's instructions, as separate steps
on the text of the turns, in all 11 languages. The voice's instructions do not control them, and your `instructions` cannot
remove them. You don't switch them on, and you can't switch them off.

## When the main check is slow

Kheelona's safety rules have more than one part. If the part that judges the meaning of a sentence cannot answer in time,
Kheelona's own safety rules still apply to that turn or sentence. That check is then less thorough, and the turn is
marked as such on Kheelona's side. You get no event for it. Don't assume a turn without a `safety` event was checked in
every way.

## What happens when a check acts

There are two places a check can act, and two things that can happen.

- **On what the person said** (stage `input`): the voice does not answer it. It says a short, friendly line in the
  person's language that changes the subject. The `safety` event has the action `redirected`, and the reply ends with
  `reply.done` and the reason `safety`.
- **On what the voice was about to say** (stage `output`): that sentence is dropped and never spoken. If nothing had
  been said yet, the voice says its friendly line instead and the reply ends. Otherwise the reply ends after the
  sentences already spoken. The `safety` event has the action `sentence_dropped`.
- A third stage, `provider`, is for a block reported by the system that writes the reply. You may never see it.

### A gentle line for a child who needs a grown-up

On a `child` voice, when what the person says is about hurting themselves, sexual content or violence (this covers a
child saying that they are being hurt), the voice does not change the subject. It answers with a calm, caring line that
tells the child to talk to a grown-up they trust, in their language, and says it is still there to play. It is the same
in every `child` voice. The event is the same `safety` event with the category `self_harm`, `sexual` or `violent`.

### Harmless replies are sometimes stopped

In every language, the checks sometimes stop a harmless reply, a few in every hundred in our tests. The voice then
says its short friendly line, or ends the reply early. The checks sometimes mistake a harmless reply for a harmful one,
and stop it.

On a `child` voice, a harmless question asked aloud is occasionally answered with the gentle "talk to a grown-up you
trust" line instead of the friendly one. This is rare, and it happens in every language.

### Beta languages

Of the [beta and alpha languages](https://docs.voice.kheelona.com/concepts/languages.md#supported-beta-and-alpha), Telugu, Kannada and Malayalam have less precise safety checks,
and they stop a few harmless sentences. On a `child` voice, such a sentence may get the gentle "talk to a
grown-up you trust" reply instead of an answer. Otherwise the voice changes the subject, or drops a sentence of its
reply. The checks stay on, even when that stops a harmless sentence. We measure them
again when they improve.

## The `safety` event

```json
{ "type": "safety", "turn": 3, "stage": "input", "category": "violent", "action": "redirected" }
```

| Field | Values |
|---|---|
| `turn` | The turn number. |
| `stage` | `input`, `output` or `provider`. |
| `category` | A short label for the kind of content. It is never the words themselves. |
| `action` | `redirected` or `sentence_dropped`. |

One event is sent per block, with one category. If several apply, the care categories `self_harm`, `sexual` and `violent`
come first. The labels today are:

- `violent`, `illegal`, `sexual`, `pii`, `self_harm`, `unethical`, `political`, `copyright`, `jailbreak`, `unknown`;
- `profanity`, `hate`, `romance`, `scary`, `dangerous_activity`, `substances`, `personal_data`;
- `unsafe`, when nothing more specific applies;
- `unchecked`, when a check could not be completed and the text was stopped to be safe.

The set can grow. **Treat a label you don't know as "other".** Kheelona does not publish the checks themselves.

## What to do with the event

The event is yours to use. For example: log it with the session ID, tell a parent or a moderator, or show a note in the
page. Kheelona does not keep the conversation, so the event is your record. In the browser SDK it is the `safety` event.
On a WebSocket it is the `safety` message. See [Live text events](https://docs.voice.kheelona.com/concepts/live-text.md) and the
[server events reference](https://docs.voice.kheelona.com/reference/generated/events/server.md).

If you decide that something in your product needs a human, such as a child who may be in danger, that is your
responsibility. Plan who reads these events before you go live.

---

# Live text events

> The events that carry what was said, as text, while the conversation runs, and what Kheelona does and does not keep.

Source: https://docs.voice.kheelona.com/concepts/live-text/

While a session runs, Kheelona sends you the text of the conversation, so you can show captions, animate a character or
keep your own records. The text is sent live. **Kheelona does not keep it.** No audio and no conversation text is stored
or logged.

## The events

| Event | What it carries |
|---|---|
| `transcript.final` | What the person said: `turn`, `text`, `language`, `language_stage`, `language_beta`. One per turn, once it is understood. A typed turn gets one too. |
| `reply.sentence` | A sentence of the reply as it starts to play: `reply_id`, `index`, `text`, `emotion`. The text has no emotion tags in it. When the voice speaks several sentences in one go, the event for a later sentence is sent at the moment its audio is estimated to start, to within a fraction of a second. |
| `reply.started` and `reply.done` | The start and the end of a reply. `reply.done` has the `reason`: `completed`, `interrupted`, `safety` or `error`. |
| `state` | `listening`, `thinking` or `speaking`: for a face, a light or a talk button. |
| `input.speech_started` and `input.speech_stopped` | Speech was detected, and the turn ended. |
| `interrupted` | The person talked over the voice. `heard_ms` is how much of the reply they heard. |
| `safety` | A safety rule acted: a stage, a category and an action, never the words. See [Safety levels](https://docs.voice.kheelona.com/concepts/safety-levels.md). |

Every field is in the [server events reference](https://docs.voice.kheelona.com/reference/generated/events/server.md). Ignore event types and fields you
don't know: new ones can be added without a new version.

In the browser SDK the same events have names such as `transcript` and `replySentence`. See the
[browser SDK guide](https://docs.voice.kheelona.com/guides/browser-sdk.md#events).

## What the text is

- `transcript.final` is what the speech recogniser heard. It can be wrong, more so in beta and alpha languages (see [Languages](https://docs.voice.kheelona.com/concepts/languages.md)), so show it as what
  was heard, not as what was said.
- `reply.sentence` is what the voice says. It is the text of the audio that is playing, one sentence at a time.
- There is no partial text. The person's words arrive after they have finished, and a reply arrives one sentence at a time.
- Some replies come with no `transcript.final` before them: a greeting, a short line that invites the person to go on, a
  request to say it again, and the check-in and goodbye of a quiet session. They are ordinary replies. See
  [Turn-taking](https://docs.voice.kheelona.com/concepts/turn-taking.md).

## What happens to the text

- Kheelona sends it to you and forgets it. Not in logs, not in the connection records, not in usage.
- You receive the only copy. What you do with it is your responsibility. See [Captions](https://docs.voice.kheelona.com/guides/captions.md).
- The `instructions` you send are never stored in readable form either.

---

# Latency

> How quickly the voice answers, how it is measured, what changes it, and how to measure it in your own page.

Source: https://docs.voice.kheelona.com/concepts/latency/

## The target

Our target is that the voice **typically answers within 3 seconds** of the person finishing, counted from the end of
their speech to the first sound of the reply. We aim to keep the 95th percentile, nineteen replies in twenty,
under 4.5 seconds. Kheelona measures this on the service and reports it.

This is a **target, not a guarantee.** It is not in any contract, and a single reply can be slower. During the partner
pilot Kheelona is best effort.

## What is in those seconds

From the end of speech to the first audio, the service:

1. decides the person has finished (a short wait to make sure they are not only pausing);
2. turns the speech into text;
3. checks the text for safety;
4. writes the first sentence of the reply;
5. checks that sentence;
6. speaks it.

Everything after the first sentence happens while the first sentence plays, so a long reply does not make you wait
longer.

## What changes it

- **Distance.** Kheelona's servers are in India. A person far from India has a longer network trip each way, and so a
  longer wait. The text of each reply is written in a US region of our cloud provider, which adds a little time, or, for a voice set up
  for India, in Mumbai, where how soon a voice starts depends on how it is set up. See [Data and privacy](https://docs.voice.kheelona.com/policies/data-and-privacy.md#where-it-is-processed).
- **The first sentence.** A reply that begins with a long sentence takes longer to start than one that begins with a
  short one.
- **Load.** When many people are talking at once, replies can be slower. When Kheelona is full, a new session is
  refused with `capacity_exhausted` instead. See [Limits](https://docs.voice.kheelona.com/concepts/limits.md).
- **Your connection.** A slow or unstable network between the person and Kheelona adds to the wait on either transport.

## Measure it yourself

Time the gap between `input.speech_stopped` and the first `reply.sentence` on your page:

- In the browser SDK, listen to the raw `event` stream for both events, and subtract the receive times.
- On a WebSocket, do the same with the events as they arrive.

This is your own measure. It leaves out the service's wait for the end of speech and includes your network each way, so it
is not the same number as the target. Use it to compare, over time and between devices. Typed turns have no speech
and no `input.speech_stopped`.

## Keep it feeling quick

- Show the `thinking` state so the person sees that something is happening.
- Start playing at once: the SDK does.
- Don't add your own waiting before you connect. Create the session just before the person starts, not long before:
  an unused client secret expires after about a minute.

---

# Server-to-server

> Connect a toy relay or a backend to a Kheelona voice over a WebSocket, send microphone audio and play the reply.

Source: https://docs.voice.kheelona.com/guides/server-to-server/

Use this path when your own server talks to Kheelona for a device: a toy relay, a phone-line bridge, a test tool. Your
server holds the API key. The device never sees it.

The relay receives the toy's microphone audio, sends it to Kheelona as it arrives, and plays the reply audio back.
[Run the quickstart](https://docs.voice.kheelona.com/get-started/quickstart-server.md) first to see all of this work in a few minutes. A web page uses the
[browser SDK](https://docs.voice.kheelona.com/guides/browser-sdk.md) instead; its [SDK reference](https://docs.voice.kheelona.com/reference/sdk/classes/kheelonavoice.md) lists every call.

## The flow

1. Your server creates a session with `POST /v1/sessions` and your API key. The answer holds a `session_id`, a one-time
   `client_secret` and the `connect.websocket_url`. See [Sessions](https://docs.voice.kheelona.com/concepts/sessions.md).
2. Your server opens a WebSocket to that URL with the client secret in a header.
3. Kheelona sends `session.ready`, then `state: listening`. From now on you may send audio.
4. You send microphone audio as binary frames. Kheelona sends events as text frames and the voice's audio as binary frames.
5. You send `session.end`, wait for `session.ended`, and close.

## Connect

Open `wss://api.voice.kheelona.com/v1/realtime?session_id=…` with these three things:

- the header `Authorization: Bearer kvcs_…` (the client secret from step 1);
- the subprotocol `kheelona.v1`;
- **no `Origin` header.**

A server client must not send `Origin`. A request that has one is treated as a browser and checked against your allowed
web origins. If your WebSocket library adds one by default, turn it off.

The client secret works once and expires about 60 seconds after it was issued. Connect straight away. If the connection
fails for any reason, create a new session: a failed attempt needs a new secret.

**Node**

```ts title="relay.ts"
/**
 * Opens the WebSocket the way a server does: the client secret in the Authorization header, the `kheelona.v1`
 * subprotocol, and no Origin header (a request with one is treated as a browser). Attach your listeners straight
 * away: the server's first event can arrive in the same moment the connection opens.
 */
function openSocket(session: CreatedSession): WebSocket {
  return new WebSocket(session.websocketUrl, ['kheelona.v1'], {
    headers: { Authorization: `Bearer ${session.clientSecret}` },
    handshakeTimeout: 20_000,
  });
}

/** Before the upgrade, errors come back as HTTP errors with a JSON body: keep the code and the request ID. */
function readRefusal(res: IncomingMessage): Promise<RelayError> {
  return new Promise((resolve) => {
    const chunks: Buffer[] = [];
    res.on('data', (chunk: Buffer) => chunks.push(chunk));
    res.on('end', () => {
      let code = `http_${String(res.statusCode)}`;
      let requestId: string | undefined;
      try {
        const body = JSON.parse(Buffer.concat(chunks).toString('utf8').slice(0, 8192)) as { error?: { code?: string; request_id?: string } };
        if (typeof body.error?.code === 'string' && /^[a-z0-9_]{1,64}$/.test(body.error.code)) code = body.error.code;
        if (typeof body.error?.request_id === 'string' && /^[A-Za-z0-9_]{1,64}$/.test(body.error.request_id)) requestId = body.error.request_id;
      } catch {
        // Not JSON: keep the HTTP status as the code.
      }
      resolve(new RelayError(code, `The connection was refused (${code})`, requestId));
    });
  });
}
```

**Python**

```python title="relay.py"
async with _http(opts) as http:
    session = await create_session(
        http,
        api_url=opts.api_url,
        api_key=opts.api_key,
        voice_id=opts.voice_id,
        languages=opts.languages,
        interruptions=opts.interruptions,
    )
secret = session["client_secret"]["value"]
try:
    # Servers send no Origin header (the websockets library adds none).
    async with connect(
        session["connect"]["websocket_url"],
        subprotocols=["kheelona.v1"],
        additional_headers={"Authorization": f"Bearer {secret}"},
    ) as ws:
        # Send nothing until the server says session.ready: audio sent before it is dropped.
        while not reply.session_id:
            await _receive(ws, reply, opts.reply_timeout)
        if opts.pause_playback:  # tell the voice this client can pause and resume its playback buffer
            await ws.send(json.dumps({"type": "client.capabilities", "playback_pause": True}))
```

Before the upgrade, a refusal is a normal HTTP error with a JSON body, such as
`{"error":{"code":"unauthenticated","message":"…","request_id":"req_…"}}`. Keep the `code` and the `request_id`. The codes
are on [Errors and retries](https://docs.voice.kheelona.com/guides/errors-and-retries.md).

## Wait for session.ready

Send nothing until `session.ready` arrives. Audio sent before it is dropped. The event tells you what the session
runs with: the voice, the languages, the audio formats, the limits and whether `interruptions` is on.

Attach your message handler as soon as the socket opens. The first event can arrive in the same moment.

## Send audio at real time

Send the microphone as binary frames:

- PCM, signed 16-bit little-endian, mono, no header;
- 16 kHz by default (`pcm16_16k`; 24 kHz and 48 kHz are also available);
- 20 ms per frame is a good size (640 bytes at 16 kHz). Anything from 10 ms to 500 ms is accepted.

Pace the frames like a microphone: one 20 ms frame every 20 ms. Time each frame against a steady clock, not with a
fixed sleep, so a late wake-up never makes you drift.

- Audio that arrives faster than twice real time is dropped, and you get one non-fatal `invalid_audio` error.
- Quiet time does not build up an allowance: at most 1 second of audio can run ahead of real time.
- A frame with an odd number of bytes, or outside 10 to 500 ms, is dropped with `invalid_audio`.
- A WebSocket message over 64 KB closes the connection.

**Keep sending while the session is live, silence included.** Kheelona decides when the person has finished speaking
from the audio stream itself. A stream that stops leaves the voice waiting. Send `{"type":"input.mute","muted":true}`
when the microphone is off. Muting pauses the quiet-session check-in and goodbye counts, but the idle timeout keeps running.

**Node**

```ts title="relay.ts"
const FRAME_MS = 20;
const SAMPLES_PER_FRAME = 16 * FRAME_MS; // 16 kHz: 320 samples = 640 bytes
const SILENCE = Buffer.alloc(SAMPLES_PER_FRAME * 2);

/** Splits microphone audio (16 kHz PCM16) into 20 ms frames; the last one is padded with silence. */
export function wavToFrames(samples: Int16Array): Buffer[] {
  const frames: Buffer[] = [];
  for (let at = 0; at < samples.length; at += SAMPLES_PER_FRAME) {
    const frame = Buffer.alloc(SAMPLES_PER_FRAME * 2);
    samples.subarray(at, at + SAMPLES_PER_FRAME).forEach((sample, i) => {
      frame.writeInt16LE(sample, i * 2);
    });
    frames.push(frame);
  }
  return frames;
}

/**
 * Sends one frame every 20 ms, in real time. Frame k goes out at start + k × 20 ms, measured against the clock, so a
 * late wake-up never makes the stream drift (the server drops audio that arrives faster than 2× real time).
 * Stops when `next` returns nothing or `stop` says so.
 */
export async function sendPaced(opts: { next: () => Buffer | undefined; send: (frame: Buffer) => void; stop: () => boolean }): Promise<void> {
  const start = performance.now();
  for (let k = 0; !opts.stop(); k += 1) {
    const frame = opts.next();
    if (!frame) return;
    opts.send(frame);
    await new Promise((resolve) => setTimeout(resolve, Math.max(0, start + (k + 1) * FRAME_MS - performance.now())));
  }
}
```

**Python**

```python title="relay.py"
class FrameSource:
    """What to send next: the speech in 20 ms frames, then silence for as long as you keep asking."""

    def __init__(self) -> None:
        self._pending = bytearray()

    def load(self, pcm: bytes) -> None:
        self._pending += pcm

    def next(self) -> bytes:
        frame = bytes(self._pending[:FRAME_BYTES])
        del self._pending[:FRAME_BYTES]
        return frame.ljust(FRAME_BYTES, b"\0") if frame else SILENCE


async def pump(send, source: FrameSource, stop: asyncio.Event) -> None:
    """Send one frame every 20 ms, in real time, until `stop` is set.

    Frame k is due at start + k * 20 ms. Sleeping until that deadline (never a fixed 20 ms) keeps the pace
    exact: a late wake-up is made up on the next frame instead of piling up. Send silence while nobody
    speaks: the server's turn detection needs a steady stream to know when a person has finished.
    """
    start = time.monotonic()
    k = 0
    while not stop.is_set():
        delay = start + k * FRAME_S - time.monotonic()
        if delay > 0:
            await asyncio.sleep(delay)
        await send(source.next())
        k += 1
        await asyncio.sleep(0)  # give the receiving side a turn
```

You can also send a typed turn as `{"type":"input.text","text":"…"}` (up to 2,000 characters). It goes through the same safety
checks. On a `child` voice, what is said aloud is checked a little more strictly than typed text, because speech
recognition can change the words. A typed turn ends whatever is in progress. The quickstart uses it so you can test
without a microphone.

## Pause support

A toy relay can make the voice stop faster and resume cleaner by pausing its own playback buffer when the person starts
to speak. It is optional.

- **Declare it** once after `session.ready`: `{"type":"client.capabilities","playback_pause":true}`.
- **Pause** your buffer on `reply.paused` and **resume** it on `reply.resumed`. Keep the audio you hold while paused.
- **End the pause** on whichever comes first: `reply.resumed`, `interrupted` (flush the queue), `reply.done`, or the next
  `reply.started`. Ignore a pause event or a `reply.done` that names a reply other than the current one.

**Node**

```ts title="relay.ts"
/**
 * The playback buffer, with pause support. Frames wait in `push` until `drain` hands them to the speaker; while the
 * voice has paused its own playback (`reply.paused`) `drain` returns nothing and the frames are kept, so nothing is lost.
 * A pause ends at the first of: `reply.resumed`, `interrupted` (the buffer is flushed), `reply.done` or the next
 * `reply.started`. A pause, resume or done for any reply other than the one that is playing (or one that has already
 * ended) is stale and ignored.
 */
export class PausePlayback {
  private queue: Buffer[] = [];
  private current: string | undefined;
  private isPaused = false;
  /** Frames thrown away by `interrupted` or by the end of the session. */
  dropped = 0;

  /** `enabled = false` when the relay did not declare `playback_pause`: the server sends no pause events then. */
  private readonly enabled: boolean;
  constructor(enabled = true) {
    this.enabled = enabled;
  }

  get paused(): boolean {
    return this.isPaused;
  }

  onEvent(event: { type: string; reply_id?: unknown }): void {
    switch (event.type) {
      case 'reply.started':
        this.current = typeof event.reply_id === 'string' ? event.reply_id : undefined; // a new reply ends any pause
        this.isPaused = false;
        break;
      case 'reply.paused':
        if (this.enabled && this.current !== undefined && event.reply_id === this.current) this.isPaused = true;
        break;
      case 'reply.resumed':
        if (event.reply_id === this.current) this.isPaused = false;
        break;
      case 'interrupted': // flush: what was queued will never play
        this.dropped += this.queue.length;
        this.queue = [];
        this.isPaused = false;
        this.current = undefined; // that reply is over: a pause naming it is stale
        break;
      case 'reply.done':
        if (event.reply_id === this.current) {
          this.isPaused = false;
          this.current = undefined;
        }
        break;
      case 'session.ended':
        this.dropped += this.queue.length;
        this.queue = [];
        this.isPaused = false;
        break;
      default:
        break;
    }
  }

  push(frame: Buffer): void {
    this.queue.push(frame);
  }

  /** The frames to play now: none while paused. */
  drain(): Buffer[] {
    if (this.isPaused) return [];
    const frames = this.queue;
    this.queue = [];
    return frames;
  }
}
```

**Python**

```python title="relay.py"
class PausePlayback:
    """The playback buffer, with pause support.

    Frames wait in `push` until `drain` hands them to the speaker. While the voice has paused its own playback
    (`reply.paused`) `drain` returns nothing and the frames are kept, so nothing is lost. A pause ends at the first of
    `reply.resumed`, `interrupted` (the buffer is flushed), `reply.done` or the next `reply.started`. A pause, resume
    or done for any reply other than the one that is playing (or one that has already ended) is stale and ignored.
    """

    def __init__(self, enabled: bool = True) -> None:
        self.enabled = enabled  # False when the relay did not declare playback_pause
        self.paused = False
        self.dropped = 0  # frames thrown away by `interrupted` or by the end of the session
        self._queue: list[bytes] = []
        self._current: str | None = None

    def on_event(self, event: dict) -> None:
        kind = event.get("type")
        if kind == "reply.started":
            self._current = event.get("reply_id")  # a new reply ends any pause
            self.paused = False
        elif kind == "reply.paused":
            if self.enabled and self._current is not None and event.get("reply_id") == self._current:
                self.paused = True
        elif kind == "reply.resumed":
            if event.get("reply_id") == self._current:
                self.paused = False
        elif kind in ("interrupted", "session.ended"):  # flush: what was queued will never play
            self.dropped += len(self._queue)
            self._queue = []
            self.paused = False
            self._current = None  # that reply is over: a pause naming it is stale
        elif kind == "reply.done":
            if event.get("reply_id") == self._current:
                self.paused = False
                self._current = None

    def push(self, frame: bytes) -> None:
        self._queue.append(frame)

    def drain(self) -> list[bytes]:
        """The frames to play now: none while paused."""
        if self.paused:
            return []
        frames, self._queue = self._queue, []
        return frames
```

Without the declaration you never get these events. See [Interruptions and echo](https://docs.voice.kheelona.com/guides/interruptions-and-echo.md) for
the details.

## Events

Every event is one JSON object in a text frame. It has a `type`. Server events also have an `event_id` and a `ts`.
**Ignore event types and fields you don't know.** New ones can be added without a new version.

The ones a relay needs:

| Event | What to do |
|---|---|
| `session.ready` | Start sending audio |
| `state` | `listening`, `thinking` or `speaking`: drive a light or a face on the toy |
| `transcript.final` | What the person said, as text (live only; Kheelona does not keep it) |
| `reply.sentence` | The text of the sentence that is playing, as a caption |
| `reply.done` | The reply is finished; `reason` says why |
| `interrupted` | The person talked over the voice: flush queued audio now |
| `reply.paused`, `reply.resumed` | Only if you declared `playback_pause`: pause and resume your playback buffer |
| `safety` | A safety rule acted: see below |
| `session.warning` | The session is about to end: 60 s before the maximum length, 20 s before the idle timeout |
| `error` | A problem; if `fatal` is true the session ends next |
| `session.ended` | The last event; close the socket |

Every field of every event is in the [server events reference](https://docs.voice.kheelona.com/reference/generated/events/server.md) and the
[client events reference](https://docs.voice.kheelona.com/reference/generated/events/client.md).

## Play the reply

The reply audio comes as binary frames, each with an 8-byte header and then PCM samples. The samples are 16-bit
little-endian mono at 24 kHz by default (`pcm16_24k`).

| Byte | Size | Field | Meaning |
|---|---|---|---|
| 0 | 1 | `kind` | `0x01` is reply audio. Ignore other values. |
| 1 | 1 | `flags` | bit 0: first frame of a sentence. bit 1: last frame of the reply. |
| 2 | 2 | `sentence` | The sentence number within the reply, little-endian. It matches `reply.sentence.index`. |
| 4 | 4 | `reply_seq` | A number that goes up by one for each reply in the session, little-endian. |
| 8 | … | samples | PCM16 audio |

- Frames are 40 ms long. The last frame of a sentence can be shorter. The last frame of a reply can be only the header,
  with no samples.
- Kheelona sends audio in real time, at most 300 ms ahead of playback. Play frames as they arrive, with a jitter
  buffer of about 100 ms.
- More about the formats: [Audio formats](https://docs.voice.kheelona.com/reference/generated/audio.md).

## When the person talks over the voice

Kheelona sends an `interrupted` event. Do two things:

1. Flush the audio you have queued for playback.
2. Drop every later frame whose `reply_seq` is the interrupted reply's or lower. Frames of that reply can still be on
   the wire.

Optionally, send `{"type":"playback.position","reply_id":"rep_…","played_ms":1840}` within 500 ms. It tells Kheelona how
much the person actually heard, so the conversation continues from the right place.

**Node**

```ts title="relay.ts"
/** Collects the reply audio, dropping what the server told us to drop. */
export class ReplyAudio {
  private readonly chunks: Buffer[] = [];
  private readonly playback: PausePlayback;
  private newestSeq = -1;
  private droppedThrough = -1;

  constructor(playback = new PausePlayback()) {
    this.playback = playback;
  }

  /** A binary frame: an 8-byte header (kind, flags, sentence, reply_seq) then PCM16 24 kHz samples. */
  push(frame: Buffer): void {
    if (frame.length < 8 || frame.readUInt8(0) !== 0x01) return; // not reply audio: ignore unknown kinds
    const seq = frame.readUInt32LE(4);
    this.newestSeq = Math.max(this.newestSeq, seq);
    if (seq <= this.droppedThrough) return; // a late frame of an interrupted reply
    this.playback.push(frame.subarray(8));
    this.play();
  }

  /** Every event goes here first, so the playback buffer pauses, resumes and flushes with the voice. */
  onEvent(event: { type: string; reply_id?: unknown }): void {
    this.playback.onEvent(event);
    this.play(); // a resume, a done or a new reply lets the held frames out
  }

  /** `interrupted`: drop every later frame of the reply being played. (The playback buffer is flushed by `onEvent`.) */
  interrupted(): void {
    this.droppedThrough = this.newestSeq;
  }

  pcm(): Buffer {
    this.play();
    return Buffer.concat(this.chunks);
  }

  // A real player would hand these frames to the speaker; this example saves them.
  private play(): void {
    this.chunks.push(...this.playback.drain());
  }
}

/** The lines printed for an event: its type, plus the text of what was heard or said. */
export function describeEvent(event: { type: string; text?: unknown }): string[] {
  const lines = [`event ${event.type}`];
  if ((event.type === 'transcript.final' || event.type === 'reply.sentence') && typeof event.text === 'string') lines.push(`  text: ${event.text}`);
  return lines;
}
```

**Python**

```python title="relay.py"
def on_binary(self, data: bytes) -> None:
    """Reply audio: an 8-byte header (kind, flags, sentence, reply_seq) then PCM16 samples."""
    if len(data) < 8:
        return
    kind, _flags, _sentence, seq = struct.unpack_from("<BBHI", data)
    if kind != 1:  # 1 = reply audio
        return
    self.max_seq = max(self.max_seq, seq)
    if seq <= self.drop_upto:
        return  # a late frame of a reply that was interrupted
    self.playback.push(data[8:])
    self._play()

def on_event(self, event: dict) -> None:
    kind = event.get("type")
    self.out(f"event {kind}")
    self.playback.on_event(event)
    self._play()  # a resume, a done or a new reply lets the held frames out
    if kind == "session.ready":
        self.session_id = event.get("session_id", self.session_id)
        self.output_rate = OUTPUT_RATES.get(event.get("audio", {}).get("output_format"), 24000)
    elif kind in ("transcript.final", "reply.sentence"):
        self.out(f"  text: {event.get('text', '')}")
    elif kind == "interrupted":
        self.drop_upto = self.max_seq  # flush what is queued, drop the rest of this reply
    elif kind == "reply.done":
        self.done += 1
        if event.get("reason") != "interrupted":  # an interrupted reply is followed by a new one
            self.answered += 1
    elif kind == "error" and event.get("fatal"):
        raise RelayError(str(event.get("code", "error")))
    elif kind == "session.ended":
        self.ended = str(event.get("reason", "unknown"))
```

### Echo cancellation is required

Your device must cancel echo: a toy whose speaker is near its microphone must not send the voice's own sound back as
if the child were talking. Kheelona checks for its own echo too, but only as a safety net. `interruptions: false`
exists for content that must play through without being cut off; it is not a substitute for echo cancellation.

## Keep the connection alive

- Kheelona sends a WebSocket ping every 15 seconds and closes the connection after 45 seconds with no pong. Every
  WebSocket library answers pings by itself.
- `{"type":"ping"}` gets `{"type":"pong"}` back, if you want a heartbeat of your own.
- **Keep reading.** A client that stops reading what Kheelona sends is disconnected, and the session ends as
  `disconnected`. That means about 20 seconds of unread output, or one write blocked for 10 seconds.

## Safety

Every turn is checked, on what the person says and on what the voice is about to say. You don't switch this on.

When a rule acts, you receive a `safety` event with a `category` and the `action` taken. It never contains the words
themselves. For a child voice, when the input is about self-harm, sexual content or violence, the voice answers with a
gentle line that tells the child to talk to a grown-up they trust, in the child's language. The reply ends with
`reply.done` and `reason: safety`.

Use the `safety` event for your own safeguarding process: log it with your session ID, alert the right person, or show a
note to a parent. Kheelona does not keep the conversation, so the event is your record.

Categories are short snake_case labels such as `self_harm`, `sexual` and `violent`. The list is stable, but it can
grow: treat a label you don't know as "other". The full list is in the
[server events reference](https://docs.voice.kheelona.com/reference/generated/events/server.md).

## Ending

- To end the session, send `{"type":"session.end"}`. Kheelona answers `session.ended` and closes the socket.
- Your server can also end it with `POST /v1/sessions/{id}/end`. The connection closes within about 10 seconds
  with the reason `partner_ended`.
- A session also ends by itself after its maximum length (1,800 seconds unless Kheelona set another) or after the idle
  timeout (120 seconds with nobody speaking and nothing playing). You get a `session.warning` first.
- `session.ended` carries a `reason`: `client_ended`, `partner_ended`, `max_duration`, `idle_timeout`, `disconnected`,
  `server_shutdown`, `provider_failure`, `suspended`, `service_stopped` (the voice service was switched off; start a new
  session once it is back) or `error`. It also has `duration_ms` and `turns`.

Every `reply.started` gets exactly one `reply.done`, even when the session ends in the middle of a reply. It arrives before
`session.ended`.

## Reconnecting means a new session

A session is one conversation on one connection. If the connection drops, the session ends as `disconnected`. To carry
on, create a new session and connect again. The new voice starts fresh: it does not remember the earlier conversation.
Reconnect politely: wait for `Retry-After` when the API sends it, and don't loop faster than that.

One refusal at connect is not a reason for a new session: if the WebSocket upgrade gets `503` `capacity_exhausted` with
`Retry-After: 1`, the server you reached is busy and the client secret is still unused. Wait a second and connect again
with the same secret, up to 3 times, while it is under 60 seconds old. With a longer `Retry-After`, create a new session
after the wait.

## Errors

During a session, a problem arrives as an `error` event with `code`, `message`, `retryable` and `fatal`. A non-fatal
error (a bad frame, a bad event) leaves the session running. After a fatal error the session ends. An `error` event
carries no request ID: when you ask for help, quote the session ID and the `code` (the example relays print both on
standard error), and the time. Every code, with what to do about it, is on
[Errors and retries](https://docs.voice.kheelona.com/guides/errors-and-retries.md).

## The whole relay

The snippets above come from the example relays, which are tested against a running service. Read the whole file when
you build your own.

**Node: examples/node-server/src/relay.ts**

```ts title="relay.ts"
// A server-to-server relay: creates a session, streams a WAV (and/or a typed turn) to the voice over the WebSocket at
// real time, prints the events, and saves the reply audio to a WAV file. Run it with `pnpm relay --text "Hello"`.
import { readFile, writeFile } from 'node:fs/promises';
import type { IncomingMessage } from 'node:http';
import { parseArgs } from 'node:util';
import { pathToFileURL } from 'node:url';
import { WebSocket, type RawData } from 'ws';
import { createSession, KheelonaApiError, readEnv, type CreatedSession } from './kheelona.ts';
import { readPcm16Wav, writeWav } from './wav.ts';

export interface RelayOptions {
  apiUrl: string;
  apiKey: string;
  voiceId: string;
  languages?: string[];
  /** A typed turn, sent first. */
  text?: string;
  /** A 16 kHz mono PCM16 WAV, sent in real time after the typed turn's reply. */
  wavFile?: string;
  /** Where the reply audio (24 kHz mono WAV) is saved. */
  outFile: string;
  /** `false` = the voice is never cut off. Default: the API's default (on). */
  interruptions?: boolean;
  /** Declare `playback_pause` and pause the playback buffer on `reply.paused`. Default: on. `false` = the voice never pauses. */
  pausePlayback?: boolean;
  /** Where event lines go (default: standard output). */
  log?: (line: string) => void;
  /** Milliseconds to wait for a reply before giving up (default 20,000). */
  replyTimeoutMs?: number;
}

export interface RelaySummary {
  sessionId: string;
  endReason: string;
  replies: number;
  audioSeconds: number;
  outFile: string;
}

/** A bad command line or input file: the CLI exits 2. */
export class UsageError extends Error {}

/**
 * A failed session: the code, the request ID when an HTTP answer carried one, and the session ID once the session
 * existed (realtime `error` events carry no request ID: quote the session ID and the code). The CLI exits 1.
 */
export class RelayError extends Error {
  readonly code: string;
  readonly requestId: string | undefined;
  sessionId: string | undefined;
  constructor(code: string, message: string, requestId?: string) {
    super(message);
    this.name = 'RelayError';
    this.code = code;
    this.requestId = requestId;
  }
}

/**
 * Opens the WebSocket the way a server does: the client secret in the Authorization header, the `kheelona.v1`
 * subprotocol, and no Origin header (a request with one is treated as a browser). Attach your listeners straight
 * away: the server's first event can arrive in the same moment the connection opens.
 */
function openSocket(session: CreatedSession): WebSocket {
  return new WebSocket(session.websocketUrl, ['kheelona.v1'], {
    headers: { Authorization: `Bearer ${session.clientSecret}` },
    handshakeTimeout: 20_000,
  });
}

/** Before the upgrade, errors come back as HTTP errors with a JSON body: keep the code and the request ID. */
function readRefusal(res: IncomingMessage): Promise<RelayError> {
  return new Promise((resolve) => {
    const chunks: Buffer[] = [];
    res.on('data', (chunk: Buffer) => chunks.push(chunk));
    res.on('end', () => {
      let code = `http_${String(res.statusCode)}`;
      let requestId: string | undefined;
      try {
        const body = JSON.parse(Buffer.concat(chunks).toString('utf8').slice(0, 8192)) as { error?: { code?: string; request_id?: string } };
        if (typeof body.error?.code === 'string' && /^[a-z0-9_]{1,64}$/.test(body.error.code)) code = body.error.code;
        if (typeof body.error?.request_id === 'string' && /^[A-Za-z0-9_]{1,64}$/.test(body.error.request_id)) requestId = body.error.request_id;
      } catch {
        // Not JSON: keep the HTTP status as the code.
      }
      resolve(new RelayError(code, `The connection was refused (${code})`, requestId));
    });
  });
}

const FRAME_MS = 20;
const SAMPLES_PER_FRAME = 16 * FRAME_MS; // 16 kHz: 320 samples = 640 bytes
const SILENCE = Buffer.alloc(SAMPLES_PER_FRAME * 2);

/** Splits microphone audio (16 kHz PCM16) into 20 ms frames; the last one is padded with silence. */
export function wavToFrames(samples: Int16Array): Buffer[] {
  const frames: Buffer[] = [];
  for (let at = 0; at < samples.length; at += SAMPLES_PER_FRAME) {
    const frame = Buffer.alloc(SAMPLES_PER_FRAME * 2);
    samples.subarray(at, at + SAMPLES_PER_FRAME).forEach((sample, i) => {
      frame.writeInt16LE(sample, i * 2);
    });
    frames.push(frame);
  }
  return frames;
}

/**
 * Sends one frame every 20 ms, in real time. Frame k goes out at start + k × 20 ms, measured against the clock, so a
 * late wake-up never makes the stream drift (the server drops audio that arrives faster than 2× real time).
 * Stops when `next` returns nothing or `stop` says so.
 */
export async function sendPaced(opts: { next: () => Buffer | undefined; send: (frame: Buffer) => void; stop: () => boolean }): Promise<void> {
  const start = performance.now();
  for (let k = 0; !opts.stop(); k += 1) {
    const frame = opts.next();
    if (!frame) return;
    opts.send(frame);
    await new Promise((resolve) => setTimeout(resolve, Math.max(0, start + (k + 1) * FRAME_MS - performance.now())));
  }
}

/**
 * The playback buffer, with pause support. Frames wait in `push` until `drain` hands them to the speaker; while the
 * voice has paused its own playback (`reply.paused`) `drain` returns nothing and the frames are kept, so nothing is lost.
 * A pause ends at the first of: `reply.resumed`, `interrupted` (the buffer is flushed), `reply.done` or the next
 * `reply.started`. A pause, resume or done for any reply other than the one that is playing (or one that has already
 * ended) is stale and ignored.
 */
export class PausePlayback {
  private queue: Buffer[] = [];
  private current: string | undefined;
  private isPaused = false;
  /** Frames thrown away by `interrupted` or by the end of the session. */
  dropped = 0;

  /** `enabled = false` when the relay did not declare `playback_pause`: the server sends no pause events then. */
  private readonly enabled: boolean;
  constructor(enabled = true) {
    this.enabled = enabled;
  }

  get paused(): boolean {
    return this.isPaused;
  }

  onEvent(event: { type: string; reply_id?: unknown }): void {
    switch (event.type) {
      case 'reply.started':
        this.current = typeof event.reply_id === 'string' ? event.reply_id : undefined; // a new reply ends any pause
        this.isPaused = false;
        break;
      case 'reply.paused':
        if (this.enabled && this.current !== undefined && event.reply_id === this.current) this.isPaused = true;
        break;
      case 'reply.resumed':
        if (event.reply_id === this.current) this.isPaused = false;
        break;
      case 'interrupted': // flush: what was queued will never play
        this.dropped += this.queue.length;
        this.queue = [];
        this.isPaused = false;
        this.current = undefined; // that reply is over: a pause naming it is stale
        break;
      case 'reply.done':
        if (event.reply_id === this.current) {
          this.isPaused = false;
          this.current = undefined;
        }
        break;
      case 'session.ended':
        this.dropped += this.queue.length;
        this.queue = [];
        this.isPaused = false;
        break;
      default:
        break;
    }
  }

  push(frame: Buffer): void {
    this.queue.push(frame);
  }

  /** The frames to play now: none while paused. */
  drain(): Buffer[] {
    if (this.isPaused) return [];
    const frames = this.queue;
    this.queue = [];
    return frames;
  }
}

/** Collects the reply audio, dropping what the server told us to drop. */
export class ReplyAudio {
  private readonly chunks: Buffer[] = [];
  private readonly playback: PausePlayback;
  private newestSeq = -1;
  private droppedThrough = -1;

  constructor(playback = new PausePlayback()) {
    this.playback = playback;
  }

  /** A binary frame: an 8-byte header (kind, flags, sentence, reply_seq) then PCM16 24 kHz samples. */
  push(frame: Buffer): void {
    if (frame.length < 8 || frame.readUInt8(0) !== 0x01) return; // not reply audio: ignore unknown kinds
    const seq = frame.readUInt32LE(4);
    this.newestSeq = Math.max(this.newestSeq, seq);
    if (seq <= this.droppedThrough) return; // a late frame of an interrupted reply
    this.playback.push(frame.subarray(8));
    this.play();
  }

  /** Every event goes here first, so the playback buffer pauses, resumes and flushes with the voice. */
  onEvent(event: { type: string; reply_id?: unknown }): void {
    this.playback.onEvent(event);
    this.play(); // a resume, a done or a new reply lets the held frames out
  }

  /** `interrupted`: drop every later frame of the reply being played. (The playback buffer is flushed by `onEvent`.) */
  interrupted(): void {
    this.droppedThrough = this.newestSeq;
  }

  pcm(): Buffer {
    this.play();
    return Buffer.concat(this.chunks);
  }

  // A real player would hand these frames to the speaker; this example saves them.
  private play(): void {
    this.chunks.push(...this.playback.drain());
  }
}

/** The lines printed for an event: its type, plus the text of what was heard or said. */
export function describeEvent(event: { type: string; text?: unknown }): string[] {
  const lines = [`event ${event.type}`];
  if ((event.type === 'transcript.final' || event.type === 'reply.sentence') && typeof event.text === 'string') lines.push(`  text: ${event.text}`);
  return lines;
}

type ServerEvent = { type: string; [key: string]: unknown };

/** Runs one session: see the file header. Rejects with `KheelonaApiError`, `RelayError` or `UsageError`. */
export async function runRelay(opts: RelayOptions): Promise<RelaySummary> {
  const log =
    opts.log ??
    ((line: string) => {
      console.log(line);
    });
  const replyTimeoutMs = opts.replyTimeoutMs ?? 20_000;
  if (!opts.text && !opts.wavFile) throw new UsageError('Give --text and/or --wav.');
  if (opts.text && Array.from(opts.text).length > 2000) throw new UsageError('--text is at most 2000 characters');
  let wavFrames: Buffer[] | undefined;
  if (opts.wavFile) {
    try {
      wavFrames = wavToFrames(readPcm16Wav(await readFile(opts.wavFile)).samples);
    } catch (error) {
      throw new UsageError(error instanceof Error && 'code' in error ? `Cannot read ${opts.wavFile}.` : (error as Error).message);
    }
  }

  const session = await createSession({
    apiUrl: opts.apiUrl,
    apiKey: opts.apiKey,
    voiceId: opts.voiceId,
    ...(opts.languages ? { languages: opts.languages } : {}),
    ...(opts.interruptions === undefined ? {} : { interruptions: opts.interruptions }),
  });
  const ws = openSocket(session);
  const pause = opts.pausePlayback ?? true;

  const audio = new ReplyAudio(new PausePlayback(pause));
  const queue: Buffer[] = []; // microphone frames waiting to go out; silence fills the gaps
  let wavQueued = false;
  let turnsSent = 0;
  let turnsAnswered = 0;
  let replies = 0;
  let stopPacer = false;
  let pacer: Promise<void> | undefined;
  let timer: NodeJS.Timeout | undefined;
  let finished = false;

  const summary = await new Promise<RelaySummary>((resolve, reject) => {
    const finish = (error?: Error, endReason = 'unknown'): void => {
      if (finished) return;
      finished = true;
      stopPacer = true;
      clearTimeout(timer);
      if (error instanceof RelayError) error.sessionId ??= session.sessionId;
      if (error) reject(error);
      else resolve({ sessionId: session.sessionId, endReason, replies, audioSeconds: audio.pcm().length / 2 / 24_000, outFile: opts.outFile });
    };
    // Nothing from the server for a while (waiting for session.ready, a reply or session.ended): give up, so a copy
    // of this never hangs. Every message from the server restarts the wait.
    const armTimer = (): void => {
      clearTimeout(timer);
      timer = setTimeout(() => {
        finish(new RelayError('no_reply', `Nothing from the server for ${String(replyTimeoutMs / 1000)} s`));
      }, replyTimeoutMs);
    };
    const sendEvent = (event: Record<string, unknown>): void => {
      ws.send(JSON.stringify(event));
    };
    const queueWav = (): void => {
      if (!wavFrames || wavQueued) return;
      wavQueued = true;
      turnsSent += 1;
      queue.push(...wavFrames);
    };
    // Microphone audio at real time for the whole session: your turn's frames, then silence.
    const startPacer = (): void => {
      pacer = sendPaced({
        next: () => {
          const frame = queue.shift();
          if (frame) return frame;
          return SILENCE;
        },
        send: (frame) => {
          if (ws.readyState === WebSocket.OPEN) ws.send(frame);
        },
        stop: () => stopPacer,
      });
    };

    ws.once('open', armTimer);
    ws.on('message', (data: RawData, isBinary: boolean) => {
      armTimer();
      if (isBinary) {
        if (Buffer.isBuffer(data)) audio.push(data);
        return;
      }
      let event: ServerEvent;
      try {
        event = JSON.parse((data as Buffer).toString('utf8')) as ServerEvent;
      } catch {
        return;
      }
      if (typeof event.type !== 'string') return;
      for (const line of describeEvent(event)) log(line);
      audio.onEvent(event);

      switch (event.type) {
        case 'session.ready':
          // Nothing is sent before this: audio sent earlier is dropped.
          if (pause) sendEvent({ type: 'client.capabilities', playback_pause: true });
          if (opts.text) {
            sendEvent({ type: 'input.text', text: opts.text });
            turnsSent += 1;
          } else {
            queueWav();
          }
          startPacer();
          break;
        case 'interrupted':
          audio.interrupted();
          break;
        case 'reply.done':
          replies += 1;
          if (event['reason'] === 'interrupted') break; // the turn goes on with a new reply
          turnsAnswered += 1;
          if (turnsAnswered < turnsSent) break;
          if (!wavQueued && wavFrames) {
            queueWav(); // the typed reply is done: now the WAV
          } else {
            stopPacer = true;
            void (pacer ?? Promise.resolve()).then(() => {
              sendEvent({ type: 'session.end' });
            });
          }
          break;
        case 'error':
          if (event['fatal'] === true) {
            finish(
              new RelayError(typeof event['code'] === 'string' ? event['code'] : 'error', 'The session failed'),
            );
          }
          break;
        case 'session.ended':
          finish(undefined, typeof event['reason'] === 'string' ? event['reason'] : 'unknown');
          break;
        default:
          break;
      }
    });
    ws.on('unexpected-response', (_req, res) => {
      void readRefusal(res).then((error) => {
        finish(error);
      });
    });
    ws.on('close', () => {
      finish(new RelayError('disconnected', 'The connection closed before the session ended'));
    });
    ws.on('error', () => {
      finish(new RelayError('connection_failed', 'The WebSocket failed'));
    });
  }).finally(() => {
    ws.terminate();
  });

  await writeFile(opts.outFile, writeWav(audio.pcm(), 24_000));
  return summary;
}

/** The command line. Returns the exit code: 0 ended normally, 1 API or session error, 2 usage error. */
export async function runCli(
  argv: string[],
  env: Record<string, string | undefined>,
  out: (line: string) => void,
  err: (line: string) => void,
  overrides: { replyTimeoutMs?: number } = {},
): Promise<number> {
  try {
    const { values } = parseArgs({
      args: argv,
      options: {
        text: { type: 'string' },
        wav: { type: 'string' },
        out: { type: 'string', default: 'reply.wav' },
        voice: { type: 'string' },
        languages: { type: 'string' },
        'no-interruptions': { type: 'boolean', default: false },
        'no-pause': { type: 'boolean', default: false },
      },
      strict: true,
    });
    const settings = readEnv(env);
    const summary = await runRelay({
      apiUrl: settings.apiUrl,
      apiKey: settings.apiKey,
      voiceId: values.voice ?? settings.voiceId,
      ...(values.languages ? { languages: values.languages.split(',').map((l) => l.trim()) } : {}),
      ...(values.text ? { text: values.text } : {}),
      ...(values.wav ? { wavFile: values.wav } : {}),
      outFile: values.out,
      ...(values['no-interruptions'] ? { interruptions: false } : {}),
      ...(values['no-pause'] ? { pausePlayback: false } : {}),
      log: out,
      ...(overrides.replyTimeoutMs === undefined ? {} : { replyTimeoutMs: overrides.replyTimeoutMs }),
    });
    out(`session ${summary.sessionId} ended ${summary.endReason} replies ${String(summary.replies)} audio ${summary.audioSeconds.toFixed(1)}s saved ${summary.outFile}`);
    return 0;
  } catch (error) {
    if (error instanceof KheelonaApiError) {
      // No answer at all (status 0): name the failure, like the Python example.
      const code = error.status === 0 ? `api_unreachable_${error.causeName ?? 'Error'}` : error.code;
      err(`error ${code} request_id ${error.requestId ?? '-'}`);
      return 1;
    }
    if (error instanceof RelayError) {
      const session = error.sessionId ? ` session ${error.sessionId}` : '';
      err(`error ${error.code}${session} request_id ${error.requestId ?? '-'}`);
      return 1;
    }
    // A usage mistake (bad flag, missing key, unreadable WAV): one line, exit 2.
    err(error instanceof Error ? error.message : 'Invalid usage.');
    return 2;
  }
}

if (process.argv[1] && pathToFileURL(process.argv[1]).href === import.meta.url) {
  process.exitCode = await runCli(
    process.argv.slice(2),
    process.env,
    (line) => {
      console.log(line);
    },
    (line) => {
      console.error(line);
    },
  );
}
```

**Python: examples/python-server/relay.py**

```python title="relay.py"
"""A server-to-server relay: send a typed turn and/or a WAV file to a Kheelona voice, print the events, save the reply.

    KHEELONA_API_KEY=... uv run python relay.py --text "Hello!" --wav question.wav --out reply.wav

Event text (what the voice heard and said) goes only to this terminal. Kheelona stores none of it, and this
example writes nothing but the reply audio file you asked for.
"""

import argparse
import asyncio
import json
import os
import struct
import sys
import time
from collections.abc import Callable, Mapping, Sequence
from dataclasses import dataclass

import httpx
from websockets.asyncio.client import connect
from websockets.exceptions import InvalidStatus, WebSocketException

from kheelona import KheelonaApiError, create_session
from wavio import read_pcm16_wav, write_wav

FRAME_S = 0.020  # one frame every 20 ms
FRAME_BYTES = 640  # 20 ms of 16 kHz, 16-bit mono audio
SILENCE = bytes(FRAME_BYTES)
OUTPUT_RATES = {"pcm16_16k": 16000, "pcm16_24k": 24000, "pcm16_48k": 48000}


class RelayError(Exception):
    """The session failed. `code`, `session_id` and `request_id` are safe to print.

    A realtime `error` event carries no request ID: quote the session ID and the code. An HTTP refusal carries a
    request ID.
    """

    def __init__(self, code: str, request_id: str | None = None, session_id: str | None = None):
        super().__init__(f"{code} (session {session_id or '-'}, request_id {request_id or '-'})")
        self.code = code
        self.request_id = request_id
        self.session_id = session_id


@dataclass
class RelaySummary:
    session_id: str
    end_reason: str
    replies: int
    audio_seconds: float
    out_file: str


@dataclass
class RelayOptions:
    api_url: str
    api_key: str
    voice_id: str = "lumi-dragon"
    languages: Sequence[str] = ("en", "hi")
    text: str | None = None
    wav_path: str | None = None
    out_path: str = "reply.wav"
    interruptions: bool = True
    pause_playback: bool = True  # declare playback_pause and pause the playback buffer on reply.paused
    out: Callable[[str], None] = print
    reply_timeout: float = 20.0  # seconds without anything from the server before giving up
    transport: httpx.AsyncBaseTransport | None = None  # a test hook


class FrameSource:
    """What to send next: the speech in 20 ms frames, then silence for as long as you keep asking."""

    def __init__(self) -> None:
        self._pending = bytearray()

    def load(self, pcm: bytes) -> None:
        self._pending += pcm

    def next(self) -> bytes:
        frame = bytes(self._pending[:FRAME_BYTES])
        del self._pending[:FRAME_BYTES]
        return frame.ljust(FRAME_BYTES, b"\0") if frame else SILENCE


async def pump(send, source: FrameSource, stop: asyncio.Event) -> None:
    """Send one frame every 20 ms, in real time, until `stop` is set.

    Frame k is due at start + k * 20 ms. Sleeping until that deadline (never a fixed 20 ms) keeps the pace
    exact: a late wake-up is made up on the next frame instead of piling up. Send silence while nobody
    speaks: the server's turn detection needs a steady stream to know when a person has finished.
    """
    start = time.monotonic()
    k = 0
    while not stop.is_set():
        delay = start + k * FRAME_S - time.monotonic()
        if delay > 0:
            await asyncio.sleep(delay)
        await send(source.next())
        k += 1
        await asyncio.sleep(0)  # give the receiving side a turn




class PausePlayback:
    """The playback buffer, with pause support.

    Frames wait in `push` until `drain` hands them to the speaker. While the voice has paused its own playback
    (`reply.paused`) `drain` returns nothing and the frames are kept, so nothing is lost. A pause ends at the first of
    `reply.resumed`, `interrupted` (the buffer is flushed), `reply.done` or the next `reply.started`. A pause, resume
    or done for any reply other than the one that is playing (or one that has already ended) is stale and ignored.
    """

    def __init__(self, enabled: bool = True) -> None:
        self.enabled = enabled  # False when the relay did not declare playback_pause
        self.paused = False
        self.dropped = 0  # frames thrown away by `interrupted` or by the end of the session
        self._queue: list[bytes] = []
        self._current: str | None = None

    def on_event(self, event: dict) -> None:
        kind = event.get("type")
        if kind == "reply.started":
            self._current = event.get("reply_id")  # a new reply ends any pause
            self.paused = False
        elif kind == "reply.paused":
            if self.enabled and self._current is not None and event.get("reply_id") == self._current:
                self.paused = True
        elif kind == "reply.resumed":
            if event.get("reply_id") == self._current:
                self.paused = False
        elif kind in ("interrupted", "session.ended"):  # flush: what was queued will never play
            self.dropped += len(self._queue)
            self._queue = []
            self.paused = False
            self._current = None  # that reply is over: a pause naming it is stale
        elif kind == "reply.done":
            if event.get("reply_id") == self._current:
                self.paused = False
                self._current = None

    def push(self, frame: bytes) -> None:
        self._queue.append(frame)

    def drain(self) -> list[bytes]:
        """The frames to play now: none while paused."""
        if self.paused:
            return []
        frames, self._queue = self._queue, []
        return frames




class Reply:
    """Collects the reply audio and counts finished replies."""

    def __init__(self, out: Callable[[str], None], pause: bool = True) -> None:
        self.out = out
        self.playback = PausePlayback(enabled=pause)
        self.pcm = bytearray()
        self.done = 0  # every reply.done, interrupted ones too
        self.answered = 0  # replies that finished a turn
        self.max_seq = 0  # highest reply_seq seen
        self.drop_upto = 0  # frames at or below this reply_seq belong to an interrupted reply
        self.output_rate = 24000
        self.session_id = ""
        self.ended: str | None = None

    def on_binary(self, data: bytes) -> None:
        """Reply audio: an 8-byte header (kind, flags, sentence, reply_seq) then PCM16 samples."""
        if len(data) < 8:
            return
        kind, _flags, _sentence, seq = struct.unpack_from("<BBHI", data)
        if kind != 1:  # 1 = reply audio
            return
        self.max_seq = max(self.max_seq, seq)
        if seq <= self.drop_upto:
            return  # a late frame of a reply that was interrupted
        self.playback.push(data[8:])
        self._play()

    def on_event(self, event: dict) -> None:
        kind = event.get("type")
        self.out(f"event {kind}")
        self.playback.on_event(event)
        self._play()  # a resume, a done or a new reply lets the held frames out
        if kind == "session.ready":
            self.session_id = event.get("session_id", self.session_id)
            self.output_rate = OUTPUT_RATES.get(event.get("audio", {}).get("output_format"), 24000)
        elif kind in ("transcript.final", "reply.sentence"):
            self.out(f"  text: {event.get('text', '')}")
        elif kind == "interrupted":
            self.drop_upto = self.max_seq  # flush what is queued, drop the rest of this reply
        elif kind == "reply.done":
            self.done += 1
            if event.get("reason") != "interrupted":  # an interrupted reply is followed by a new one
                self.answered += 1
        elif kind == "error" and event.get("fatal"):
            raise RelayError(str(event.get("code", "error")))
        elif kind == "session.ended":
            self.ended = str(event.get("reason", "unknown"))


    def _play(self) -> None:
        # A real player would hand these frames to the speaker; this example saves them.
        for frame in self.playback.drain():
            self.pcm += frame


async def _receive(ws, reply: Reply, timeout: float) -> None:
    """Handle one message from the server; give up after `timeout` seconds without one."""
    try:
        message = await asyncio.wait_for(ws.recv(), timeout)
    except TimeoutError:
        raise RelayError("no_reply") from None
    if isinstance(message, bytes):
        reply.on_binary(message)
    else:
        reply.on_event(json.loads(message))


def _refusal(err: InvalidStatus) -> RelayError:
    try:
        detail = json.loads(err.response.body or b"")["error"]
        return RelayError(str(detail["code"]), detail.get("request_id"))
    except (ValueError, KeyError, TypeError):
        return RelayError(f"http_{err.response.status_code}")


def _http(opts: RelayOptions) -> httpx.AsyncClient:
    """The HTTP client for creating the session, with a 15 second timeout."""
    return httpx.AsyncClient(transport=opts.transport, timeout=15.0)


async def run_relay(opts: RelayOptions) -> RelaySummary:
    speech = read_pcm16_wav(opts.wav_path)[1] if opts.wav_path else b""
    reply = Reply(opts.out, pause=opts.pause_playback)

    async with _http(opts) as http:
        session = await create_session(
            http,
            api_url=opts.api_url,
            api_key=opts.api_key,
            voice_id=opts.voice_id,
            languages=opts.languages,
            interruptions=opts.interruptions,
        )
    secret = session["client_secret"]["value"]
    try:
        # Servers send no Origin header (the websockets library adds none).
        async with connect(
            session["connect"]["websocket_url"],
            subprotocols=["kheelona.v1"],
            additional_headers={"Authorization": f"Bearer {secret}"},
        ) as ws:
            # Send nothing until the server says session.ready: audio sent before it is dropped.
            while not reply.session_id:
                await _receive(ws, reply, opts.reply_timeout)
            if opts.pause_playback:  # tell the voice this client can pause and resume its playback buffer
                await ws.send(json.dumps({"type": "client.capabilities", "playback_pause": True}))
            source = FrameSource()
            stop = asyncio.Event()
            turns = 0
            if opts.text:  # the typed turn goes first, and its reply is awaited before any audio is sent
                await ws.send(json.dumps({"type": "input.text", "text": opts.text}))
                turns += 1
            pumping = asyncio.create_task(pump(ws.send, source, stop))
            try:
                while reply.answered < turns:
                    await _receive(ws, reply, opts.reply_timeout)
                if speech:
                    source.load(speech)  # real time, 20 ms frames, then silence until the reply is done
                    turns += 1
                    while reply.answered < turns:
                        await _receive(ws, reply, opts.reply_timeout)
            finally:
                stop.set()
                await asyncio.gather(pumping, return_exceptions=True)  # the first error wins
            await ws.send(json.dumps({"type": "session.end"}))
            while reply.ended is None:
                await _receive(ws, reply, opts.reply_timeout)
    except InvalidStatus as err:
        raise _refusal(err) from None
    except RelayError as err:
        err.session_id = err.session_id or reply.session_id or None
        raise
    except (WebSocketException, OSError) as err:
        raise RelayError(f"connection_failed_{type(err).__name__}", session_id=reply.session_id or None) from None

    write_wav(opts.out_path, bytes(reply.pcm), reply.output_rate)
    seconds = len(reply.pcm) / 2 / reply.output_rate
    summary = RelaySummary(reply.session_id, reply.ended, reply.done, seconds, opts.out_path)
    opts.out(
        f"session {summary.session_id} ended {summary.end_reason} replies {summary.replies} "
        f"audio {seconds:.1f}s saved {summary.out_file}"
    )
    return summary


def main(argv: Sequence[str] | None = None, env: Mapping[str, str] = os.environ) -> int:
    parser = argparse.ArgumentParser(description="Talk to a Kheelona voice from a script and save the reply audio.")
    parser.add_argument("--text", help="a typed turn (the voice answers it first)")
    parser.add_argument("--wav", help="a 16-bit mono 16 kHz WAV file, sent as a spoken turn")
    parser.add_argument("--out", default="reply.wav", help="where to save the reply audio (default reply.wav)")
    parser.add_argument("--voice", default=env.get("KHEELONA_VOICE_ID", "lumi-dragon"))
    parser.add_argument("--languages", default="en,hi", help="comma-separated language codes (default en,hi)")
    parser.add_argument("--no-interruptions", action="store_true", help="the voice is never cut off")
    parser.add_argument(
        "--no-pause", action="store_true", help="do not declare playback_pause (the voice never pauses)"
    )
    args = parser.parse_args(argv)
    if not args.text and not args.wav:
        parser.error("give --text, --wav or both")
    if args.text and len(args.text) > 2000:
        parser.error("--text is at most 2000 characters")
    api_key = env.get("KHEELONA_API_KEY")
    if not api_key:
        print("Set KHEELONA_API_KEY to your Kheelona Voice API key (server side only).", file=sys.stderr)
        return 2
    if args.wav:
        try:
            read_pcm16_wav(args.wav)
        except (ValueError, OSError) as err:
            print(err, file=sys.stderr)
            return 2
    opts = RelayOptions(
        api_url=env.get("KHEELONA_API_URL", "https://api.voice.kheelona.com").rstrip("/"),
        api_key=api_key,
        voice_id=args.voice,
        languages=[code.strip() for code in args.languages.split(",") if code.strip()],
        text=args.text,
        wav_path=args.wav,
        out_path=args.out,
        interruptions=not args.no_interruptions,
        pause_playback=not args.no_pause,
    )
    try:
        asyncio.run(run_relay(opts))
    except KheelonaApiError as err:
        print(f"error {err.code} request_id {err.request_id or '-'}", file=sys.stderr)
        return 1
    except RelayError as err:
        session = f" session {err.session_id}" if err.session_id else ""
        print(f"error {err.code}{session} request_id {err.request_id or '-'}", file=sys.stderr)
        return 1
    except httpx.HTTPError as err:
        print(f"error api_unreachable_{type(err).__name__} request_id -", file=sys.stderr)
        return 1
    return 0


if __name__ == "__main__":
    sys.exit(main())
```

The examples are Kheelona's own and are given to partners privately. Contact the Kheelona team for licensing.

---

# Browser SDK

> Connect a web page to a Kheelona voice. Options, transports, events, controls, security policy and errors.

Source: https://docs.voice.kheelona.com/guides/browser-sdk/

The browser SDK connects a web page to a voice. It sets up the audio, tries WebRTC first and falls back to a
WebSocket by itself, and gives you events to drive your page. It has no runtime dependencies.

The SDK is proprietary and shared privately. Install the package you received from the Kheelona team, for example
`npm install ./kheelona-voice-<version>.tgz`. It is not on a public registry. Contact the Kheelona team for access.
The [browser quickstart](https://docs.voice.kheelona.com/get-started/quickstart-browser.md) runs the whole example. Every call is listed in the
[SDK reference](https://docs.voice.kheelona.com/reference/sdk/classes/kheelonavoice.md).

## The flow

1. Your **server** creates a session with your API key and sends the page the session ID, the client secret and the
   connection addresses. Never send the key. See [Sessions](https://docs.voice.kheelona.com/concepts/sessions.md).
2. The page calls `KheelonaVoice.connect(...)` from a click.
3. The page listens to the session's events.

Here is the example page doing steps 1 and 2. Call `connect` from a click or a tap: the browser starts the audio and
the microphone only after a user gesture.

```js title="public/main.js"
async function connectToVoice() {
  // Your server creates the session with your API key. The page only ever gets a single-use client secret.
  const res = await fetch('/api/voice-session', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: '{}',
  });
  if (!res.ok) {
    const body = await res.json().catch(() => ({}));
    const retryAfter = Number(res.headers.get('Retry-After'));
    throw new MintError(body.error?.code ?? 'mint_failed', res.status, retryAfter > 0 ? retryAfter * 1000 : undefined);
  }
  const { sessionId, clientSecret, endpoints } = await res.json();
  // Call this from a click: the browser starts the audio and the microphone only after a user gesture.
  // A failed connect needs a new session (a new secret), so every attempt mints again.
  return KheelonaVoice.connect({ sessionId, clientSecret, endpoints, workletBaseUrl: '/sdk/' });
}
```

`connect` returns a session once the voice is ready. If it fails, it rejects with a `KheelonaVoiceError`. **A failed
`connect` needs a new session**: create a new one on your server and try again. The client secret may be spent or
expired. The one exception is `capacity_exhausted` with a `retryAfterMs`: you may retry with the same secret after that
wait, while the secret is under 60 seconds old. When the wait is a second or two (one server is busy), `connect` already
retries by itself, up to 3 times, before it rejects.

## Options

These are the options of `connect`. The full list is in the
[ConnectOptions reference](https://docs.voice.kheelona.com/reference/sdk/interfaces/connectoptions.md).

| Option | What it does |
|---|---|
| `sessionId`, `clientSecret` | Required. From your server's `POST /v1/sessions` answer. The secret is never your API key. |
| `endpoints` | `{ webrtcUrl, websocketUrl }`: the `connect` addresses from the same answer. Without it the SDK uses the production service. |
| `transport` | `auto` (the default), `webrtc` or `websocket`. |
| `microphone` | `{ deviceId }` to choose an input device, or `false` for typed turns only: no microphone is opened. |
| `workletBaseUrl` | Where the audio worklet files are served from. See [Content Security Policy](#content-security-policy). |

## Transports

With `auto` the SDK tries **WebRTC** first. If WebRTC cannot connect, it opens a **WebSocket** with the same client
secret. This happens when:

- the browser has no WebRTC, or it fails to set up the connection;
- the network fails before the connection is up;
- the connection is not ready within **3 seconds**.

It does not happen when the service answers with an error (an HTTP error answer rejects `connect` with that error), and
it does not happen after the connection has opened. Both transports give you the same events. After `connect`,
`session.transport` is `webrtc` or `websocket`.

`transport: 'webrtc'` or `'websocket'` forces one and never falls back. The WebSocket path loads two small audio worklets.

## Subscribe after `await connect()`

Attach your listeners right after `await connect()`, as the example does:

```js title="public/main.js"
function listen(voice) {
  // Attach after `await connect()`: nothing is missed, and `state` first gives the current state.
  voice.on('state', (state) => {
    $('state').textContent = state;
  });
  voice.on('transcript', ({ text }) => addCaption('You', text));
  voice.on('replySentence', ({ text }) => addCaption('Voice', text));
  voice.on('error', (e) => {
    if (e.code === 'audio_unavailable') {
      // The browser refused to play the voice: ask for a tap (see the "Tap to allow sound" button).
      $('allow-sound').hidden = false;
    } else {
      showError(e);
    }
  });
  voice.on('ended', () => finish());
}
```

Nothing is missed. Events that arrived while `connect` was running, including the ready event, are delivered to you in a
moment after it resolves. A new `state` listener first gets the current state, once, and then every change. At that point
`session.state` is `listening`. `on` returns a function that removes the listener.

## Events

| Event | When | Payload |
|---|---|---|
| `state` | The session changes state: `listening`, `thinking`, `speaking`. It is `ended` at the end. | the state |
| `transcript` | What the person said, once the turn is understood. | `turn`, `text`, `language`, `languageStage`, `languageBeta` |
| `replySentence` | One sentence of the reply, as it starts to play. | `replyId`, `index`, `text`, `emotion` |
| `interrupted` | The reply was cut off. | `replyId`, `heardMs` |
| `safety` | A safety rule acted on a turn. See [Safety levels](https://docs.voice.kheelona.com/concepts/safety-levels.md). | `turn`, `stage`, `category`, `action` |
| `warning` | The session will end soon. | `code` (`max_duration_soon` or `idle_soon`), `secondsLeft` |
| `error` | A problem. If `fatal` is false the session goes on. | `code`, `message`, `retryable`, `fatal` |
| `ended` | The session is over. Fires once. | `reason`, `durationMs`, `turns` |
| `event` | Every event from the service, as received. | the raw event |

Captions and transcripts are for your page to show. Kheelona does not keep them. See [Captions](https://docs.voice.kheelona.com/guides/captions.md).
The names and types are in the [SessionEvents reference](https://docs.voice.kheelona.com/reference/sdk/interfaces/sessionevents.md).

## Mute and interrupt

`mute(true)` stops sending the microphone. `mute(false)` starts again. While you are muted, the idle timer is paused.
`interrupt()` stops the voice now, as if the person had started to talk. While the voice is speaking, the reply is cut
and you get `interrupted`. When it is not speaking, nothing is cut and you get no `interrupted` event. Use it for a Stop button or a push-to-talk page.

```js title="public/controls.js"
/** Wires a mute toggle and an interrupt button. `getSession` returns the live session, or null when there is none. */
export function wireControls(getSession, { muteButton, interruptButton }) {
  muteButton.addEventListener('click', () => {
    const session = getSession();
    if (!session) return;
    // The button holds the state: aria-pressed is "true" while the microphone is muted.
    const muted = muteButton.getAttribute('aria-pressed') !== 'true';
    session.mute(muted);
    muteButton.setAttribute('aria-pressed', String(muted));
    muteButton.textContent = muted ? 'Unmute' : 'Mute';
  });
  interruptButton.addEventListener('click', () => {
    // Stops the voice now, as if the person had started speaking.
    getSession()?.interrupt();
  });
}
```

## Typed turns, loudness and ending

Three more calls have no snippet here: they are one line each. See the reference for each.

- [`sendText(text)`](https://docs.voice.kheelona.com/reference/sdk/interfaces/voicesession.md#sendtext) sends a typed turn of at most 2,000 characters. It
  goes through the same safety checks. On a `child` voice, what is said aloud is checked a little more strictly than
  typed text, because speech recognition can change the words. A typed turn ends whatever the voice is doing. Longer text throws a `RangeError`.
  With `microphone: false` it is the only way to talk.
- [`levels()`](https://docs.voice.kheelona.com/reference/sdk/interfaces/voicesession.md#levels) returns `{ input, output }`, each from 0 to 1, for a
  meter. Call it from `requestAnimationFrame`.
- [`end()`](https://docs.voice.kheelona.com/reference/sdk/interfaces/voicesession.md#end) ends the session and releases the microphone. It returns a
  promise that resolves when the service has confirmed, or after 5 seconds at most. Call it when your page closes the
  conversation. A session also ends by itself at its time limit or after a long silence.

## SDK version

The SDK has its own version number, which is the number in the package file's name (`kheelona-voice-<version>.tgz`). It
follows semantic versioning: a major version can change how your page calls the SDK. The
[changelog](https://docs.voice.kheelona.com/policies/changelog.md) lists new versions. Test your page after you update.

## No sound

If the browser refuses to play the voice, for example because `connect` was not called from a click, the SDK sends a
non-fatal `error` event with the code `audio_unavailable`. The session goes on, and captions still work. The example
page shows a **Tap to allow sound** button. When the person taps it, call `session.resumeAudio()` **synchronously in the
click handler**, with no `await` before it. The example does it like this:

```js title="public/main.js"
// A tap is the gesture the browser needs: call resumeAudio() directly in the handler, with no await before it.
$('allow-sound').addEventListener('click', () => {
  session?.resumeAudio();
  $('allow-sound').hidden = true;
});
```

The call resumes the audio and tries to play the voice again inside the tap. If the browser still refuses, you get
`audio_unavailable` again.

## Content Security Policy

If your page has a Content Security Policy, three things matter:

- **Connections.** `connect-src` must allow the origins of the `connect` URLs your server got, for example
  `https://api.voice.kheelona.com` and `wss://api.voice.kheelona.com`.
- **Audio worklets on the WebSocket path.** The SDK loads two worklet files. Either host them yourself, from your own
  origin, and pass `workletBaseUrl`: the files are `capture.worklet.js` and `playback.worklet.js` in the SDK package's
  `dist` folder, and the example does this with `workletBaseUrl: '/sdk/'`. Or allow `script-src blob:`, and the SDK loads
  them from its own bundle. If a worklet cannot load, you get `audio_unavailable` rather than a hang.
- **Scripts and styles.** The example needs only `'self'` for both. It runs with no inline script and no inline style.

The complete policy of the example is on the [Security](https://docs.voice.kheelona.com/guides/security.md#a-content-security-policy-for-your-page) page.

## Errors

`connect` rejects, and the `error` event carries, a code and two flags:

- `code`: branch on this, never on the message. The [error list](https://docs.voice.kheelona.com/guides/errors-and-retries.md) has every code. The SDK adds
  four of its own: `microphone_unavailable` (missing, blocked or in use), `audio_unavailable` (see above), `network`
  (the network failed) and `transport_failed` (neither transport could be set up).
- `retryable`: trying the same thing again later can work.
- `retryAfterMs`: when the service said how long to wait, in milliseconds. It is set only when `connect` fails on the
  WebRTC path.

**Do not log `error.cause` as it is.** When a browser's own error is kept as the cause, its text can contain connection
details. Log the `code` and the `message`; both are safe.

A connection that drops after it was up ends the session: you get an `error` with the code `network` and `fatal: true`,
then `ended` with the reason `disconnected`. Start a new session to continue. The new one has no memory of the old one.

## Supported browsers

The SDK is built for the current versions of Chrome, Edge, Firefox and Safari, on desktop and on phones. We test it in
Chromium, Firefox and WebKit. It needs:

- `getUserMedia`, which browsers allow only on `https` pages and on `localhost`;
- `AudioWorklet`;
- WebRTC, or a WebSocket for the fallback.

Safari can refuse to play the voice when `microphone` is `false`: treat `audio_unavailable` as above.

## Next

- [Interruptions and echo](https://docs.voice.kheelona.com/guides/interruptions-and-echo.md)
- [Captions](https://docs.voice.kheelona.com/guides/captions.md)
- [Security](https://docs.voice.kheelona.com/guides/security.md)

---

# Errors and retries

> Every error code, whether it ends the session, and what to do. How to retry without making things worse.

Source: https://docs.voice.kheelona.com/guides/errors-and-retries/

Errors come in two places.

- **Before a connection exists**, as an HTTP answer with a JSON body:
  `{"error":{"code":"…","message":"…","request_id":"req_…"}}`. This covers `POST /v1/sessions` and the WebSocket upgrade.
- **During a session**, as an `error` event with `code`, `message`, `retryable` and `fatal`. An `error` event has no request ID. If `fatal` is true, the
  session ends next with `session.ended`. If not, the session carries on.

Match on `code`, never on the message text. New codes can be added: treat one you don't know as a failure and look at the
HTTP status. **Keep the `request_id` of every HTTP answer**: it is what lets us find the request. For an `error` event, keep the session ID and the `code`.

## Every code

| Code | HTTP status | Fatal in a session | What to do |
|---|---|---|---|
| `invalid_request` | 400 | no | Fix the request or the event. `details.field` names the field. A replayed `Idempotency-Key` that doesn't match, or whose session is already live, answers 409 with this code: send a new `Idempotency-Key` for a new request, and reuse the same key only to retry the same request. |
| `invalid_audio` | none (event only) | no | Fix the audio: PCM16, mono, frames of 10 to 500 ms, no faster than 2 times real time. The bad frame is dropped. |
| `unauthenticated` | 401 | yes | The API key or client secret is missing or wrong. Check the key and the `Authorization` header. |
| `secret_expired` | 401 | yes | The client secret is older than about 60 s or was already used. Create a new session. |
| `forbidden` | 403 | yes | The key lacks the scope, the account is suspended, or the monthly minutes are used up (`details.reason`). Retrying won't help. |
| `origin_not_allowed` | 403 | yes | Browsers only: add the page's origin in your console (see [Security](https://docs.voice.kheelona.com/guides/security.md#allowed-origins)). A server client must not send an `Origin` header at all. |
| `voice_not_found` | 404 | no | The voice is unknown, retired or not enabled for you. List yours with `GET /v1/voices`. |
| `session_not_found` | 404 | yes | No such session for your account. Check the ID. |
| `capacity_exhausted` | 503 with `Retry-After` | no | Kheelona is full right now. Wait the `Retry-After` seconds (1 to 30), then try again. At connect, `Retry-After: 1` means only the server you reached is busy: connect again with the same client secret after a second. When `details.reason` is `voice_service_starting` (waking up, `Retry-After: 60`) or `voice_service_asleep` (switched off, `Retry-After: 300`), the voice service is not running: wait that long. |
| `partner_limit_reached` | 429 | no | Your account is at its concurrent-session limit. Wait for a session to end. Ask the Kheelona team if you need more. |
| `rate_limited` | 429 with `Retry-After` | no | Too many API calls, or typed turns sent too fast in a session. Wait the `Retry-After` seconds (during a session, about a second) and slow down. |
| `provider_unavailable` | none (event only) | no, then yes | A step of the reply failed. Usually the voice asks the person to say it again. If the voice's own speech failed, there is no spoken line: the reply just ends with `reply.done` and `reason: error`. Either way the session goes on. After 3 failures in a row the error is fatal and the session ends as `provider_failure`. |
| `internal` | 500 | yes | Our fault. Quote the `request_id`. Try again once after a short wait. |
| `not_found` | 404 | not used | REST only: the resource doesn't exist, or isn't visible to your key. |
| `conflict` | 409 | not used | REST only: the request clashes with the current state. Change the request. |
| `unavailable` | 503 with `Retry-After` | not used | REST only: the API is briefly unavailable. Wait the `Retry-After` seconds and try again. |

"Not used" means the code is never sent on the WebSocket. `not_found`, `conflict` and `unavailable` appear only on REST
calls.

## The retry policy

Retry only what can succeed.

- **Honour `Retry-After`.** When an answer has it, wait at least that many seconds. It is in whole seconds. Don't retry
  sooner, and don't retry in parallel. The example clients wait up to 30 seconds and give up after 3 attempts.
- **Retry on 429 and 503** (`rate_limited`, `capacity_exhausted`, `partner_limit_reached`, `unavailable`). Use one
  `Idempotency-Key` for all attempts, so a retry can never make a second session. For a 500 `internal`, one retry after
  a short wait is fine.
- **Don't retry 400, 401, 403 or 404.** They fail the same way each time. Fix the cause.
- **`secret_expired` means a new session.** Create one with a fresh `Idempotency-Key` and connect at once. Connect within
  the 60 seconds: don't hold a secret in a queue.
- **A failed connection means a new session.** Once the voice was ready, the client secret is used up, and a failure
  before that may have spent it too. Creating a new session is always safe.
- **`capacity_exhausted` is normal at busy moments.** Show the user a short, friendly "busy" message and retry later
  rather than looping. See [Limits](https://docs.voice.kheelona.com/concepts/limits.md). If `details.reason` is `voice_service_starting` or
  `voice_service_asleep`, the voice service is waking up or switched off: tell the user it will be back shortly and
  retry after the `Retry-After` seconds (60 or 300).
- **`partner_limit_reached`** usually means sessions are left open. Make sure every session ends, with `session.end` or
  `POST /v1/sessions/{id}/end`.
- **`rate_limited`:** slow down. On `POST /v1/sessions` it means too many requests. **During a session** it means typed turns (`input.text`) came faster than 5 at once, then 1 per second: that turn was dropped, the error is not fatal and the session goes on, so send the turn again after a second. **At connect** it means the client
  secret was refused, and many refusals came from your address in the last minute: don't retry that secret. Find why
  connects are refused (an expired secret, a wrong origin), then create a new session, after `Retry-After`. A valid
  secret still connects from the same address.

Give up after a few attempts and tell the user. A retry loop with no end can make a busy moment worse for everyone.

## Troubleshooting

**A browser can't connect and the SDK reports `network`.** The WebSocket upgrade was refused or never completed, and a
browser doesn't show why. Check two things before anything else:

- **The origin list.** The page's exact origin (scheme, host and port) must be in your console's allowed origins. The
  server answers `origin_not_allowed`, but the browser only tells your code the connection failed. See
  [Security](https://docs.voice.kheelona.com/guides/security.md#allowed-origins).
- **The client secret's age.** A secret is good for about 60 seconds and works once. If the session was created earlier,
  or a first attempt already used the secret, create a new session and connect at once.

If both are right, quote the session ID to us.

## Quote the request ID

Every HTTP error answer has a `request_id`. An `error` event during a session has none: quote the session ID and the error
code instead. The example relays print what they have on standard error as
`error <code> session <session id> request_id <id>`, with `session` only once the session existed and `-` where there is no
request ID. Send us that line and the time, and we can find the session. It holds no conversation.

---

# Interruptions and echo

> How a person can talk over the voice, what your client has to do when it happens, and why every device must cancel echo.

Source: https://docs.voice.kheelona.com/guides/interruptions-and-echo/

A person can talk while the voice is speaking, and the voice stops. This is called a barge-in. This page explains what
happens, what your client does, and how to switch it off for devices that cannot tell the voice from the person.

## What happens in a barge-in

By default a session has `interruptions` on. When the person starts to speak while the voice is speaking, the voice
pauses at once: Kheelona stops sending audio. What happens next depends on how long the sound lasts and what was said.

**The 1 s rule.** If the person is still talking a second later, it is a barge-in. You get `interrupted` with the
`reply_id` and `heard_ms`, how much of the reply the person heard. The reply then ends with `reply.done` and the reason
`interrupted`. The voice forgets the part of the reply the person did not hear, and the person's speech is a new turn.

**A shorter sound** is listened to. Kheelona works out what it was, and the words decide:

| What the person said | What the voice does |
|---|---|
| Nothing useful: a cough, a bang, a noise, or the voice's own echo | Carries on where it paused. No event. |
| A short word of agreement or surprise ("hmm", "okay", "wow") | Carries on. |
| A stop word ("stop", "wait") | Stops. The person's words become their turn, and the voice answers briefly, in character, then listens. |
| A worry ("help", "I'm scared") | Stops. The words go through the safety checks like any turn. |
| Anything, when the voice has just asked a question, or has just checked in, nudged or asked the person to repeat | Stops. The words are the person's answer. |
| Words that the safety checks block | Stops. The words are handled like any blocked turn. |

When the voice stops this way you get `interrupted` and `reply.done` with the reason `interrupted`, as above. The words
are never lost: they are the next turn.

**A long acknowledgement.** If the person says only "haan, haan" for a second or more, or laughs, the voice does not
answer it as a turn. The old reply ends with `interrupted` and `reply.done` with the reason `interrupted`, as above (flush
as usual), and the voice carries on in a **new** reply: a short cue line, with a new `reply.started` and no new
`transcript.final`. It never continues the old reply, so you never get `reply.resumed` for this. This happens at most
once in a row, and only after an ordinary reply. It never happens over an invitation: after a question, a check-in, a
nudge or a request to repeat, a long "haan haan" is the person's answer and is answered as a normal turn.

**Other words said over the voice.** When the voice carries on after a short sound, those other words are not used
today: the voice does not take them into account in its next reply. Stop words, worries, blocked words and answers over
a question always stop the voice and become the person's turn (the table above), so nothing that needs an answer is
dropped this way.

Your client does not need to do anything for this. The voice pauses and resumes inside Kheelona. The only events you see
are the ones above, plus the optional pause events below.

**Your client must stop playing at once** on `interrupted` and drop any reply audio it has queued. The browser SDK does
this for you on both transports. A client on the WebSocket path, like a toy relay, flushes its own queue when
`interrupted` arrives, and drops frames of the interrupted reply that arrive after it.

If the person keeps talking after a pause and the voice has not started to answer yet, the new speech joins the same turn.
Nothing is cut off, because nothing was playing.

You can also stop the voice yourself, for a Stop button or a push-to-talk page. The browser SDK has `session.interrupt()`
and the WebSocket has the `input.interrupt` event. It works whether `interruptions` is on or off.

## Pause events (optional)

When the voice pauses, a WebSocket client keeps playing whatever audio it has already queued, up to about 300 ms. A
relay can do better by pausing its own buffer. This is opt-in, so a client that does nothing behaves as before.

**Declare support.** After `session.ready`, send once:

```json
{ "type": "client.capabilities", "playback_pause": true }
```

Only declare it if your client really can pause and resume its playback. Unknown fields are ignored. Without the
declaration you never get the events below.

**Then handle two events,** each with a `reply_id`:

- `reply.paused`: the voice paused its own playback because the person started to speak. Pause your playback buffer now.
  Keep the audio you hold.
- `reply.resumed`: the pause ended and the same reply plays on. Resume your buffer.

`reply.paused` describes the voice's playback. It is not a cue that the person is speaking, and it does not mean the
voice has stopped for good. You only get these events for a reply you have already seen start. Ignore a `reply.paused`, `reply.resumed` or
`reply.done` that names a reply other than the current one.

**A pause ends in one of four ways.** The first of these ends it:

1. `reply.resumed`: the same reply plays on.
2. `interrupted`: the barge-in was confirmed. Flush your queue, as always.
3. `reply.done`: the reply ended during the pause (for example after an error or a safety redirect).
4. The next `reply.started`: a new reply begins.

Your client must end its pause on any of the four, so it can never stay paused. While a pause lasts, Kheelona stops its
own playback clock, so `heard_ms` and caption timing stay right.

## How a reply is spoken, and what gets cut

The voice may speak several sentences of a reply in one go, so that the intonation flows. The first sentence is spoken
first and alone, so the reply starts quickly. If the person interrupts, the voice still forgets exactly the part of the
reply that was not heard, as above. `reply.sentence` events for the later sentences are sent when each sentence is
estimated to start playing, to within a fraction of a second, so captions stay close to the audio.

## Echo cancellation

A microphone near a loudspeaker hears the voice. If that sound goes back to Kheelona as if it were the person, the voice
interrupts itself.

- **In a browser,** the SDK asks the browser for echo cancellation and noise suppression when it opens the microphone, so
  this is handled for you. Headphones remove the problem completely. A phone on loudspeaker or a laptop with the volume
  up can still leak: a short leak only pauses the voice for a moment, a longer one interrupts it.
- **On your own server or device,** echo cancellation is required, on every device, toys included: the voice's audio
  must not come back in the microphone audio you send. Kheelona still checks for its own echo, but only as a safety net.
  Because words now count when they are said over the voice, a clean microphone matters more than before: a device that
  leaks a lot makes the voice more careful with short sounds for the rest of the session.

## `interruptions: false`

Create the session with `"interruptions": false`:

```json
{ "voice_id": "lumi-dragon", "interruptions": false }
```

Then the voice is never cut off by sound. While the voice is speaking, anything the microphone sends is ignored, and
the person's turn starts after the voice has finished. This also means that nothing said over the voice is heard, not
even a worry. Use it for content that must play through, and keep a button for `interrupt()`. `session.ready` says which setting the session runs with. The
example relay takes `--no-interruptions` to try it.

The trade-off is that people cannot interrupt by talking. Give them a button that calls `interrupt()` if they need one.

## `playback.position` for WebSocket clients

On the WebSocket path the voice's audio is sent in small steps ahead of playback, and Kheelona estimates how much of it
was heard. When you cut the reply, you can send `{"type":"playback.position","reply_id":"…","played_ms":1234}` with how
much you actually played. Kheelona uses it to forget exactly the part that was not heard. It is optional: without it
Kheelona uses its own estimate. It is for WebSocket clients only. The browser SDK sends it for you on the WebSocket
path and sends nothing on WebRTC.

## Next

- [Turn-taking](https://docs.voice.kheelona.com/concepts/turn-taking.md): how the voice decides the person has finished, and the slow-reply line.
- [Server-to-server](https://docs.voice.kheelona.com/guides/server-to-server.md): flushing audio in a relay.
- [Client events](https://docs.voice.kheelona.com/reference/generated/events/client.md): `input.interrupt`, `playback.position` and `client.capabilities`.

---

# Captions

> Show what the person said and what the voice says, as it happens. What the text is, and what to do with it.

Source: https://docs.voice.kheelona.com/guides/captions/

Kheelona sends you the text of the conversation as it happens, so you can show captions or drive an animation. It sends
it live and does not keep it. See [Live text](https://docs.voice.kheelona.com/concepts/live-text.md) for the events.

## What you get

| What | Event | Browser SDK event |
|---|---|---|
| What the person said | `transcript.final`, once the turn is understood | `transcript` |
| What the voice says | `reply.sentence`, one sentence at a time, as that sentence starts to play | `replySentence` |

A person's text arrives **once per turn**, after they have finished. There is no word-by-word text while they speak.
The `language` field says which language was heard. `languageStage` is `supported`, `beta` or `alpha`, and
`languageBeta` is true for the last two, where recognition is less reliable. Show a spoken turn in that case as an
approximation, and label it "Beta" or "Alpha".

A reply's sentences come one after another, each as its audio starts. The `index` counts from 0. A sentence's `emotion`
is how it is spoken. It is a hint for an animation, such as a face, and it is not text: the text has no tags in it.
Today the values are `happy`, `excited`, `sad`, `giggly` and `soft`: `happy` is the voice's normal warm delivery, and the
others mark a sentence spoken with more excitement, more sadness, a laugh or a hushed voice. Older values such as `calm` and
`surprised` are no longer sent. New emotion names can appear, so treat one you don't know as neutral.

## Show them safely

Set captions as text, never as HTML. What the person says and what the voice replies are never trusted markup. The
example does this with `textContent`:

```js title="public/main.js"
// Captions are set with textContent only: what people and the voice say is never read as HTML.
function addCaption(who, text) {
  const line = document.createElement('p');
  line.className = who === 'You' ? 'you' : 'voice';
  line.textContent = `${who}: ${text}`;
  $('captions').append(line);
}
```

## When the voice is cut off

A sentence is shown when it starts to play. If the person interrupts, the sentence may have been heard only in part. The
`interrupted` event has `heardMs`, and the reply then ends. You may mark the last caption as cut off, for example with
an ellipsis.

## Typed turns

A typed turn (`sendText`) also gets a `transcript` event, with the text you sent. Show captions from that event and you
treat spoken and typed turns the same way. Don't add the typed text to the captions yourself as well, or it shows
twice.

## What not to do

- **Don't store captions without a reason and a notice.** Kheelona keeps no conversation text, so the captions you
  receive are the only copy. What you do with them is your responsibility, including telling your users and getting
  consent, especially for children. See the [data and privacy policy](https://docs.voice.kheelona.com/policies/data-and-privacy.md).
- **Don't send captions to third parties** you have not told your users about.
- **Don't log them in your server logs by accident.** Treat them like any other personal conversation.

## Next

- [Browser SDK](https://docs.voice.kheelona.com/guides/browser-sdk.md): the events.
- [Server events reference](https://docs.voice.kheelona.com/reference/generated/events/server.md): every field.

---

# Security

> How to keep your API key safe, who may start a session, which pages may connect, and a Content Security Policy for your page.

Source: https://docs.voice.kheelona.com/guides/security/

Two secrets are in play. Your **API key** is yours alone and lives on your server. A **client secret** is given to one
client for one session and works for about a minute. Most of this page is about keeping them apart.

## Keep API keys on your server

- Put the key in an environment variable or your secret store. Never in a web page, a mobile app, a toy, a repository or
  a log.
- The examples read it only from `KHEELONA_API_KEY` and never print it.
- Give each service its own key, with only the scopes it needs. A route that only starts sessions needs the
  `sessions:write` scope.
- If a key might have leaked, rotate it now (see below).

## Authenticate before you mint

Your server turns a signed-in user into a client secret. If that route is open, anyone who can reach it can start
sessions on your key and use up your limits. Check who is asking first. The example marks the place:

```ts title="server.ts"
/** The mint route: create the session with your API key, hand the page only what it needs. */
async function mintSession(env: BrowserEnv, req: IncomingMessage, res: ServerResponse, csp: string): Promise<void> {
  // Same-origin by design (no CORS headers). Only your own page may call this route: a request with no Origin, or
  // another one, never reaches Kheelona. JSON only, so a cross-site HTML form cannot spend your key.
  if (req.headers.origin !== env.allowedOrigin) {
    sendJson(res, csp, 403, { error: { code: 'origin_not_allowed' } });
    return;
  }
  if (!req.headers['content-type']?.toLowerCase().startsWith('application/json')) {
    sendJson(res, csp, 415, { error: { code: 'unsupported_media_type' } });
    return;
  }
  // Authenticate your user here: only signed-in users may start a session.
  // The request body is never read (this route takes no input), so its size and Content-Length don't matter.
  // One attempt only: to retry on 429/503, reuse one Idempotency-Key as examples/node-server does.
  const upstream = await fetch(`${env.apiUrl.replace(/\/+$/, '')}/v1/sessions`, {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${env.apiKey}`,
      'Content-Type': 'application/json',
      'Idempotency-Key': globalThis.crypto.randomUUID(),
    },
    body: JSON.stringify({
      voice_id: env.voiceId,
      languages: ['en', 'hi'],
      end_user_ref: 'demo-user', // replace with the signed-in user's opaque ID
    }),
    signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS),
  });
  if (!upstream.ok) {
    const body = (await upstream.json().catch(() => ({}))) as { error?: { code?: unknown; request_id?: unknown } };
    const code = plain(body.error?.code) ?? 'unknown_error';
    const requestId = plain(body.error?.request_id);
    console.error(`error ${code} request_id ${requestId ?? '-'}`);
    const retryable = upstream.status === 429 || upstream.status === 503;
    const retryAfter = upstream.headers.get('retry-after');
    sendJson(res, csp, retryable ? upstream.status : 502, { error: { code, ...(requestId ? { requestId } : {}) } }, retryable && retryAfter && /^\d+$/.test(retryAfter) ? { 'Retry-After': retryAfter } : {});
    return;
  }
  const created = (await upstream.json()) as {
    session_id: string;
    client_secret: { value: string };
    connect: { webrtc_url: string; websocket_url: string };
  };
  // Return only what the page needs: never the API key, and nothing else from the answer.
  sendJson(res, csp, 201, {
    sessionId: created.session_id,
    clientSecret: created.client_secret.value,
    endpoints: { webrtcUrl: created.connect.webrtc_url, websocketUrl: created.connect.websocket_url },
  });
}
```

Also:

- Accept the call only from your own page: check the `Origin` header, as the example does, and accept only JSON.
- Send no CORS headers. The route should be same-origin with your page, or behind your own gateway.
- Return only the session ID, the client secret and the connection addresses. Never anything else from the answer.
- Pass your own opaque ID for the user as `end_user_ref`.

## Allowed origins

A browser connects to Kheelona with the client secret, and Kheelona checks the page's origin against your **allowed
origins** list. Open **Allowed origins** in the console and add every page that will use the SDK.

- An origin is a scheme, a host and a port, for example `https://app.example.com`. It is matched exactly.
- Use `https`. Browsers give a page the microphone only on `https` pages and on `localhost`. The console refuses an
  `http` origin except for `localhost` while development access is on.
- Use a public host name. IP addresses and one-word hosts work only while development access is on.
- A server client, such as a toy relay, sends no `Origin` and is not checked this way. It must not send one at all: see
  [Server-to-server](https://docs.voice.kheelona.com/guides/server-to-server.md).

A refused origin gives `origin_not_allowed`, and the client secret is not used up.

### Development access

While you build, your page runs on `http://localhost:<port>`. A Kheelona admin turns on **development access** for your
account, and then any `http://localhost` address is accepted, listed or not. The console shows a banner while it is on.
**Ask the Kheelona team to turn it off before you go live.**

## A pseudonymous `end_user_ref`

`end_user_ref` ties a session record to a user of yours. Use an ID that only you can map to a person, such as `u_8f3a2c`:
at most 128 characters, from letters, digits and `. _ : -`. Never a name, an email address, a phone number or a child's
details. It is kept with the session record. So are `metadata` values: no personal data there either.

## Sign in to the console

From the public launch, every console account sets up a second sign-in step at its first sign-in: a code from an authenticator app on the person's phone. Each person does this for their own account. Before the public launch the second step may be optional.

## Revoke and rotate keys

- **Revoke a key** in the console, under **API keys**. A revoked key stops working within 60 seconds.
- **Rotate** without a gap: create a new key, deploy it, check that sessions still start, then revoke the old one.
- Rotate right away if a key was shown in a log, a chat, a screenshot or a commit.
- Client secrets need no rotation. Each works once and expires after about 60 seconds.

## Instructions are never kept readable

`instructions` is free text that shapes the voice for one session. It is sent encrypted to the voice and Kheelona never
stores or logs it in readable form, and it is processed in a US region, or in Mumbai for a voice set up to write replies in India (and in the US if the service in
India is unavailable; see the [data and privacy
policy](https://docs.voice.kheelona.com/policies/data-and-privacy.md#where-it-is-processed)). Put in only what the voice needs: a first name and an age band rather than a full profile.
Instructions cannot switch off a voice's safety rules.

## A Content Security Policy for your page

A strict policy keeps stray scripts out of the page that holds a live microphone. This is the one the example sends with
every response. It allows only your own origin and the origins of the `connect` URLs:

```ts title="server.ts"
/** The strict policy the page runs under: this origin's files, and the Kheelona realtime origins to connect to. */
export function contentSecurityPolicy(realtimeOrigins: string): string {
  return [
    "default-src 'self'",
    "script-src 'self'",
    "style-src 'self'",
    `connect-src 'self' ${realtimeOrigins}`,
    "img-src 'self'",
    "media-src 'self'",
    "object-src 'none'",
    "base-uri 'self'",
    "form-action 'self'",
    "frame-ancestors 'none'",
  ].join('; ');
}
```

- `connect-src` has your own origin and the Kheelona origins, for example `https://api.voice.kheelona.com` and
  `wss://api.voice.kheelona.com`. The example reads them from `KHEELONA_REALTIME_ORIGINS`.
- `script-src 'self'` works because the example hosts the SDK's audio worklets itself and passes `workletBaseUrl`. If you
  cannot host them, add `blob:` to `script-src`. See the [browser SDK guide](https://docs.voice.kheelona.com/guides/browser-sdk.md#content-security-policy).
- No inline scripts and no inline styles. Put your code in files.

## Next

- [Going live](https://docs.voice.kheelona.com/guides/going-live.md)
- [Sessions and client secrets](https://docs.voice.kheelona.com/concepts/sessions.md)

---

# Going live

> A checklist to work through before real users reach your integration, and what to know about the missing sandbox.

Source: https://docs.voice.kheelona.com/guides/going-live/

There is **no separate sandbox or test mode**. You integrate against the live service. The first launch is limited and
partners are hand-picked, so Kheelona may add a test mode later. Until then, build carefully, and expect your tests to
use real sessions.

Every session counts against your account: it takes one of your concurrent places and its minutes are counted. Test within
your **session limit**. Ask the Kheelona team what your limits are: the maximum number of sessions at once, the
longest session and, if one is set, a monthly allowance. See [Limits](https://docs.voice.kheelona.com/concepts/limits.md).

Kheelona is **best effort during the partner pilot**, and planned maintenance is announced in advance. Your product should
cope with a refused session, a busy answer and a dropped connection.

## Checklist

**Keys and secrets**

- [ ] The API key is only on your server, in a secret store, and in no repository, page, app or log.
- [ ] Your mint route authenticates the user before it creates a session.
- [ ] You know how to rotate a key, and you have done it once.

**Browsers**

- [ ] Your production page is on `https`, and its origin is in **Allowed origins**.
- [ ] You asked Kheelona to turn **development access** off, and the page still works.
- [ ] Your page has a strict Content Security Policy. See [Security](https://docs.voice.kheelona.com/guides/security.md).
- [ ] The microphone is requested from a click, and the page explains what to do when it is denied.
- [ ] "Tap to allow sound" is handled with `resumeAudio()`.
- [ ] You tested on the browsers and phones your users have, on a mobile network too, which may use the WebSocket
  fallback.

**Errors and limits**

- [ ] You handle every code in [Errors and retries](https://docs.voice.kheelona.com/guides/errors-and-retries.md), and you retry only what is retryable,
  with a new session after a failed connect.
- [ ] When Kheelona is busy, your product says so kindly and tries again after the wait.
- [ ] When the session is about to end (the `warning` event), your product can say goodbye or start a new session.
- [ ] You keep the `request_id` of every HTTP error, and the session ID with the code of every `error` event, so we can find it.

**Users and privacy**

- [ ] `end_user_ref` is an opaque ID of yours, and `metadata` holds no personal data.
- [ ] `instructions` holds only what the voice needs.
- [ ] You have the consent you need from your users, including a parent's approval for children, and you tell them that
  conversations are handled by AI services. See the [data and privacy policy](https://docs.voice.kheelona.com/policies/data-and-privacy.md).
- [ ] You know what you do with captions and transcripts: Kheelona does not keep them.
- [ ] If you use the `safety` event for safeguarding, someone is responsible for reading it.

**The conversation**

- [ ] You tried every language you declare. Beta and alpha languages are marked, so test them with care.
- [ ] You chose `interruptions` for your devices. See [Interruptions and echo](https://docs.voice.kheelona.com/guides/interruptions-and-echo.md).
- [ ] A long silence and the time limit end a session cleanly in your product.

## Then

Start small. Let a few real users in, watch the **Connection log** and **Usage** pages of your console and your own logs, then widen.

---

# REST API

> The /v1 REST API: base URL, authentication, scopes and every operation.

Source: https://docs.voice.kheelona.com/reference/generated/api/

The REST API lives at `https://api.voice.kheelona.com`. All paths start with `/v1`. Requests and responses are JSON.

## Authentication

`Authorization: Bearer kv_live_…`: a partner API key, created in your Kheelona console. Server-to-server only; never ship it to a browser or a device. Revocation takes effect within 60 s.

Every response carries an `X-Request-Id` header. Errors use one shape (see [Errors](https://docs.voice.kheelona.com/reference/generated/errors.md)).

## Scopes

An API key carries the scopes it was created with; an operation needs the scope shown.

| Scope | Operations |
| --- | --- |
| `sessions:read` | [`listSessions`](https://docs.voice.kheelona.com/reference/generated/api/listsessions.md), [`getSession`](https://docs.voice.kheelona.com/reference/generated/api/getsession.md) |
| `sessions:write` | [`createSession`](https://docs.voice.kheelona.com/reference/generated/api/createsession.md), [`endSession`](https://docs.voice.kheelona.com/reference/generated/api/endsession.md) |
| `usage:read` | [`getUsage`](https://docs.voice.kheelona.com/reference/generated/api/getusage.md) |
| `voices:read` | [`listVoices`](https://docs.voice.kheelona.com/reference/generated/api/listvoices.md), [`getVoice`](https://docs.voice.kheelona.com/reference/generated/api/getvoice.md) |

## Operations

### sessions

Create sessions, read the connection log, end sessions

| Operation | Endpoint | Summary |
| --- | --- | --- |
| [`listSessions`](https://docs.voice.kheelona.com/reference/generated/api/listsessions.md) | `GET /v1/sessions` | The partner's connection log, newest first. |
| [`createSession`](https://docs.voice.kheelona.com/reference/generated/api/createsession.md) | `POST /v1/sessions` | Creates a session. |
| [`getSession`](https://docs.voice.kheelona.com/reference/generated/api/getsession.md) | `GET /v1/sessions/{id}` | One of the partner's sessions. |
| [`endSession`](https://docs.voice.kheelona.com/reference/generated/api/endsession.md) | `POST /v1/sessions/{id}/end` | Ends a live session (reason `partner_ended`; the live connection closes within 10 s) or cancels a reserved one. Ending a finished session is a no-op. |

### voices

The voices enabled for your partner

| Operation | Endpoint | Summary |
| --- | --- | --- |
| [`listVoices`](https://docs.voice.kheelona.com/reference/generated/api/listvoices.md) | `GET /v1/voices` | The voices enabled for this partner. |
| [`getVoice`](https://docs.voice.kheelona.com/reference/generated/api/getvoice.md) | `GET /v1/voices/{id}` | One voice. |

### languages

The supported languages

| Operation | Endpoint | Summary |
| --- | --- | --- |
| [`listLanguages`](https://docs.voice.kheelona.com/reference/generated/api/listlanguages.md) | `GET /v1/languages` | The supported languages. |

### usage

Usage for invoicing and cost analysis

| Operation | Endpoint | Summary |
| --- | --- | --- |
| [`getUsage`](https://docs.voice.kheelona.com/reference/generated/api/getusage.md) | `GET /v1/usage` | Usage for the key's partner. |

### health

Service health

| Operation | Endpoint | Summary |
| --- | --- | --- |
| [`getHealth`](https://docs.voice.kheelona.com/reference/generated/api/gethealth.md) | `GET /v1/health` | The service's health. |

---

# The partner's connection log, newest first

> GET /v1/sessions: The partner's connection log, newest first.

Source: https://docs.voice.kheelona.com/reference/generated/api/listsessions/

`GET /v1/sessions`

| Property | Value |
| --- | --- |
| Operation | `listSessions` |
| Scope | `sessions:read` |

## Parameters

| Name | In | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- | --- |
| `from` | query | string | no | Created at or after (RFC 3339); default: `to` − 31 days. | format `date-time` |
| `to` | query | string | no | Created before (RFC 3339); default: now (plus a 5-minute clock-skew margin). At most 31 days after `from`. | format `date-time` |
| `voice_id` | query | string | no | Only this voice. |  |
| `status` | query | string | no | Only this status. | one of `reserved`, `live`, `ended`, `expired`, `rejected`, `error` |
| `end_reason` | query | string | no | Only sessions that ended for this reason. | one of `client_ended`, `partner_ended`, `max_duration`, `idle_timeout`, `disconnected`, `server_shutdown`, `server_lost`, `provider_failure`, `suspended`, `service_stopped`, `error` |
| `end_user_ref` | query | string | no | Only this end user. |  |
| `api_key_id` | query | string | no | Only sessions created by this key. |  |
| `limit` | query | integer | no | Rows per page, 1–100 (default 25). | format `int32`; min 1; max 100; default `25` |
| `cursor` | query | string | no | `next_cursor` from the previous page; absent for the first page. |  |

## Responses

### 200

A page of sessions, newest first

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `data` | array of object | yes | The rows, in the list's order (newest first, except the members lists, which are oldest first). |  |
| `data[].id` | string | yes | `ses_…` |  |
| `data[].status` | string | yes | Lifecycle state. A session's lifecycle state. | one of `reserved`, `live`, `ended`, `expired`, `rejected`, `error` |
| `data[].voice` | object | yes | The voice and its pinned version. |  |
| `data[].voice.id` | string | yes | The voice ID. |  |
| `data[].voice.version` | integer | no | The pinned version. | format `int32`; may be null |
| `data[].languages` | array of string | yes | The declared languages. |  |
| `data[].client` | object | yes | Where the client connected from. |  |
| `data[].client.kind` | string | yes | Who connected. `browser`: your web page, through the SDK; `server`: your server or device relay; `playground`: a session started from your console's playground; `admin`: a Kheelona support or test session (internal tools). | one of `browser`, `server`, `playground`, `admin` |
| `data[].client.ip` | string | no | The client's IP; a network (`203.0.113.0/24`) after 30 days. | may be null |
| `data[].client.region` | string | no | ISO 3166-2 region of the IP, e.g. `IN-KA`, looked up offline (country and region only, never an exact location). Region codes are recorded for India only for now; absent for any other country, and when the address is private. | may be null |
| `data[].client.transport` | string | no | How (once connected). How the client connected. | one of `webrtc`, `websocket`; may be null |
| `data[].client.user_agent` | string | no | The client's user agent. | may be null |
| `data[].created_at` | string | yes | Created. | format `date-time` |
| `data[].turns` | integer | yes | Turns. | format `int32` |
| `data[].interruptions` | integer | yes | Barge-ins. | format `int32` |
| `data[].languages_heard` | array of string | yes | Languages the person spoke. |  |
| `data[].latency` | object | yes | Latency percentiles. |  |
| `data[].latency.v2v_p50_ms` | integer | no | Median end-of-speech → first reply audio. | format `int32`; may be null |
| `data[].latency.v2v_p95_ms` | integer | no | 95th percentile. | format `int32`; may be null |
| `data[].audio` | object | yes | Audio totals. |  |
| `data[].audio.user_speech_ms` | integer | yes | The person's speech. | format `int32` |
| `data[].audio.reply_audio_ms` | integer | yes | The voice's audio. | format `int32` |
| `data[].tokens` | object | yes | Reply model tokens. |  |
| `data[].tokens.input` | integer | yes | Input tokens. | format `int32` |
| `data[].tokens.output` | integer | yes | Output tokens. | format `int32` |
| `data[].tokens.thinking` | integer | yes | Thinking tokens. | format `int32` |
| `data[].safety` | object | yes | Safety interventions. |  |
| `data[].safety.input` | integer | yes | The person's speech blocked. | format `int64`; min 0 |
| `data[].safety.output` | integer | yes | A reply blocked. | format `int64`; min 0 |
| `data[].safety.provider` | integer | yes | Blocked by the model provider. | format `int64`; min 0 |
| `data[].errors` | object | yes | Error counts by error code. |  |
| `data[].api_key_id` | string | no | The key that created it. | may be null |
| `data[].connected_at` | string | no | Reached `session.ready`. | format `date-time`; may be null |
| `data[].duration_ms` | integer | no | Connected time. | format `int32`; may be null |
| `data[].end_reason` | string | no | Why it ended, e.g. `client_ended`, `partner_ended`, `idle_timeout`. | may be null |
| `data[].end_user_ref` | string | no | Your opaque user reference. | may be null |
| `data[].ended_at` | string | no | Ended. | format `date-time`; may be null |
| `data[].metadata` | object | no | Your metadata. | may be null |
| `data[].reject_reason` | string | no | Why admission refused it (`status = rejected`): `partner_limit_reached`, `capacity_exhausted`, … | may be null |
| `next_cursor` | string | no | Pass as `cursor` to get the next page; `null` at the end. | may be null |

### 400

Invalid filter or cursor

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

### 401

Missing or invalid API key

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

### 403

The key lacks `sessions:read`

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

### 429

Rate limited

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `Retry-After` | integer | Seconds to wait before retrying |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

---

# Creates a session

> POST /v1/sessions: Creates a session.

Source: https://docs.voice.kheelona.com/reference/generated/api/createsession/

`POST /v1/sessions`

| Property | Value |
| --- | --- |
| Operation | `createSession` |
| Scope | `sessions:write` |

## Parameters

| Name | In | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- | --- |
| `Idempotency-Key` | header | string | no | Retry safely: the same key within 10 minutes returns the same session with a new client secret (the previous one stops working). 1–128 characters from `A–Z a–z 0–9 . _ : -`. |  |

## Request body

Content type `application/json`, required.

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `voice_id` | string | yes | A voice from `GET /v1/voices`: published and enabled for your partner. | pattern `^[a-z][a-z0-9-]{2,39}$` |
| `audio` | object | no | WebSocket audio formats. | may be null |
| `audio.input_format` | string | no | Client → server (default `pcm16_16k`). A PCM audio format: 16-bit little-endian mono at the named sample rate. | one of `pcm16_16k`, `pcm16_24k`, `pcm16_48k`; may be null |
| `audio.output_format` | string | no | Server → client (default `pcm16_24k`). A PCM audio format: 16-bit little-endian mono at the named sample rate. | one of `pcm16_16k`, `pcm16_24k`, `pcm16_48k`; may be null |
| `end_user_ref` | string | no | Your opaque user ID (not personal data): ≤ 128 characters from `A–Z a–z 0–9 . _ : -`. Stored in logs. | pattern `^[A-Za-z0-9._:-]{1,128}$`; may be null |
| `instructions` | string | no | Free text for this session (e.g. the child's name), ≤ 4,000 characters after NFC normalisation. Never stored or logged: it reaches the voice only encrypted. | max length 4000; may be null |
| `interruptions` | boolean | no | `false`: the voice is never cut off by speech while it speaks (for content that must play through). Default `true`. | may be null |
| `languages` | array of string | no | 1–11 distinct codes from `GET /v1/languages`; default `["en", "hi"]`. Speech recognition listens only for these; the voice replies in whichever the person speaks. More than one Indian language, or a beta language, is accepted with a warning. | min items 1; max items 11; may be null |
| `max_duration_s` | integer | no | The longest the session may run, ≤ your partner's limit (default: that limit). | format `int32`; min 1; max 3600; may be null |
| `metadata` | object | no | Your tags: ≤ 10 keys, string values ≤ 200 characters, ≤ 2 KB in total. Stored in logs; no personal data. | may be null |
| `transport` | string | no | Which connection URLs to return (default `auto`: both). Which connection URLs to return. | one of `auto`, `webrtc`, `websocket`; may be null |

Example:

```json
{
  "audio": {
    "input_format": "pcm16_16k",
    "output_format": "pcm16_24k"
  },
  "end_user_ref": "u_8f3a2c",
  "instructions": "The child's name is Aarav. He is 6 and loves dinosaurs. Keep answers very short.",
  "interruptions": true,
  "languages": [
    "en",
    "ta"
  ],
  "max_duration_s": 900,
  "metadata": {
    "app_version": "2.3.1",
    "screen": "home"
  },
  "transport": "auto",
  "voice_id": "lumi-dragon"
}
```

## Responses

### 201

Created (or replayed)

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `session_id` | string | yes | `ses_…` |  |
| `client_secret` | object | yes | The client secret. |  |
| `client_secret.value` | string | yes | `kvcs_…` |  |
| `client_secret.expires_at` | string | yes | When it stops working if unused. | format `date-time` |
| `connect` | object | yes | Where to connect. |  |
| `connect.webrtc_url` | string | no | WebRTC offer endpoint (for browsers). | may be null |
| `connect.websocket_url` | string | no | WebSocket endpoint. | may be null |
| `voice` | object | yes | The pinned voice. |  |
| `voice.id` | string | yes | The voice ID. |  |
| `voice.version` | integer | yes | The pinned version. | format `int32` |
| `voice.name` | string | yes | Display name. |  |
| `languages` | array of string | yes | The declared languages (default `["en", "hi"]`). |  |
| `limits` | object | yes | The session's limits. |  |
| `limits.max_duration_s` | integer | yes | The session ends after this long. | format `int32` |
| `limits.idle_timeout_s` | integer | yes | The session ends after this long without speech. | format `int32` |
| `warnings` | array of Warning | yes | Language advice; empty when none. |  |
| `warnings[].code` | string | yes | `multiple_indian_languages`, `beta_languages` or `alpha_languages`. |  |
| `warnings[].message` | string | yes | What it means (for `beta_languages` and `alpha_languages`, naming the session's languages of that stage). |  |
| `warnings[].languages` | array of string | yes | The languages concerned. |  |

### 400

`invalid_request`: `details.field` names the field

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

### 401

Missing or invalid API key

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

### 403

`forbidden`: the key lacks `sessions:write`, the partner is suspended, or the monthly quota is used up (`details.reason`)

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

### 404

`voice_not_found`

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

### 409

`invalid_request`: an `Idempotency-Key` replay of a session that is live or finished, of a different request, or of one still in progress (`details.reason`)

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

### 429

`partner_limit_reached` (at the partner's concurrent sessions) or `rate_limited`

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `Retry-After` | integer | Seconds to wait before retrying |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

### 503

`capacity_exhausted`: retry after `Retry-After` seconds. `details.reason` is `voice_service_starting` (retry in 60 s) or `voice_service_asleep` (retry in 300 s) while the voice service is not running; it is absent when all capacity is in use

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `Retry-After` | integer | Seconds to wait before retrying |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

---

# One of the partner's sessions

> GET /v1/sessions/{id}: One of the partner's sessions.

Source: https://docs.voice.kheelona.com/reference/generated/api/getsession/

`GET /v1/sessions/{id}`

| Property | Value |
| --- | --- |
| Operation | `getSession` |
| Scope | `sessions:read` |

## Parameters

| Name | In | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- | --- |
| `id` | path | string | yes | The session ID (`ses_…`) |  |

## Responses

### 200

The session (metadata and summary metrics, no content)

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `id` | string | yes | `ses_…` |  |
| `status` | string | yes | Lifecycle state. A session's lifecycle state. | one of `reserved`, `live`, `ended`, `expired`, `rejected`, `error` |
| `voice` | object | yes | The voice and its pinned version. |  |
| `voice.id` | string | yes | The voice ID. |  |
| `voice.version` | integer | no | The pinned version. | format `int32`; may be null |
| `languages` | array of string | yes | The declared languages. |  |
| `client` | object | yes | Where the client connected from. |  |
| `client.kind` | string | yes | Who connected. `browser`: your web page, through the SDK; `server`: your server or device relay; `playground`: a session started from your console's playground; `admin`: a Kheelona support or test session (internal tools). | one of `browser`, `server`, `playground`, `admin` |
| `client.ip` | string | no | The client's IP; a network (`203.0.113.0/24`) after 30 days. | may be null |
| `client.region` | string | no | ISO 3166-2 region of the IP, e.g. `IN-KA`, looked up offline (country and region only, never an exact location). Region codes are recorded for India only for now; absent for any other country, and when the address is private. | may be null |
| `client.transport` | string | no | How (once connected). How the client connected. | one of `webrtc`, `websocket`; may be null |
| `client.user_agent` | string | no | The client's user agent. | may be null |
| `created_at` | string | yes | Created. | format `date-time` |
| `turns` | integer | yes | Turns. | format `int32` |
| `interruptions` | integer | yes | Barge-ins. | format `int32` |
| `languages_heard` | array of string | yes | Languages the person spoke. |  |
| `latency` | object | yes | Latency percentiles. |  |
| `latency.v2v_p50_ms` | integer | no | Median end-of-speech → first reply audio. | format `int32`; may be null |
| `latency.v2v_p95_ms` | integer | no | 95th percentile. | format `int32`; may be null |
| `audio` | object | yes | Audio totals. |  |
| `audio.user_speech_ms` | integer | yes | The person's speech. | format `int32` |
| `audio.reply_audio_ms` | integer | yes | The voice's audio. | format `int32` |
| `tokens` | object | yes | Reply model tokens. |  |
| `tokens.input` | integer | yes | Input tokens. | format `int32` |
| `tokens.output` | integer | yes | Output tokens. | format `int32` |
| `tokens.thinking` | integer | yes | Thinking tokens. | format `int32` |
| `safety` | object | yes | Safety interventions. |  |
| `safety.input` | integer | yes | The person's speech blocked. | format `int64`; min 0 |
| `safety.output` | integer | yes | A reply blocked. | format `int64`; min 0 |
| `safety.provider` | integer | yes | Blocked by the model provider. | format `int64`; min 0 |
| `errors` | object | yes | Error counts by error code. |  |
| `api_key_id` | string | no | The key that created it. | may be null |
| `connected_at` | string | no | Reached `session.ready`. | format `date-time`; may be null |
| `duration_ms` | integer | no | Connected time. | format `int32`; may be null |
| `end_reason` | string | no | Why it ended, e.g. `client_ended`, `partner_ended`, `idle_timeout`. | may be null |
| `end_user_ref` | string | no | Your opaque user reference. | may be null |
| `ended_at` | string | no | Ended. | format `date-time`; may be null |
| `metadata` | object | no | Your metadata. | may be null |
| `reject_reason` | string | no | Why admission refused it (`status = rejected`): `partner_limit_reached`, `capacity_exhausted`, … | may be null |

### 401

Missing or invalid API key

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

### 403

The key lacks `sessions:read`

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

### 404

`session_not_found`

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

### 429

Rate limited

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `Retry-After` | integer | Seconds to wait before retrying |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

---

# Ends a live session (reason `partner_ended`; the live connection closes within 10 s) or cancels a reserved one. Ending
a finished session is a no-op

> POST /v1/sessions/{id}/end: Ends a live session (reason `partner_ended`; the live connection closes within 10 s) or cancels a reserved one. Ending
a finished session is a no-op.

Source: https://docs.voice.kheelona.com/reference/generated/api/endsession/

`POST /v1/sessions/{id}/end`

| Property | Value |
| --- | --- |
| Operation | `endSession` |
| Scope | `sessions:write` |

## Parameters

| Name | In | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- | --- |
| `id` | path | string | yes | The session ID (`ses_…`) |  |

## Responses

### 204

Ended, cancelled, or already finished

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

### 401

Missing or invalid API key

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

### 403

The key lacks `sessions:write`

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

### 404

`session_not_found`

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

### 429

Rate limited

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `Retry-After` | integer | Seconds to wait before retrying |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

---

# The voices enabled for this partner

> GET /v1/voices: The voices enabled for this partner.

Source: https://docs.voice.kheelona.com/reference/generated/api/listvoices/

`GET /v1/voices`

| Property | Value |
| --- | --- |
| Operation | `listVoices` |
| Scope | `voices:read` |

## Responses

### 200

The catalogue enabled for the key's partner

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `data` | array of object | yes | The rows. |  |
| `data[].id` | string | yes | The voice ID (use it as `voice_id` in `POST /v1/sessions`). | pattern `^[a-z][a-z0-9-]{2,39}$` |
| `data[].name` | string | yes | Display name. |  |
| `data[].description` | string | yes | Catalogue description. |  |
| `data[].safety_level` | string | yes | The safety level of the current version. How strictly a voice's input and output are checked. | one of `child`, `general` |
| `data[].version` | integer | yes | The current version (new sessions pin it). | format `int32` |
| `data[].sample_url` | string | no | A sample clip (signed, valid 1 hour); `null` until the voice has one. | may be null |

### 401

Missing or invalid API key

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

### 403

The key lacks `voices:read`

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

### 429

Rate limited

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `Retry-After` | integer | Seconds to wait before retrying |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

---

# One voice

> GET /v1/voices/{id}: One voice.

Source: https://docs.voice.kheelona.com/reference/generated/api/getvoice/

`GET /v1/voices/{id}`

| Property | Value |
| --- | --- |
| Operation | `getVoice` |
| Scope | `voices:read` |

## Parameters

| Name | In | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- | --- |
| `id` | path | string | yes | The voice ID, e.g. `lumi-dragon` |  |

## Responses

### 200

The voice

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `id` | string | yes | The voice ID (use it as `voice_id` in `POST /v1/sessions`). | pattern `^[a-z][a-z0-9-]{2,39}$` |
| `name` | string | yes | Display name. |  |
| `description` | string | yes | Catalogue description. |  |
| `safety_level` | string | yes | The safety level of the current version. How strictly a voice's input and output are checked. | one of `child`, `general` |
| `version` | integer | yes | The current version (new sessions pin it). | format `int32` |
| `sample_url` | string | no | A sample clip (signed, valid 1 hour); `null` until the voice has one. | may be null |

### 401

Missing or invalid API key

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

### 403

The key lacks `voices:read`

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

### 404

`voice_not_found`: unknown, retired or not enabled for this partner

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

### 429

Rate limited

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `Retry-After` | integer | Seconds to wait before retrying |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

---

# The supported languages

> GET /v1/languages: The supported languages.

Source: https://docs.voice.kheelona.com/reference/generated/api/listlanguages/

`GET /v1/languages`

| Property | Value |
| --- | --- |
| Operation | `listLanguages` |
| Scope | none (no API key needed) |

## Responses

### 200

The 11 supported languages

| Header | Type | Description |
| --- | --- | --- |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `data` | array of Language | yes | All 11 languages, in a fixed order. |  |
| `data[].code` | string | yes | ISO 639-1 code. |  |
| `data[].name` | string | yes | English name. |  |
| `data[].stage` | string | yes | `supported`, `beta` or `alpha`: how far the language is tested (see the Languages page). |  |
| `data[].beta` | boolean | yes | Not supported: beta or alpha (kept for older clients; read `stage`). |  |
| `default` | array of string | yes | A session's languages when it declares none. |  |
| `recommendation` | string | yes | How to choose. |  |

---

# Usage for the key's partner

> GET /v1/usage: Usage for the key's partner.

Source: https://docs.voice.kheelona.com/reference/generated/api/getusage/

`GET /v1/usage`

| Property | Value |
| --- | --- |
| Operation | `getUsage` |
| Scope | `usage:read` |

## Parameters

| Name | In | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- | --- |
| `from` | query | string | no | First day (`YYYY-MM-DD`); default: the 1st of this month. | format `date` |
| `to` | query | string | no | Last day, inclusive (`YYYY-MM-DD`); default: today. At most 366 days after `from`. | format `date` |
| `granularity` | query | string | no | `day` (default) or `month`. | one of `day`, `month` |
| `group_by` | query | string | no | `none` (default), `voice` or `api_key`. | one of `none`, `voice`, `api_key` |

## Responses

### 200

Usage rows

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `from` | string | yes | First day. | format `date` |
| `to` | string | yes | Last day (inclusive). | format `date` |
| `granularity` | string | yes | Day or month rows. Rows per day or per month. | one of `day`, `month` |
| `group_by` | string | yes | The extra split. What rows are split by, besides the period and the client kind. | one of `none`, `voice`, `api_key` |
| `data` | array of UsageRowObject | yes | The rows, oldest first. Usage is rolled up every 15 minutes. |  |
| `data[].period` | string | yes | The day, or the month's first day (`YYYY-MM-DD`). | format `date` |
| `data[].client_kind` | string | yes | Who connected. `browser`: your web page, through the SDK; `server`: your server or device relay; `playground`: a session started from your console's playground; `admin`: a Kheelona support or test session (internal tools). | one of `browser`, `server`, `playground`, `admin` |
| `data[].sessions` | integer | yes | Admitted sessions. | format `int64` |
| `data[].rejected_sessions` | integer | yes | Sessions refused at admission. | format `int64` |
| `data[].connected_seconds` | integer | yes | Billable connected seconds (each session rounded up to the next second). | format `int64` |
| `data[].user_speech_seconds` | integer | yes | The people's speech. | format `int64` |
| `data[].reply_audio_seconds` | integer | yes | The voices' audio. | format `int64` |
| `data[].llm_input_tokens` | integer | yes | Reply model input tokens. | format `int64` |
| `data[].llm_output_tokens` | integer | yes | Reply model output tokens. | format `int64` |
| `data[].llm_thinking_tokens` | integer | yes | Reply model thinking tokens. | format `int64` |
| `data[].api_key_id` | string | no | The API key (with `group_by=api_key`; `null` for sessions without one). | may be null |
| `data[].voice_id` | string | no | The voice (with `group_by=voice`). | may be null |

### 400

Invalid dates or span

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

### 401

Missing or invalid API key

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

### 403

The key lacks `usage:read`

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

### 429

Rate limited

| Header | Type | Description |
| --- | --- | --- |
| `RateLimit-Limit` | integer | Requests allowed at once for this key |
| `RateLimit-Remaining` | integer | Requests left right now |
| `RateLimit-Reset` | integer | Seconds until the limit is full again |
| `Retry-After` | integer | Seconds to wait before retrying |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `error` | object | yes | The error. |  |
| `error.code` | string | yes |  | one of `invalid_request`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `internal`, `not_found`, `conflict`, `unavailable` |
| `error.message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `error.request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `error.details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

---

# The service's health

> GET /v1/health: The service's health.

Source: https://docs.voice.kheelona.com/reference/generated/api/gethealth/

`GET /v1/health`

| Property | Value |
| --- | --- |
| Operation | `getHealth` |
| Scope | none (no API key needed) |

## Responses

### 200

The service's health (cached for 5 s)

| Header | Type | Description |
| --- | --- | --- |
| `X-Request-Id` | string | The request's ID (`req_…`); quote it in reports. |

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `status` | string | yes | The service's state. | one of `ok`, `degraded`, `down` |
| `capacity` | string | yes | Whether a new session would find a slot. Whether a new session would find a slot right now. | one of `available`, `busy` |

---

# Client events

> Client events: sent by the client to Kheelona Voice.

Source: https://docs.voice.kheelona.com/reference/generated/events/client/

Client events: sent by the client to Kheelona Voice, as JSON text frames. Every event has a `type` field.

Every client event also carries:

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `event_id` | string | no | Optional ID chosen by the client: 1–64 characters of `A–Z a–z 0–9 _ -`. | pattern `^[A-Za-z0-9_-]{1,64}$`; default `null`; may be null |

| Event | When |
| --- | --- |
| [`input.interrupt`](#inputinterrupt) | Stop the current reply now, as if the person had started speaking. |
| [`input.text`](#inputtext) | Inject a turn as text, skipping speech recognition. It goes through the safety checks, with the same logging as voice turns. Invisible characters are removed from `text` when it is parsed; text that is empty once cleaned is refused as `invalid_request`. |
| [`input.mute`](#inputmute) | The client mic was muted or unmuted. |
| [`playback.position`](#playbackposition) | WebSocket only: how much of a reply the client actually played when interrupted. |
| [`client.capabilities`](#clientcapabilities) | Optional, sent once after `session.ready`: what the client can do. Capabilities this server doesn't know are ignored. |
| [`session.end`](#sessionend) | End the session normally. |
| [`ping`](#ping) | Application heartbeat; the server answers `pong`. |

## `input.interrupt`

Stop the current reply now, as if the person had started speaking.

No fields besides `type`.

## `input.text`

Inject a turn as text, skipping speech recognition. It goes through the safety checks, with the same logging as voice turns. Invisible characters are removed from `text` when it is parsed; text that is empty once cleaned is refused as `invalid_request`.

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `text` | string | yes | The turn's text (at most 2,000 characters; never stored). | max length 2000 |

## `input.mute`

The client mic was muted or unmuted.

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `muted` | boolean | yes | Whether the mic is muted. |  |

## `playback.position`

WebSocket only: how much of a reply the client actually played when interrupted.

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `reply_id` | [ReplyId](#replyid) | yes | The interrupted reply. | pattern `^rep_[0-9A-HJKMNP-TV-Z]{26}$` |
| `played_ms` | integer | yes | Milliseconds of it the client played. | format `uint32`; min 0 |

## `client.capabilities`

Optional, sent once after `session.ready`: what the client can do. Capabilities this server doesn't know are ignored.

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `playback_pause` | boolean | no | The client pauses and resumes its playback buffer on `reply.paused` and `reply.resumed`. Without it the server never sends those two events. | default `false` |

## `session.end`

End the session normally.

No fields besides `type`.

## `ping`

Application heartbeat; the server answers `pong`.

No fields besides `type`.

## Types

### ReplyId

string, pattern `^rep_[0-9A-HJKMNP-TV-Z]{26}$`.

---

# Server events

> Server events: sent by Kheelona Voice to the client.

Source: https://docs.voice.kheelona.com/reference/generated/events/server/

Server events: sent by Kheelona Voice to the client, as JSON text frames. Every event has a `type` field.

Every server event also carries:

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `event_id` | [EventId](#eventid) | yes | Server-generated ID. | pattern `^evt_[0-9A-HJKMNP-TV-Z]{26}$` |
| `ts` | string | yes | When the event was created: RFC 3339 UTC with milliseconds. | format `date-time` |

| Event | When |
| --- | --- |
| [`session.ready`](#sessionready) | The connection is accepted. |
| [`state`](#state) | The conversation state changed. |
| [`input.speech_started`](#inputspeech_started) | Voice activity detection heard speech. |
| [`input.speech_stopped`](#inputspeech_stopped) | The turn detector decided the turn is complete. |
| [`transcript.final`](#transcriptfinal) | What was heard (live text; never stored). |
| [`reply.started`](#replystarted) | The voice began a reply. |
| [`reply.sentence`](#replysentence) | A sentence's audio started being delivered. `text` has no emotion tags (live text; never stored). |
| [`reply.done`](#replydone) | The reply finished. |
| [`interrupted`](#interrupted) | The person talked over the voice. The client MUST flush queued reply audio immediately. |
| [`reply.paused`](#replypaused) | Only for a client that declared `playback_pause` (`client.capabilities`), and only for a reply it saw start: the voice paused its own playback because the person started to speak. The client SHOULD pause its playback buffer now. Not a listening cue. The pause ends at the first of `reply.resumed`, `interrupted`, `reply.done` or the next `reply.started`. |
| [`reply.resumed`](#replyresumed) | Only after `reply.paused`: the pause ended and the same reply plays on. The client resumes its playback buffer. |
| [`safety`](#safety) | A safety rule fired. Category only, never text. |
| [`session.warning`](#sessionwarning) | A limit is close: 60 s before max duration, 20 s before the idle timeout. |
| [`error`](#error) | A problem. If `fatal`, the session ends next. |
| [`session.ended`](#sessionended) | The last event before the connection closes. |
| [`pong`](#pong) | Answer to `ping`. |

## `session.ready`

The connection is accepted.

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `session_id` | [SessionId](#sessionid) | yes | The session. | pattern `^ses_[0-9A-HJKMNP-TV-Z]{26}$` |
| `voice` | [VoiceInfo](#voiceinfo) | yes | The voice and the version this session pinned. |  |
| `languages` | array of [Lang](#lang) | yes | The session's declared languages (default `["en","hi"]`). |  |
| `audio` | [AudioInfo](#audioinfo) | yes | Audio formats in each direction. |  |
| `limits` | [Limits](#limits) | yes | Duration and idle limits. |  |
| `interruptions` | boolean | yes | Whether the person can interrupt the voice. |  |

## `state`

The conversation state changed.

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `value` | [StateValue](#statevalue) | yes | The new state. | one of `listening`, `thinking`, `speaking` |

## `input.speech_started`

Voice activity detection heard speech.

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `turn` | integer | yes | The turn number. | format `uint32`; min 0 |

## `input.speech_stopped`

The turn detector decided the turn is complete.

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `turn` | integer | yes | The turn number. | format `uint32`; min 0 |

## `transcript.final`

What was heard (live text; never stored).

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `turn` | integer | yes | The turn number. | format `uint32`; min 0 |
| `text` | string | yes | The transcript. |  |
| `language` | [Lang](#lang) | yes | The language it was heard in. | one of `en`, `hi`, `mr`, `bn`, `as`, `gu`, `pa`, `ta`, `te`, `kn`, `ml` |
| `language_beta` | boolean | yes | True for every language that is not supported (beta or alpha; kept for older clients). |  |
| `language_stage` | [LangStage](#langstage) | yes | The language's stage: `supported`, `beta` or `alpha`. | one of `supported`, `beta`, `alpha` |

## `reply.started`

The voice began a reply.

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `turn` | integer | yes | The turn being answered. | format `uint32`; min 0 |
| `reply_id` | [ReplyId](#replyid) | yes | The reply. | pattern `^rep_[0-9A-HJKMNP-TV-Z]{26}$` |

## `reply.sentence`

A sentence's audio started being delivered. `text` has no emotion tags (live text; never stored).

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `reply_id` | [ReplyId](#replyid) | yes | The reply. | pattern `^rep_[0-9A-HJKMNP-TV-Z]{26}$` |
| `index` | integer | yes | The sentence index within the reply (also in the audio frame header). | format `uint16`; min 0; max 65535 |
| `text` | string | yes | The sentence as caption text. |  |
| `emotion` | [Emotion](#emotion) | yes | The emotion it's spoken with. | one of `excited`, `happy`, `giggly`, `calm`, `soft`, `sad`, `surprised` |

## `reply.done`

The reply finished.

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `reply_id` | [ReplyId](#replyid) | yes | The reply. | pattern `^rep_[0-9A-HJKMNP-TV-Z]{26}$` |
| `reason` | [ReplyDoneReason](#replydonereason) | yes | Why it finished. | one of `completed`, `interrupted`, `safety`, `error` |
| `sentences_spoken` | integer | yes | Sentences actually spoken. | format `uint16`; min 0; max 65535 |

## `interrupted`

The person talked over the voice. The client MUST flush queued reply audio immediately.

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `reply_id` | [ReplyId](#replyid) | yes | The interrupted reply. | pattern `^rep_[0-9A-HJKMNP-TV-Z]{26}$` |
| `heard_ms` | integer | yes | Milliseconds of it the person heard. | format `uint32`; min 0 |

## `reply.paused`

Only for a client that declared `playback_pause` (`client.capabilities`), and only for a reply it saw start: the voice paused its own playback because the person started to speak. The client SHOULD pause its playback buffer now. Not a listening cue. The pause ends at the first of `reply.resumed`, `interrupted`, `reply.done` or the next `reply.started`.

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `reply_id` | [ReplyId](#replyid) | yes | The paused reply. | pattern `^rep_[0-9A-HJKMNP-TV-Z]{26}$` |

## `reply.resumed`

Only after `reply.paused`: the pause ended and the same reply plays on. The client resumes its playback buffer.

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `reply_id` | [ReplyId](#replyid) | yes | The resumed reply. | pattern `^rep_[0-9A-HJKMNP-TV-Z]{26}$` |

## `safety`

A safety rule fired. Category only, never text.

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `turn` | integer | yes | The turn. | format `uint32`; min 0 |
| `stage` | [SafetyStage](#safetystage) | yes | Where the rule fired. | one of `input`, `output`, `provider` |
| `category` | string | yes | What kind of content fired, as a stable `snake_case` label: from the classifier `violent`, `illegal`, `sexual`, `pii`, `self_harm`, `unethical`, `political`, `copyright`, `jailbreak`, `unknown`, `unsafe` (no category named); from the blocklists also `profanity`, `hate`, `romance`, `scary`, `dangerous_activity`, `substances`, `personal_data`. A free string: new labels may be added, so treat an unknown one as "other". |  |
| `action` | [SafetyAction](#safetyaction) | yes | What the voice did. | one of `redirected`, `sentence_dropped` |

## `session.warning`

A limit is close: 60 s before max duration, 20 s before the idle timeout.

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `code` | [WarningCode](#warningcode) | yes | Which limit. | one of `max_duration_soon`, `idle_soon` |
| `seconds_left` | integer | yes | Seconds until it's reached. | format `uint32`; min 0 |

## `error`

A problem. If `fatal`, the session ends next.

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `code` | [ErrorCode](#errorcode) | yes | The error code. | one of `invalid_request`, `invalid_audio`, `unauthenticated`, `secret_expired`, `forbidden`, `origin_not_allowed`, `voice_not_found`, `session_not_found`, `capacity_exhausted`, `partner_limit_reached`, `rate_limited`, `provider_unavailable`, `internal`, `not_found`, `conflict`, `unavailable` |
| `message` | string | yes | A human-readable explanation (no conversation content). |  |
| `retryable` | boolean | yes | Whether retrying can help. |  |
| `fatal` | boolean | yes | Whether the session ends next. |  |

## `session.ended`

The last event before the connection closes.

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `reason` | [EndReason](#endreason) | yes | Why it ended. | one of `client_ended`, `partner_ended`, `max_duration`, `idle_timeout`, `disconnected`, `server_shutdown`, `provider_failure`, `suspended`, `service_stopped`, `error` |
| `duration_ms` | integer | yes | Connected time in milliseconds. | format `uint64`; min 0 |
| `turns` | integer | yes | Turns in the session. | format `uint32`; min 0 |

## `pong`

Answer to `ping`.

No fields besides `type`.

## Types

### AudioFormat

PCM16 little-endian mono at one of three sample rates.

| Value | Description |
| --- | --- |
| `pcm16_16k` | 16,000 Hz (the input default). |
| `pcm16_24k` | 24,000 Hz (the output default). |
| `pcm16_48k` | 48,000 Hz. |

### AudioInfo

Audio formats chosen at session creation.

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `input_format` | [AudioFormat](#audioformat) | yes | Client → server audio. | one of `pcm16_16k`, `pcm16_24k`, `pcm16_48k` |
| `output_format` | [AudioFormat](#audioformat) | yes | Server → client audio. | one of `pcm16_16k`, `pcm16_24k`, `pcm16_48k` |

### Emotion

The emotion a sentence is spoken with.

| Value | Description |
| --- | --- |
| `excited` | Excited |
| `happy` | Happy: the first sentence's default when the model gives no tag. |
| `giggly` | Giggly: only when the person is laughing. |
| `calm` | Calm |
| `soft` | Soft |
| `sad` | Sad |
| `surprised` | Surprised |

### EndReason

Why a session ended.

| Value | Description |
| --- | --- |
| `client_ended` | The client sent `session.end`. |
| `partner_ended` | The partner called `POST /v1/sessions/{id}/end`. |
| `max_duration` | The maximum duration was reached. |
| `idle_timeout` | Nobody spoke for the idle timeout. |
| `disconnected` | The connection dropped. |
| `server_shutdown` | The server is shutting down. |
| `provider_failure` | Three provider failures in a row. |
| `suspended` | The partner was suspended mid-session. |
| `service_stopped` | The voice service was switched off. |
| `error` | Another error. |

### ErrorCode

A machine-readable error code.

| Value | Description |
| --- | --- |
| `invalid_request` | Fix the request. |
| `invalid_audio` | A malformed audio frame; the frame is dropped. |
| `unauthenticated` | Missing or invalid API key or client secret. |
| `secret_expired` | The client secret is older than 60 s or already used: create a new session. |
| `forbidden` | The key lacks the scope, or the partner is suspended. |
| `origin_not_allowed` | The web origin isn't allowed for this partner. |
| `voice_not_found` | Unknown or retired voice, or not enabled for this partner. |
| `session_not_found` | No such session. |
| `capacity_exhausted` | The whole service is full; retry after the hinted seconds. |
| `partner_limit_reached` | The partner is at its maximum concurrent sessions. |
| `rate_limited` | Too many API calls. |
| `provider_unavailable` | A processing step failed. Non-fatal for one turn; fatal after 3 failures in a row. |
| `internal` | Our bug; the request ID finds it. |
| `not_found` | REST only: the resource doesn't exist, or isn't visible to your API key (a resource that belongs to someone else is `not_found`, so its existence isn't revealed). |
| `conflict` | REST only: the request conflicts with the current state of the resource; change the request and retry. |
| `unavailable` | REST only: the API is temporarily unavailable; retry later. |

### EventId

string, pattern `^evt_[0-9A-HJKMNP-TV-Z]{26}$`.

### Lang

A supported language, as its ISO 639-1 code. Odia is not supported.

| Value | Description |
| --- | --- |
| `en` | English |
| `hi` | Hindi |
| `mr` | Marathi (beta) |
| `bn` | Bengali (beta) |
| `as` | Assamese (alpha) |
| `gu` | Gujarati |
| `pa` | Punjabi (beta) |
| `ta` | Tamil |
| `te` | Telugu (beta) |
| `kn` | Kannada (beta) |
| `ml` | Malayalam (alpha) |

### LangStage

How far a language is tested: supported, beta or alpha.

| Value | Description |
| --- | --- |
| `supported` | Tested end to end. |
| `beta` | Works, with more recognition mistakes. |
| `alpha` | Early: spoken turns are often not recognised. |

### Limits

Session limits.

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `max_duration_s` | integer | yes | Maximum session length in seconds. | format `uint32`; min 0 |
| `idle_timeout_s` | integer | yes | Idle timeout in seconds. | format `uint32`; min 0 |

### ReplyDoneReason

Why a reply finished.

| Value | Description |
| --- | --- |
| `completed` | Every sentence was spoken. |
| `interrupted` | The person talked over it. |
| `safety` | A safety rule ended it. |
| `error` | A provider failed. |

### ReplyId

string, pattern `^rep_[0-9A-HJKMNP-TV-Z]{26}$`.

### SafetyAction

What the voice did when a safety rule fired.

| Value | Description |
| --- | --- |
| `redirected` | Spoke the voice's fallback line instead. |
| `sentence_dropped` | Dropped the sentence. |

### SafetyStage

Where a safety rule fired.

| Value | Description |
| --- | --- |
| `input` | On the person's turn, before the reply model. |
| `output` | On a reply sentence, before speech. |
| `provider` | Reported by a provider. |

### SessionId

string, pattern `^ses_[0-9A-HJKMNP-TV-Z]{26}$`.

### StateValue

Conversation state.

| Value | Description |
| --- | --- |
| `listening` | Waiting for the person. |
| `thinking` | From the end of the person's turn until the first reply audio. |
| `speaking` | Reply audio is being delivered. |

### VoiceInfo

The voice a session uses.

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `id` | [VoiceSlug](#voiceslug) | yes | Public voice ID, e.g. `lumi-dragon`. | pattern `^[a-z][a-z0-9-]{2,39}$` |
| `version` | integer | yes | The pinned version (from 1). | format `uint32`; min 0 |
| `name` | string | yes | Display name. |  |

### VoiceSlug

string, pattern `^[a-z][a-z0-9-]{2,39}$`.

### WarningCode

Which limit a `session.warning` is about.

| Value | Description |
| --- | --- |
| `max_duration_soon` | Maximum duration in 60 s. |
| `idle_soon` | Idle timeout in 20 s. |

---

# Errors

> Every error code, what it means and where it occurs.

Source: https://docs.voice.kheelona.com/reference/generated/errors/

Errors from the REST API are an `ErrorResponse`: `{ "error": { "code", "message", "request_id", "details" } }`. On the realtime connection an `error` event carries the same `code`, with `fatal` and `retryable` flags. Quote the `request_id` when you report a problem.

## Error codes

| Code | Meaning | REST (/v1) | Realtime |
| --- | --- | --- | --- |
| `invalid_request` | Fix the request. | yes | yes |
| `invalid_audio` | A malformed audio frame; the frame is dropped. | no | yes |
| `unauthenticated` | Missing or invalid API key or client secret. | yes | yes |
| `secret_expired` | The client secret is older than 60 s or already used: create a new session. | yes | yes |
| `forbidden` | The key lacks the scope, or the partner is suspended. | yes | yes |
| `origin_not_allowed` | The web origin isn't allowed for this partner. | yes | yes |
| `voice_not_found` | Unknown or retired voice, or not enabled for this partner. | yes | yes |
| `session_not_found` | No such session. | yes | yes |
| `capacity_exhausted` | The whole service is full; retry after the hinted seconds. | yes | yes |
| `partner_limit_reached` | The partner is at its maximum concurrent sessions. | yes | yes |
| `rate_limited` | Too many API calls. | yes | yes |
| `provider_unavailable` | A processing step failed. Non-fatal for one turn; fatal after 3 failures in a row. | no | yes |
| `internal` | Our bug; the request ID finds it. | yes | yes |
| `not_found` | The resource doesn't exist, or isn't visible to your API key (a resource that belongs to someone else is `not_found`, so its existence isn't revealed). | yes | no |
| `conflict` | The request conflicts with the current state of the resource; change the request and retry. | yes | no |
| `unavailable` | The API is temporarily unavailable; retry later. | yes | no |

## The error object

One error: a stable code, a short message and the request ID.

| Field | Type | Required | Description | Rules |
| --- | --- | --- | --- | --- |
| `code` | string | yes | One of the codes above. |  |
| `message` | string | yes | A short, fixed description. Never quotes the request. |  |
| `request_id` | string | yes | The request's ID (also in the `X-Request-Id` header); quote it when reporting a problem. | pattern `^req_[0-9A-HJKMNP-TV-Z]{26}$` |
| `details` | any | no | Structured detail, e.g. `{"field": "languages[1]"}` or `{"reason": "quota_exhausted"}`. |  |

---

# Audio formats

> The audio formats and the binary frame layouts of the realtime connection.

Source: https://docs.voice.kheelona.com/reference/generated/audio/

Audio is PCM, 16-bit little-endian, mono. Pick the formats when you create the session; the `session.ready` event confirms them.

## Formats

| Value | Description |
| --- | --- |
| `pcm16_16k` | 16,000 Hz (the input default). |
| `pcm16_24k` | 24,000 Hz (the output default). |
| `pcm16_48k` | 48,000 Hz. |

Defaults: input `pcm16_16k`, output `pcm16_24k`. Set them with `audio.input_format` and `audio.output_format` in the request that creates the session.

## Binary frames

JSON events travel as WebSocket text frames. Audio travels as binary frames.

### Input audio (client to server)

PCM16 little-endian mono in the session's input format (default 16 kHz), 10–500 ms per frame, no header.

### Reply audio (server to client)

8-byte header (kind u8 = 0x01, flags u8: bit0 first frame of a sentence, bit1 last frame of the reply; sentence u16 LE; reply_seq u32 LE) followed by PCM16 LE samples in the output format (default 24 kHz). The last frame of a reply may carry no samples.

---

# Supported languages

> The languages a session can declare, and which of them are supported, beta or alpha.

Source: https://docs.voice.kheelona.com/reference/languages/

Declare the languages of a session in `languages` when you create it. The list below is what `GET /v1/languages`
returns. The voice replies in whichever declared language the person speaks.

| Code | Language | Status | Notes |
|---|---|---|---|
| `en` | English | Supported |  |
| `hi` | Hindi | Supported |  |
| `mr` | Marathi | Beta | Beta: recognition makes more mistakes than in Hindi. Test it with your users. |
| `bn` | Bengali | Beta | Beta: recognition makes more mistakes. Test it with your users. |
| `as` | Assamese | Alpha | Alpha: spoken Assamese is not recognised today: use typed turns. The voice's Assamese pronunciation is limited. |
| `gu` | Gujarati | Supported |  |
| `pa` | Punjabi | Beta | Beta: expect mistakes in recognition and pronunciation. A short turn can be taken for English if the session also declares English. |
| `ta` | Tamil | Supported |  |
| `te` | Telugu | Beta | Beta: recognition makes more mistakes, and the safety checks stop a few harmless sentences, so a child may get the gentle "talk to a grown-up" reply instead of an answer. |
| `kn` | Kannada | Beta | Beta: expect mistakes in recognition and pronunciation, and the safety checks stop a few harmless sentences. |
| `ml` | Malayalam | Alpha | Alpha: spoken Malayalam is often not recognised, so a turn may come back with no transcript. Typed turns work. The safety checks stop a few harmless sentences. |

Sessions default to `en, hi` when you declare no languages.

Beta and alpha languages are accepted, with a warning in the response. In the alpha languages, Malayalam and
Assamese, spoken turns are often not recognised: use typed turns. See
[Languages](https://docs.voice.kheelona.com/concepts/languages.md#supported-beta-and-alpha). Odia is not supported.

---

# KheelonaVoice

Source: https://docs.voice.kheelona.com/reference/sdk/classes/kheelonavoice/

The entry point of the SDK. Call [KheelonaVoice.connect](https://docs.voice.kheelona.com/reference/sdk/classes/kheelonavoice.md#connect) to start a session. The Browser SDK guide
(`/guides/browser-sdk/`) walks through it with code.

## Constructors

### Constructor

> **new KheelonaVoice**(): `KheelonaVoice`

#### Returns

`KheelonaVoice`

## Methods

### connect()

> `static` **connect**(`o`): `Promise`\<[`VoiceSession`](https://docs.voice.kheelona.com/reference/sdk/interfaces/voicesession.md)\>

Connects a session. Call it from a user gesture (a click): the audio context and the microphone start here.
Rejects with a `KheelonaVoiceError`. After a failed `connect()`, create a new session (a new client secret) before
trying again: the secret may be spent or expired. The exception is `capacity_exhausted` with `retryAfterMs`: retry
the same secret after that wait while it is under 60 s old. A busy-server answer with a wait of 2 s or less (one
server at its session limit) is retried here, up to 3 times, before `connect()` rejects. `error.cause` may hold the browser's own exception,
whose text can quote SDP lines (ephemeral ICE credentials): don't log it verbatim.

#### Parameters

##### o

[`ConnectOptions`](https://docs.voice.kheelona.com/reference/sdk/interfaces/connectoptions.md)

The session id and client secret from your server, and optional settings.

#### Returns

`Promise`\<[`VoiceSession`](https://docs.voice.kheelona.com/reference/sdk/interfaces/voicesession.md)\>

---

# KheelonaVoiceError

Source: https://docs.voice.kheelona.com/reference/sdk/classes/kheelonavoiceerror/

Every failure `KheelonaVoice.connect` rejects with. After a failed `connect()`, create a new session (a new client
secret) before trying again.

## Extends

- `Error`

## Constructors

### Constructor

> **new KheelonaVoiceError**(`code`, `message`, `options?`): `KheelonaVoiceError`

#### Parameters

##### code

`"invalid_request"` \| `"invalid_audio"` \| `"unauthenticated"` \| `"secret_expired"` \| `"forbidden"` \| `"origin_not_allowed"` \| `"voice_not_found"` \| `"session_not_found"` \| `"capacity_exhausted"` \| `"partner_limit_reached"` \| `"rate_limited"` \| `"provider_unavailable"` \| `"internal"` \| `"not_found"` \| `"conflict"` \| `"unavailable"` \| [`SdkErrorCode`](https://docs.voice.kheelona.com/reference/sdk/type-aliases/sdkerrorcode.md)

The error code.

##### message

`string`

A readable description. Log it freely.

##### options?

`cause` keeps the browser's own error as `Error.cause`: don't log it verbatim, its text can quote
connection details.

###### cause?

`unknown`

###### retryable?

`boolean`

###### retryAfterMs?

`number`

#### Returns

`KheelonaVoiceError`

#### Overrides

`Error.constructor`

## Properties

### code

> **code**: `"invalid_request"` \| `"invalid_audio"` \| `"unauthenticated"` \| `"secret_expired"` \| `"forbidden"` \| `"origin_not_allowed"` \| `"voice_not_found"` \| `"session_not_found"` \| `"capacity_exhausted"` \| `"partner_limit_reached"` \| `"rate_limited"` \| `"provider_unavailable"` \| `"internal"` \| `"not_found"` \| `"conflict"` \| `"unavailable"` \| [`SdkErrorCode`](https://docs.voice.kheelona.com/reference/sdk/type-aliases/sdkerrorcode.md)

What went wrong. Branch on this, not on `message`.

***

### name

> **name**: `string` = `'KheelonaVoiceError'`

#### Overrides

`Error.name`

***

### retryable

> **retryable**: `boolean`

Retrying the same call later can succeed.

***

### retryAfterMs?

> `optional` **retryAfterMs?**: `number`

How long to wait before retrying, in milliseconds, when the service said so.

---

# ConnectOptions

Source: https://docs.voice.kheelona.com/reference/sdk/interfaces/connectoptions/

Options of `KheelonaVoice.connect`.

## Properties

### clientSecret

> **clientSecret**: `string`

The short-lived secret your server got from `POST /v1/sessions`. Never your API key.

***

### endpoints?

> `optional` **endpoints?**: `object`

Where to connect, if your server's `POST /v1/sessions` answer gave addresses (`connect.webrtc_url` and
`connect.websocket_url`). Defaults to the production service.

#### webrtcUrl?

> `optional` **webrtcUrl?**: `string`

#### websocketUrl?

> `optional` **websocketUrl?**: `string`

***

### microphone?

> `optional` **microphone?**: `false` \| \{ `deviceId?`: `string`; \}

`false` for typed turns only (no microphone is opened); `deviceId` picks the input device.

***

### sessionId

> **sessionId**: `string`

The session id returned alongside the secret.

***

### transport?

> `optional` **transport?**: `"webrtc"` \| `"websocket"` \| `"auto"`

Default `auto`: WebRTC, falling back to the WebSocket when the network refuses WebRTC.

***

### workletBaseUrl?

> `optional` **workletBaseUrl?**: `string`

Where the AudioWorklet modules are served from, for a strict CSP.

---

# SessionErrorInfo

Source: https://docs.voice.kheelona.com/reference/sdk/interfaces/sessionerrorinfo/

The payload of the SDK's `error` event.

## Properties

### code

> **code**: `"invalid_request"` \| `"invalid_audio"` \| `"unauthenticated"` \| `"secret_expired"` \| `"forbidden"` \| `"origin_not_allowed"` \| `"voice_not_found"` \| `"session_not_found"` \| `"capacity_exhausted"` \| `"partner_limit_reached"` \| `"rate_limited"` \| `"provider_unavailable"` \| `"internal"` \| `"not_found"` \| `"conflict"` \| `"unavailable"` \| [`SdkErrorCode`](https://docs.voice.kheelona.com/reference/sdk/type-aliases/sdkerrorcode.md)

***

### fatal

> **fatal**: `boolean`

***

### message

> **message**: `string`

***

### retryable

> **retryable**: `boolean`

***

### retryAfterMs?

> `optional` **retryAfterMs?**: `number`

---

# SessionEvents

Source: https://docs.voice.kheelona.com/reference/sdk/interfaces/sessionevents/

What `session.on(name, fn)` delivers.

## Properties

### ended

> **ended**: `object`

The session is over. Fires once. `reason` is one of `client_ended`, `partner_ended`, `max_duration`,
`idle_timeout`, `disconnected`, `server_shutdown`, `provider_failure`, `suspended`, `service_stopped` (the voice
service was switched off; start a new session once it is back) or `error`.

#### durationMs

> **durationMs**: `number`

#### reason

> **reason**: `string`

#### turns

> **turns**: `number`

***

### error

> **error**: [`SessionErrorInfo`](https://docs.voice.kheelona.com/reference/sdk/interfaces/sessionerrorinfo.md)

A problem. `fatal: false` means the session goes on (for example `audio_unavailable`).

***

### event

> **event**: [`ServerEvent`](https://docs.voice.kheelona.com/reference/sdk/type-aliases/serverevent.md)

Every event from the service, as parsed, for logging or for events the SDK doesn't map.

***

### interrupted

> **interrupted**: `object`

The reply was cut off (the person spoke, or `interrupt()` was called). `heardMs` is how much was heard.

#### heardMs

> **heardMs**: `number`

#### replyId

> **replyId**: `string`

***

### replySentence

> **replySentence**: `object`

One sentence of the voice's reply, with its emotion, as it is spoken.

#### emotion

> **emotion**: `string`

#### index

> **index**: `number`

#### replyId

> **replyId**: `string`

#### text

> **text**: `string`

***

### safety

> **safety**: `object`

The safety layer acted on a turn: the stage, the category (a level, never the text) and the action taken.

#### action

> **action**: `string`

#### category

> **category**: `string`

#### stage

> **stage**: `string`

#### turn

> **turn**: `number`

***

### state

> **state**: [`SessionState`](https://docs.voice.kheelona.com/reference/sdk/type-aliases/sessionstate.md)

The session's state changed.

***

### transcript

> **transcript**: `object`

What the person said, once the service has understood the turn. `languageStage` is the language's stage; `languageBeta` is true for any
language that is not supported (beta or alpha).

#### language

> **language**: `string`

#### languageBeta

> **languageBeta**: `boolean`

#### languageStage

> **languageStage**: [`LanguageStage`](https://docs.voice.kheelona.com/reference/sdk/type-aliases/languagestage.md)

#### text

> **text**: `string`

#### turn

> **turn**: `number`

***

### warning

> **warning**: `object`

A heads-up before the session ends, for example `secondsLeft` before the time limit.

#### code

> **code**: `string`

#### secondsLeft

> **secondsLeft**: `number`

---

# SessionReadyEvent

Source: https://docs.voice.kheelona.com/reference/sdk/interfaces/sessionreadyevent/

The service's first event: the session is live.

## Properties

### audio

> **audio**: `object`

#### input\_format

> **input\_format**: `string`

#### output\_format

> **output\_format**: `string`

***

### interruptions

> **interruptions**: `boolean`

***

### languages

> **languages**: `string`[]

***

### limits

> **limits**: `object`

#### idle\_timeout\_s

> **idle\_timeout\_s**: `number`

#### max\_duration\_s

> **max\_duration\_s**: `number`

***

### session\_id

> **session\_id**: `string`

***

### type

> **type**: `"session.ready"`

***

### voice

> **voice**: `object`

#### id

> **id**: `string`

#### name

> **name**: `string`

#### version

> **version**: `number`

---

# VoiceSession

Source: https://docs.voice.kheelona.com/reference/sdk/interfaces/voicesession/

A live voice session.

## Properties

### sessionId

> `readonly` **sessionId**: `string`

The session id.

***

### state

> `readonly` **state**: [`SessionState`](https://docs.voice.kheelona.com/reference/sdk/type-aliases/sessionstate.md)

The current state. `listening` once `connect()` has resolved.

***

### transport

> `readonly` **transport**: `"webrtc"` \| `"websocket"`

The transport in use: `webrtc`, or `websocket` after the fallback.

## Methods

### end()

> **end**(): `Promise`\<`void`\>

Ends the session and releases the microphone.

#### Returns

`Promise`\<`void`\>

***

### interrupt()

> **interrupt**(): `void`

Stops the reply now, as if the person had started speaking.

#### Returns

`void`

***

### levels()

> **levels**(): `object`

Loudness, 0..1, for meters.

#### Returns

`object`

##### input

> **input**: `number`

##### output

> **output**: `number`

***

### mute()

> **mute**(`muted`): `void`

Mutes or unmutes the microphone.

#### Parameters

##### muted

`boolean`

#### Returns

`void`

***

### on()

> **on**\<`K`\>(`name`, `fn`): () => `void`

Subscribes; the returned function unsubscribes. A new `state` listener first gets the current state, once, in a
microtask after `on()` returns; then every change. Events that arrived while `connect()` was running are
delivered in a task after it resolves, so listeners attached right after `await connect()` miss none of them.

#### Type Parameters

##### K

`K` *extends* keyof [`SessionEvents`](https://docs.voice.kheelona.com/reference/sdk/interfaces/sessionevents.md)

#### Parameters

##### name

`K`

##### fn

(`e`) => `void`

#### Returns

() => `void`

***

### resumeAudio()

> **resumeAudio**(): `void`

After a non-fatal `audio_unavailable` (the browser refused to play the voice): call it
synchronously in a click or tap handler. It resumes the audio and retries playing the voice inside that gesture;
the conversation goes on. If the browser still refuses, `audio_unavailable` is reported again. Does nothing after
`ended`.

#### Returns

`void`

***

### sendText()

> **sendText**(`text`): `void`

Sends a typed turn (at most 2,000 characters; longer throws `RangeError`).

#### Parameters

##### text

`string`

#### Returns

`void`

---

# ClientEvent

Source: https://docs.voice.kheelona.com/reference/sdk/type-aliases/clientevent/

> **ClientEvent** = \{ `type`: `"input.interrupt"`; \} \| \{ `text`: `string`; `type`: `"input.text"`; \} \| \{ `muted`: `boolean`; `type`: `"input.mute"`; \} \| \{ `playback_pause`: `boolean`; `type`: `"client.capabilities"`; \} \| \{ `played_ms`: `number`; `reply_id`: `string`; `type`: `"playback.position"`; \} \| \{ `type`: `"session.end"`; \} \| \{ `type`: `"ping"`; \}

Events the SDK sends to the service.

---

# Emotion

Source: https://docs.voice.kheelona.com/reference/sdk/type-aliases/emotion/

> **Emotion** = `string`

The emotion tag on a reply sentence, for example `neutral`. An open set: new tags can appear.

---

# ErrorCode

Source: https://docs.voice.kheelona.com/reference/sdk/type-aliases/errorcode/

> **ErrorCode** = *typeof* [`ERROR_CODES`](https://docs.voice.kheelona.com/reference/sdk/variables/error_codes.md)\[`number`\]

A code the service can send.

---

# LanguageStage

Source: https://docs.voice.kheelona.com/reference/sdk/type-aliases/languagestage/

> **LanguageStage** = `"supported"` \| `"beta"` \| `"alpha"`

How far a language is tested: `supported`, `beta` or `alpha`.

---

# SdkErrorCode

Source: https://docs.voice.kheelona.com/reference/sdk/type-aliases/sdkerrorcode/

> **SdkErrorCode** = `"microphone_unavailable"` \| `"audio_unavailable"` \| `"network"` \| `"transport_failed"`

Codes the SDK raises itself, never sent by the service:
- `microphone_unavailable`: the microphone is missing, blocked or in use.
- `audio_unavailable`: the browser refused to play the voice (autoplay). Call `VoiceSession.resumeAudio()` from a
  click handler.
- `network`: the network failed before a connection was made.
- `transport_failed`: neither WebRTC nor the WebSocket could be set up.

---

# ServerEvent

Source: https://docs.voice.kheelona.com/reference/sdk/type-aliases/serverevent/

> **ServerEvent** = [`SessionReadyEvent`](https://docs.voice.kheelona.com/reference/sdk/interfaces/sessionreadyevent.md) \| \{ `type`: `"state"`; `value`: `"listening"` \| `"thinking"` \| `"speaking"`; \} \| \{ `turn`: `number`; `type`: `"input.speech_started"`; \} \| \{ `turn`: `number`; `type`: `"input.speech_stopped"`; \} \| \{ `language`: `string`; `language_beta`: `boolean`; `language_stage`: [`LanguageStage`](https://docs.voice.kheelona.com/reference/sdk/type-aliases/languagestage.md); `text`: `string`; `turn`: `number`; `type`: `"transcript.final"`; \} \| \{ `reply_id`: `string`; `turn`: `number`; `type`: `"reply.started"`; \} \| \{ `emotion`: [`Emotion`](https://docs.voice.kheelona.com/reference/sdk/type-aliases/emotion.md); `index`: `number`; `reply_id`: `string`; `text`: `string`; `type`: `"reply.sentence"`; \} \| \{ `reason`: `string`; `reply_id`: `string`; `sentences_spoken`: `number`; `type`: `"reply.done"`; \} \| \{ `reply_id`: `string`; `type`: `"reply.paused"`; \} \| \{ `reply_id`: `string`; `type`: `"reply.resumed"`; \} \| \{ `heard_ms`: `number`; `reply_id`: `string`; `type`: `"interrupted"`; \} \| \{ `action`: `string`; `category`: `string`; `stage`: `string`; `turn`: `number`; `type`: `"safety"`; \} \| \{ `code`: `string`; `seconds_left`: `number`; `type`: `"session.warning"`; \} \| \{ `code`: [`ErrorCode`](https://docs.voice.kheelona.com/reference/sdk/type-aliases/errorcode.md); `fatal`: `boolean`; `message`: `string`; `retryable`: `boolean`; `type`: `"error"`; \} \| \{ `duration_ms`: `number`; `reason`: `string`; `turns`: `number`; `type`: `"session.ended"`; \} \| \{ `type`: `"pong"`; \}

Events the service sends (known fields only).

---

# SessionState

Source: https://docs.voice.kheelona.com/reference/sdk/type-aliases/sessionstate/

> **SessionState** = `"connecting"` \| `"listening"` \| `"thinking"` \| `"speaking"` \| `"ended"`

What the session is doing. `connecting` until the service's first state.

---

# ERROR_CODES

Source: https://docs.voice.kheelona.com/reference/sdk/variables/error_codes/

> `const` **ERROR\_CODES**: readonly \[`"invalid_request"`, `"invalid_audio"`, `"unauthenticated"`, `"secret_expired"`, `"forbidden"`, `"origin_not_allowed"`, `"voice_not_found"`, `"session_not_found"`, `"capacity_exhausted"`, `"partner_limit_reached"`, `"rate_limited"`, `"provider_unavailable"`, `"internal"`, `"not_found"`, `"conflict"`, `"unavailable"`\]

The error codes the service can send. The list can grow: treat a code you don't know as `internal`.

---

# Data and privacy

> What Kheelona Voice keeps and what it never keeps, how long, where it is processed, and what you are responsible for.

Source: https://docs.voice.kheelona.com/policies/data-and-privacy/

Kheelona Voice is built for conversations with children, so it keeps as little as it can. This page lists what is kept
about the people who talk to the voices, and what your own team's accounts add.

## What Kheelona never keeps

**No audio and no conversation text is ever stored or logged.** That covers:

- what the person says, as sound and as text;
- what the voice says, as sound and as text;
- the text you send with `input.text`;
- the `instructions` you send for a session, in readable form.

Kheelona holds these only while the conversation is happening. Nothing readable is written to a database, a file or a
log, and the code that handles them is built to stop that happening by accident.

`instructions` are the one exception to "nothing is written". They wait for the connection to start in a short-lived
hand-off record, **encrypted** with a key Kheelona does not keep. The record is deleted within about two minutes. A copy
of it can remain, still encrypted and unreadable, in a database backup for up to 14 days.

## What Kheelona keeps

For each session, Kheelona keeps a record made of counts and times instead of words:

- **Who and how:** your partner and API key, your `end_user_ref` and `metadata`, the client kind, the transport, the
  voice and its version, the declared languages and the languages heard, and the session's settings, including whether you
  sent `instructions` and how long they were.
- **When and how long:** the times the session was created, connected and ended, and its duration.
- **How it went:** the number of turns and interruptions, response times, audio seconds, token counts, and counts of safety
  actions (by category) and errors (by code).
- **Each turn:** its timings, the language, a voice-energy level (low, medium or high), whether the speaker was laughing
  or interrupted, and any safety action with its category. No words.
- **Why it ended:** the reason.
- **The connection:** the IP address, the user agent and the web origin, if any, and the country and region the IP address
  comes from (see [Location](#location)).

A connection that is refused (for example a wrong or used client secret) is also recorded, with the IP address, the user
agent and the reason. Usage totals per day are kept for your invoices and your console.

### Location

Each session record holds the **country and region** of the connecting IP address, for example `IN` and `IN-KA`. They are
looked up in an offline database when the session is created and again when it connects, so no request leaves Kheelona to
find them and nothing about the person is sent to anyone. It is never an exact location: no city, no coordinates, only the
country and, where the database has one, the region. Regions are recorded for India only for now, so for a session
from elsewhere only the country is kept. An address that belongs to no place, such as a private network, has
none. The database is credited on the [Credits](https://docs.voice.kheelona.com/policies/credits.md) page.

With **server-to-server** sessions, where your server (or a toy's relay) creates and connects the session, the address is
your server's, so the location recorded is your server's, not the end user's.

The firewall in front of the service also logs each request with the country our cloud provider sees it coming from. See
the table below.

## How long it is kept

| What | What it holds | How long |
|---|---|---|
| Session and turn records | The counts and times above | 12 months. Whole months are removed at a time, so a record can stay about a month longer. |
| Country and region of a session | Two codes, from the IP address (see Location) | As long as the session record: 12 months, as above. |
| IP addresses in session and connection records | The address of the device | About 30 days in full. Within a day after that they are shortened to a `/24` (IPv4) or `/48` (IPv6) network: three parts or groups, not a single device. |
| Refused connections | IP address, user agent, reason. No content. | 12 months, with the IP shortened as above. |
| Firewall and network logs | IP address, user agent, country, the address requested. No content. | 30 days. |
| Daily usage totals | Counts per partner per day | 7 years. |
| Database backups | A copy of the records above | 14 days. A deleted record can stay in a backup that long. |
| Operational logs, traces and metrics | Timings, counts and labels. No conversation content. | Logs 30 days, metrics 15 days, traces 7 days. |

You can read your session records with `GET /v1/sessions` and `GET /v1/sessions/{id}`, and in your console. See
[Sessions](https://docs.voice.kheelona.com/concepts/sessions.md#what-kheelona-stores).

## Where it is processed

Processing happens in **India**: the voice, speech recognition, the safety checks, and the database, with one exception.

The step that writes the reply runs in **a US region of our cloud provider**, or, for voices set up to write replies in India, in the provider's
**Mumbai region in India**. If the service in India is unavailable, the reply is written in the US instead. To write a reply,
these are sent to whichever of those regions writes it:

- the text of the conversation so far, including words said while the voice is speaking and anything else the microphone
  picks up, such as someone else in the room;
- a short note on how each spoken turn sounded, for example its energy and whether the speaker was laughing (never the
  audio);
- your `instructions`, including any personal detail you put in them, such as a child's first name.

The provider states that this service does not store these and does not use them to train models. It returns the reply
text, which is spoken in India. A reply written in a US region adds a little time. A voice set up to write replies in India
can start a little sooner or later than the others, depending on how it is set up. See [Latency](https://docs.voice.kheelona.com/concepts/latency.md).

## What you are responsible for

- **Consent.** Get the consent you need from your users, including a parent's approval for children, and tell them that
  conversations are handled by AI services.
- **`instructions`.** Put in only what the voice needs. They are processed in the region above, so for a voice that uses a US
  region, or whose reply falls back to the US because the service in India is unavailable, a first name or an age band
  leaves India.
- **`end_user_ref`.** Use an opaque ID of your own, such as a random string. Never a name, an email address or a phone number.
- **`metadata`.** It is stored, so keep personal data out of it.
- **What you receive.** Kheelona keeps no conversation, so the captions and text you receive are the only copy. What you
  store, and for how long, is your responsibility. See [Captions](https://docs.voice.kheelona.com/guides/captions.md).
- **Your own logs.** Keep conversation text and `instructions` out of your server logs.

## Team accounts (console users)

The people on your team who sign in to the console are not end users, and a different record applies to them:

- their sign-in details, including the email address they sign in with;
- an audit log of what they and your API keys do in the console and the API (what changed, when, from which IP address
  and browser), kept **24 months**, with the IP address in full;
- request logs of the documentation and console websites, which are served worldwide, kept 30 days in a US region.

## Questions

For anything this page does not answer, or to ask about your records, write to the Kheelona team at
[apoorva@kheelona.com](mailto:apoorva@kheelona.com).

---

# Acceptable use

> What you may and may not do with Kheelona Voice. These rules protect the children who talk to the voices.

Source: https://docs.voice.kheelona.com/policies/acceptable-use/

Kheelona voices talk with children. These rules keep that safe. They apply to everything you build on Kheelona Voice.

## What you agree to

- **Use it for children's safety and wellbeing, never against it.** Do not build anything that frightens, pressures,
  deceives or manipulates a child, or that tries to learn private details about a child or their family.
- **Leave the safety checks alone.** Do not try to get around, switch off or trick them, and do not ask a voice to say
  what it is not allowed to say. Do not send text meant to make a voice behave unsafely, through `instructions`, typed
  text or speech.
- **Keep the law.** Use Kheelona Voice only for lawful purposes, and follow the laws on children's data and consent in
  every place your users live.
- **Do not pretend to be someone.** Do not use a voice to imitate a real person, and do not let a user believe they are
  talking to a real person or to someone they are not.
- **Be honest with users.** Tell them, and their parents, that they are talking to an AI voice and that conversations
  are handled by AI services.
- **Keep your credentials safe.** Your API key stays on your server. Do not share it, and do not put it in a web page
  or an app. See [Security](https://docs.voice.kheelona.com/guides/security.md).
- **Respect the limits.** Do not work around session length, idle timeouts, rate limits or capacity. When Kheelona is
  busy, wait and try again. See [Limits](https://docs.voice.kheelona.com/concepts/limits.md).
- **Do not misuse the service.** Do not copy, resell or reverse-engineer it, and do not use a voice to advertise to or sell to children. If you find a weakness, tell the Kheelona team and do not test it further.
- **Look after your users.** If you use the `safety` event to protect your users, make sure someone reads it and knows
  what to do.

## What happens if the rules are broken

Kheelona may limit or suspend access to protect children, with a warning when that is possible, and without one when it
is not.

## Questions

If you are not sure whether something is allowed, ask the Kheelona team before you build it.

---

# Versioning

> How the API and the realtime protocol change, and what Kheelona promises not to break.

Source: https://docs.voice.kheelona.com/policies/versioning/

Kheelona Voice has two versioned surfaces. Both are at version 1.

| Surface | Version | Where you see it |
|---|---|---|
| The REST API | `v1` | The path: `/v1/sessions`. |
| The realtime protocol | `kheelona.v1` | The WebSocket subprotocol: `kheelona.v1`. |

## What is not a breaking change

Kheelona may add these at any time, without a new version:

- new endpoints, and new optional request fields;
- new fields in responses and in realtime events;
- new event types;
- new error codes;
- new voices, languages and options.

**Your client must ignore what it does not know:** unknown fields, unknown event types and unknown error codes. Treat an
unknown error code by its HTTP status and its `fatal` flag. The other direction is strict: `POST /v1/sessions` refuses
fields it does not know, so send only documented fields. See [Errors and retries](https://docs.voice.kheelona.com/guides/errors-and-retries.md).

## What is a breaking change

Removing or renaming anything, or changing what something means, is a breaking change. It only ever ships as a new
version: `v2` for the API, `kheelona.v2` for the protocol.

## How long the old version lasts

When a new version arrives, both versions are supported side by side for **at least six months**. The
[changelog](https://docs.voice.kheelona.com/policies/changelog.md) announces it.

## The browser SDK

The browser SDK has its own version number, in the file name of the package you received (`kheelona-voice-<version>.tgz`). It follows semantic versioning. See the [browser SDK guide](https://docs.voice.kheelona.com/guides/browser-sdk.md#sdk-version).

---

# Changelog

> What changed in Kheelona Voice, newest first.

Source: https://docs.voice.kheelona.com/policies/changelog/

New entries are added at the top. A breaking change is always announced here at least six months before the old version
stops. See [Versioning](https://docs.voice.kheelona.com/policies/versioning.md).

## v1 (first release)

The first release of the REST API (`v1`) and the realtime protocol (`kheelona.v1`).

- Sessions created from your server, with a one-time client secret for browsers and apps.
- Live voice over WebRTC and WebSocket, with live captions and typed input.
- Interruptions, echo handling and turn-taking.
- Eleven languages, with beta and alpha languages marked.
- Safety checks on what children say and on what the voices say.
- The browser SDK, and examples for the browser, Node, Python and curl.

---

# Credits

> The open-source tools this documentation site is built with, the data it credits, and their licences.

Source: https://docs.voice.kheelona.com/policies/credits/

This documentation site is built with open-source tools. Thank you to the people who make them. One data source is credited as its licence asks.

| Tool | What it does here | Licence |
|---|---|---|
| [Starlight](https://github.com/withastro/starlight) | The documentation framework | [MIT](https://github.com/withastro/starlight/blob/main/LICENSE) |
| [Astro](https://github.com/withastro/astro) | What Starlight runs on | [MIT](https://github.com/withastro/astro/blob/main/LICENSE) |
| [Pagefind](https://github.com/CloudCannon/pagefind) | The search | [MIT](https://github.com/CloudCannon/pagefind/blob/main/LICENSE) |
| [Expressive Code](https://github.com/expressive-code/expressive-code) | The code blocks | [MIT](https://github.com/expressive-code/expressive-code/blob/main/LICENSE) |
| [Outfit](https://github.com/Outfitio/Outfit-Fonts) | The font | [SIL Open Font License 1.1](https://github.com/Outfitio/Outfit-Fonts/blob/main/OFL.txt) |
| [IP Geolocation by DB-IP](https://db-ip.com) | The country and region a connection comes from, looked up offline. See [Data and privacy](https://docs.voice.kheelona.com/policies/data-and-privacy.md#location). | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) |

---

# Node server example

> A Node mint route and a server-to-server relay, with every file and how to run them.

Source: https://docs.voice.kheelona.com/examples/node-server/

Two small programs for your own Node server:

1. **The mint route** (`src/server.ts`): `POST /api/voice-session` creates a session with your API key and hands your
   page only the session ID, the client secret and the connection addresses. The key never leaves your server.
2. **The relay** (`src/relay.ts`): a server-to-server client for toys and backends. It creates a session, connects over a
   WebSocket, sends a typed turn and/or a WAV file at real time, prints the events and saves the reply audio.

`src/kheelona.ts` holds `createSession`: one idempotency key per session, up to 3 attempts on `429` and `503`, and errors
that carry the status, the code and the request ID.

You need Node 24 or newer and pnpm, and the example folder: the examples you received from the Kheelona team, with the SDK package.

## Settings

| Variable | Default | Meaning |
|---|---|---|
| `KHEELONA_API_KEY` | none (exit 2 with a hint when missing) | Your API key. Server side only. |
| `KHEELONA_API_URL` | `https://api.voice.kheelona.com` | The API |
| `KHEELONA_VOICE_ID` | `lumi-dragon` | The voice |
| `HOST` / `PORT` | `127.0.0.1` / `5181` | Where the mint route listens |
| `ALLOWED_ORIGIN` | `http://localhost:5180` | The one page origin allowed to call the mint route (blank or `null` is refused at start) |

## Run

```sh
cd examples/node-server
pnpm install
export KHEELONA_API_KEY=kv_live_…
pnpm start                             # the mint route on port 5181

pnpm relay --text "Hello Lumi"         # a typed turn
pnpm relay --text "Hello" --wav question.wav --out reply.wav
```

Relay flags: `--text <s>`, `--wav <file>` (at least one), `--out <file>` (default `reply.wav`), `--voice`,
`--languages en,hi` and `--no-interruptions`. The WAV must be 16 kHz, 16-bit, mono:
`ffmpeg -i in.wav -ac 1 -ar 16000 -sample_fmt s16 out.wav`.

The relay prints each event on one line and ends with `session <id> ended <reason> replies <n> audio <s>s saved <file>`.
Errors go to standard error as `error <code> request_id <id>`. Exit codes: 0 ended normally, 1 API or session error, 2
usage error. See the [server quickstart](https://docs.voice.kheelona.com/get-started/quickstart-server.md) for a walk through.

## Before you copy it

- The mint route is same-origin by design and sends no CORS headers. It refuses a request whose `Origin` is missing or
  not `ALLOWED_ORIGIN`, and anything but JSON.
- **Add your own sign-in check** where `server.ts` says `Authenticate your user here`. Pass that user's opaque ID as
  `end_user_ref`. See [Security](https://docs.voice.kheelona.com/guides/security.md).
- Never print, log or return the API key. Errors show the code, the status and the request ID only.

## The files

**node-server/src/kheelona.ts**

```ts title="kheelona.ts"
// The Kheelona REST helper shared by the mint route and the relay, and the example's environment.

/** The settings every example reads (see Settings on the example's docs page). */
export interface ExampleEnv {
  apiKey: string;
  apiUrl: string;
  voiceId: string;
  /** Mint route only: the one page origin that may call it. */
  allowedOrigin: string;
  host: string;
  port: number;
}

/**
 * Reads the settings from environment variables. Throws an Error with a one-line hint when the key is missing or
 * `PORT` is not a port number (the commands exit 2 with that line).
 */
export function readEnv(env: Record<string, string | undefined>): ExampleEnv {
  const apiKey = env['KHEELONA_API_KEY'];
  if (!apiKey) throw new Error('KHEELONA_API_KEY is not set. Create an API key in the Kheelona console and export it.');
  const allowedOrigin = (env['ALLOWED_ORIGIN'] ?? 'http://localhost:5180').trim();
  if (allowedOrigin === '' || allowedOrigin.toLowerCase() === 'null') {
    throw new Error("ALLOWED_ORIGIN must be your page's origin, for example https://app.example.com");
  }
  const port = Number(env['PORT'] ?? '5181');
  if (!Number.isInteger(port) || port < 0 || port > 65535) throw new Error('PORT must be a port number.');
  return {
    apiKey,
    apiUrl: env['KHEELONA_API_URL'] || 'https://api.voice.kheelona.com',
    voiceId: env['KHEELONA_VOICE_ID'] || 'lumi-dragon',
    allowedOrigin,
    host: env['HOST'] || '127.0.0.1',
    port,
  };
}

const REQUEST_TIMEOUT_MS = 15_000;
/** The longest wait for one retry, whatever `Retry-After` says. */
const MAX_WAIT_S = 30;

export interface CreateSessionOptions {
  apiUrl: string;
  /** Your API key. Keep it on the server: read it from the environment, never ship it to a browser or a toy. */
  apiKey: string;
  voiceId: string;
  /** Default `["en", "hi"]`. */
  languages?: string[];
  /** An opaque ID for your end user (not a name, e-mail address or other personal data). */
  endUserRef?: string;
  /** `false` = the voice is never cut off (for devices without echo cancellation). */
  interruptions?: boolean;
}

export interface CreatedSession {
  sessionId: string;
  /** The single-use client secret (`kvcs_…`). It is valid for 60 seconds. */
  clientSecret: string;
  websocketUrl: string;
  webrtcUrl: string;
}

/** An error answer from the API: the HTTP status, the error code and the request ID to quote to support. */
export class KheelonaApiError extends Error {
  readonly status: number;
  readonly code: string;
  readonly requestId: string | undefined;
  /** Seconds from the `Retry-After` header, if the API sent one. */
  readonly retryAfterSeconds: number | undefined;
  /** For status 0 (no answer): the name of the failure, e.g. `TimeoutError`. Never its message. */
  readonly causeName: string | undefined;

  constructor(status: number, code: string, requestId?: string, retryAfterSeconds?: number, causeName?: string) {
    super(`Kheelona API error ${String(status)} ${code}${requestId ? ` (request_id ${requestId})` : ''}`);
    this.name = 'KheelonaApiError';
    this.status = status;
    this.code = code;
    this.requestId = requestId;
    this.retryAfterSeconds = retryAfterSeconds;
    this.causeName = causeName;
  }
}

/** Creates a session with `POST /v1/sessions`. Retries 429 and 503 (up to 3 attempts) with one idempotency key. */
export async function createSession(opts: CreateSessionOptions): Promise<CreatedSession> {
  const idempotencyKey = globalThis.crypto.randomUUID(); // one per logical session: a retry never creates a second one
  const body = JSON.stringify({
    voice_id: opts.voiceId,
    languages: opts.languages ?? ['en', 'hi'],
    ...(opts.endUserRef === undefined ? {} : { end_user_ref: opts.endUserRef }),
    ...(opts.interruptions === undefined ? {} : { interruptions: opts.interruptions }),
  });
  for (let attempt = 1; ; attempt += 1) {
    let res: Response;
    try {
      res = await fetch(`${opts.apiUrl.replace(/\/+$/, '')}/v1/sessions`, {
        method: 'POST',
        headers: { Authorization: `Bearer ${opts.apiKey}`, 'Content-Type': 'application/json', 'Idempotency-Key': idempotencyKey },
        body,
        signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), // never wait forever for the API
      });
    } catch (error) {
      // Keep only the kind of failure: its message can echo the request.
      throw new KheelonaApiError(0, 'upstream_unreachable', undefined, undefined, error instanceof Error ? error.name : 'Error');
    }
    if (res.ok) {
      const created = (await res.json()) as {
        session_id: string;
        client_secret: { value: string };
        connect: { websocket_url: string; webrtc_url: string };
      };
      return {
        sessionId: created.session_id,
        clientSecret: created.client_secret.value,
        websocketUrl: created.connect.websocket_url,
        webrtcUrl: created.connect.webrtc_url,
      };
    }
    const error = await readError(res);
    const retryable = res.status === 429 || res.status === 503;
    if (!retryable || attempt >= 3) throw error;
    await new Promise((resolve) => setTimeout(resolve, Math.min(error.retryAfterSeconds ?? 1, MAX_WAIT_S) * 1000));
  }
}

/** Keeps only the error code and request ID from the answer (never the whole body). */
async function readError(res: Response): Promise<KheelonaApiError> {
  const header = res.headers.get('retry-after');
  const retryAfter = header !== null && /^\d+$/.test(header) ? Math.min(Number(header), Number.MAX_SAFE_INTEGER) : undefined; // a huge value stays a number the cap handles
  try {
    const parsed = (await res.json()) as { error?: { code?: unknown; request_id?: unknown } };
    const code = plain(parsed.error?.code) ?? 'unknown_error';
    const requestId = plain(parsed.error?.request_id);
    return new KheelonaApiError(res.status, code, requestId, retryAfter);
  } catch {
    return new KheelonaApiError(res.status, 'unknown_error', undefined, retryAfter);
  }
}

/** A short identifier made of plain characters, or undefined: nothing else from an answer is repeated. */
function plain(value: unknown): string | undefined {
  return typeof value === 'string' && /^[A-Za-z0-9_.:-]{1,64}$/.test(value) ? value : undefined;
}
```

**node-server/src/server.ts**

```ts title="server.ts"
// The mint route: your server creates the session with your API key and hands the page only a single-use client
// secret. Run it with `npm start` (see Settings on the example's docs page).
import type { Server } from 'node:http';
import { pathToFileURL } from 'node:url';
import express from 'express';
import { createSession, KheelonaApiError, readEnv, type ExampleEnv } from './kheelona.ts';

export function createApp(env: ExampleEnv): express.Express {
  const app = express();
  app.disable('x-powered-by');

  // Same-origin by design (no CORS headers): serve your page and this route from one origin.
  // Only your own page may call this route: a request with no Origin, or another one, never reaches Kheelona.
  const guard: express.RequestHandler = (req, res, next) => {
    const origin = req.get('origin');
    if (origin !== env.allowedOrigin) {
      res.status(403).json({ error: { code: 'origin_not_allowed' } });
      return;
    }
    res.set({ Vary: 'Origin', 'Cache-Control': 'no-store' });
    // JSON only: a cross-site HTML form cannot send it, so it cannot spend your key.
    if (!req.is('application/json')) {
      res.status(415).json({ error: { code: 'unsupported_media_type' } });
      return;
    }
    next();
  };

  app.all('/api/voice-session', guard);
  // express.json (body-parser) checks Content-Length against the limit before it reads the body.
  app.post('/api/voice-session', express.json({ limit: '2kb', type: 'application/json' }), async (req, res) => {
    const body = req.body as Record<string, unknown> | undefined;
    // Only `languages` is accepted. (Language codes here are 2-3 letters, as the API's are; the API validates too.)
    const languages = body?.['languages'];
    const knownKeys = typeof body === 'object' && !Array.isArray(body) && Object.keys(body).every((key) => key === 'languages');
    const validLanguages =
      languages === undefined ||
      (Array.isArray(languages) && languages.length >= 1 && languages.length <= 11 && languages.every((l) => typeof l === 'string' && /^[a-z]{2,3}$/.test(l)));
    if (!knownKeys || !validLanguages) {
      res.status(400).json({ error: { code: 'invalid_request' } });
      return;
    }
    try {
      // Authenticate your user here: only signed-in users may start a session.
      const session = await createSession({
        apiUrl: env.apiUrl,
        apiKey: env.apiKey,
        voiceId: env.voiceId,
        ...(languages === undefined ? {} : { languages: languages as string[] }),
        endUserRef: 'demo-user', // replace with the signed-in user's opaque ID
      });
      // Return only what the page needs: never the API key, and nothing else from the answer.
      res.status(201).json({
        sessionId: session.sessionId,
        clientSecret: session.clientSecret,
        endpoints: { webrtcUrl: session.webrtcUrl, websocketUrl: session.websocketUrl },
      });
    } catch (error) {
      const api = error instanceof KheelonaApiError ? error : undefined;
      console.error(`mint failed: ${api ? `${String(api.status)} ${api.code} request_id ${api.requestId ?? '-'}` : 'unexpected error'}`);
      if (api && (api.status === 429 || api.status === 503)) {
        if (api.retryAfterSeconds !== undefined) res.set('Retry-After', String(api.retryAfterSeconds));
        res.status(api.status);
      } else {
        res.status(502);
      }
      res.json({ error: { code: api?.code ?? 'internal', ...(api?.requestId ? { requestId: api.requestId } : {}) } });
    }
  });

  // A malformed JSON body is the caller's mistake, not ours.
  // Express recognises an error handler by its four parameters, so `_next` stays.
  // eslint-disable-next-line @typescript-eslint/no-unused-vars
  app.use((error: unknown, _req: express.Request, res: express.Response, _next: express.NextFunction) => {
    const tooLarge = (error as { status?: number }).status === 413;
    res.status(tooLarge ? 413 : 400).json({ error: { code: tooLarge ? 'payload_too_large' : 'invalid_request' } });
  });
  return app;
}

/** Starts the mint route on `env.host:env.port` (loopback by default) and resolves when it is listening. */
export function startServer(env: ExampleEnv): Promise<Server> {
  return new Promise((resolve, reject) => {
    const server = createApp(env).listen(env.port, env.host, () => {
      resolve(server);
    });
    server.once('error', reject);
  });
}

if (process.argv[1] && pathToFileURL(process.argv[1]).href === import.meta.url) {
  let env: ExampleEnv;
  try {
    env = readEnv(process.env);
  } catch (error) {
    console.error((error as Error).message);
    process.exit(2);
  }
  const server = await startServer(env);
  const address = server.address();
  console.log(`mint route on http://${env.host}:${typeof address === 'object' && address ? String(address.port) : String(env.port)}/api/voice-session (page origin ${env.allowedOrigin})`);
}
```

**node-server/src/relay.ts**

```ts title="relay.ts"
// A server-to-server relay: creates a session, streams a WAV (and/or a typed turn) to the voice over the WebSocket at
// real time, prints the events, and saves the reply audio to a WAV file. Run it with `pnpm relay --text "Hello"`.
import { readFile, writeFile } from 'node:fs/promises';
import type { IncomingMessage } from 'node:http';
import { parseArgs } from 'node:util';
import { pathToFileURL } from 'node:url';
import { WebSocket, type RawData } from 'ws';
import { createSession, KheelonaApiError, readEnv, type CreatedSession } from './kheelona.ts';
import { readPcm16Wav, writeWav } from './wav.ts';

export interface RelayOptions {
  apiUrl: string;
  apiKey: string;
  voiceId: string;
  languages?: string[];
  /** A typed turn, sent first. */
  text?: string;
  /** A 16 kHz mono PCM16 WAV, sent in real time after the typed turn's reply. */
  wavFile?: string;
  /** Where the reply audio (24 kHz mono WAV) is saved. */
  outFile: string;
  /** `false` = the voice is never cut off. Default: the API's default (on). */
  interruptions?: boolean;
  /** Declare `playback_pause` and pause the playback buffer on `reply.paused`. Default: on. `false` = the voice never pauses. */
  pausePlayback?: boolean;
  /** Where event lines go (default: standard output). */
  log?: (line: string) => void;
  /** Milliseconds to wait for a reply before giving up (default 20,000). */
  replyTimeoutMs?: number;
}

export interface RelaySummary {
  sessionId: string;
  endReason: string;
  replies: number;
  audioSeconds: number;
  outFile: string;
}

/** A bad command line or input file: the CLI exits 2. */
export class UsageError extends Error {}

/**
 * A failed session: the code, the request ID when an HTTP answer carried one, and the session ID once the session
 * existed (realtime `error` events carry no request ID: quote the session ID and the code). The CLI exits 1.
 */
export class RelayError extends Error {
  readonly code: string;
  readonly requestId: string | undefined;
  sessionId: string | undefined;
  constructor(code: string, message: string, requestId?: string) {
    super(message);
    this.name = 'RelayError';
    this.code = code;
    this.requestId = requestId;
  }
}

/**
 * Opens the WebSocket the way a server does: the client secret in the Authorization header, the `kheelona.v1`
 * subprotocol, and no Origin header (a request with one is treated as a browser). Attach your listeners straight
 * away: the server's first event can arrive in the same moment the connection opens.
 */
function openSocket(session: CreatedSession): WebSocket {
  return new WebSocket(session.websocketUrl, ['kheelona.v1'], {
    headers: { Authorization: `Bearer ${session.clientSecret}` },
    handshakeTimeout: 20_000,
  });
}

/** Before the upgrade, errors come back as HTTP errors with a JSON body: keep the code and the request ID. */
function readRefusal(res: IncomingMessage): Promise<RelayError> {
  return new Promise((resolve) => {
    const chunks: Buffer[] = [];
    res.on('data', (chunk: Buffer) => chunks.push(chunk));
    res.on('end', () => {
      let code = `http_${String(res.statusCode)}`;
      let requestId: string | undefined;
      try {
        const body = JSON.parse(Buffer.concat(chunks).toString('utf8').slice(0, 8192)) as { error?: { code?: string; request_id?: string } };
        if (typeof body.error?.code === 'string' && /^[a-z0-9_]{1,64}$/.test(body.error.code)) code = body.error.code;
        if (typeof body.error?.request_id === 'string' && /^[A-Za-z0-9_]{1,64}$/.test(body.error.request_id)) requestId = body.error.request_id;
      } catch {
        // Not JSON: keep the HTTP status as the code.
      }
      resolve(new RelayError(code, `The connection was refused (${code})`, requestId));
    });
  });
}

const FRAME_MS = 20;
const SAMPLES_PER_FRAME = 16 * FRAME_MS; // 16 kHz: 320 samples = 640 bytes
const SILENCE = Buffer.alloc(SAMPLES_PER_FRAME * 2);

/** Splits microphone audio (16 kHz PCM16) into 20 ms frames; the last one is padded with silence. */
export function wavToFrames(samples: Int16Array): Buffer[] {
  const frames: Buffer[] = [];
  for (let at = 0; at < samples.length; at += SAMPLES_PER_FRAME) {
    const frame = Buffer.alloc(SAMPLES_PER_FRAME * 2);
    samples.subarray(at, at + SAMPLES_PER_FRAME).forEach((sample, i) => {
      frame.writeInt16LE(sample, i * 2);
    });
    frames.push(frame);
  }
  return frames;
}

/**
 * Sends one frame every 20 ms, in real time. Frame k goes out at start + k × 20 ms, measured against the clock, so a
 * late wake-up never makes the stream drift (the server drops audio that arrives faster than 2× real time).
 * Stops when `next` returns nothing or `stop` says so.
 */
export async function sendPaced(opts: { next: () => Buffer | undefined; send: (frame: Buffer) => void; stop: () => boolean }): Promise<void> {
  const start = performance.now();
  for (let k = 0; !opts.stop(); k += 1) {
    const frame = opts.next();
    if (!frame) return;
    opts.send(frame);
    await new Promise((resolve) => setTimeout(resolve, Math.max(0, start + (k + 1) * FRAME_MS - performance.now())));
  }
}

/**
 * The playback buffer, with pause support. Frames wait in `push` until `drain` hands them to the speaker; while the
 * voice has paused its own playback (`reply.paused`) `drain` returns nothing and the frames are kept, so nothing is lost.
 * A pause ends at the first of: `reply.resumed`, `interrupted` (the buffer is flushed), `reply.done` or the next
 * `reply.started`. A pause, resume or done for any reply other than the one that is playing (or one that has already
 * ended) is stale and ignored.
 */
export class PausePlayback {
  private queue: Buffer[] = [];
  private current: string | undefined;
  private isPaused = false;
  /** Frames thrown away by `interrupted` or by the end of the session. */
  dropped = 0;

  /** `enabled = false` when the relay did not declare `playback_pause`: the server sends no pause events then. */
  private readonly enabled: boolean;
  constructor(enabled = true) {
    this.enabled = enabled;
  }

  get paused(): boolean {
    return this.isPaused;
  }

  onEvent(event: { type: string; reply_id?: unknown }): void {
    switch (event.type) {
      case 'reply.started':
        this.current = typeof event.reply_id === 'string' ? event.reply_id : undefined; // a new reply ends any pause
        this.isPaused = false;
        break;
      case 'reply.paused':
        if (this.enabled && this.current !== undefined && event.reply_id === this.current) this.isPaused = true;
        break;
      case 'reply.resumed':
        if (event.reply_id === this.current) this.isPaused = false;
        break;
      case 'interrupted': // flush: what was queued will never play
        this.dropped += this.queue.length;
        this.queue = [];
        this.isPaused = false;
        this.current = undefined; // that reply is over: a pause naming it is stale
        break;
      case 'reply.done':
        if (event.reply_id === this.current) {
          this.isPaused = false;
          this.current = undefined;
        }
        break;
      case 'session.ended':
        this.dropped += this.queue.length;
        this.queue = [];
        this.isPaused = false;
        break;
      default:
        break;
    }
  }

  push(frame: Buffer): void {
    this.queue.push(frame);
  }

  /** The frames to play now: none while paused. */
  drain(): Buffer[] {
    if (this.isPaused) return [];
    const frames = this.queue;
    this.queue = [];
    return frames;
  }
}

/** Collects the reply audio, dropping what the server told us to drop. */
export class ReplyAudio {
  private readonly chunks: Buffer[] = [];
  private readonly playback: PausePlayback;
  private newestSeq = -1;
  private droppedThrough = -1;

  constructor(playback = new PausePlayback()) {
    this.playback = playback;
  }

  /** A binary frame: an 8-byte header (kind, flags, sentence, reply_seq) then PCM16 24 kHz samples. */
  push(frame: Buffer): void {
    if (frame.length < 8 || frame.readUInt8(0) !== 0x01) return; // not reply audio: ignore unknown kinds
    const seq = frame.readUInt32LE(4);
    this.newestSeq = Math.max(this.newestSeq, seq);
    if (seq <= this.droppedThrough) return; // a late frame of an interrupted reply
    this.playback.push(frame.subarray(8));
    this.play();
  }

  /** Every event goes here first, so the playback buffer pauses, resumes and flushes with the voice. */
  onEvent(event: { type: string; reply_id?: unknown }): void {
    this.playback.onEvent(event);
    this.play(); // a resume, a done or a new reply lets the held frames out
  }

  /** `interrupted`: drop every later frame of the reply being played. (The playback buffer is flushed by `onEvent`.) */
  interrupted(): void {
    this.droppedThrough = this.newestSeq;
  }

  pcm(): Buffer {
    this.play();
    return Buffer.concat(this.chunks);
  }

  // A real player would hand these frames to the speaker; this example saves them.
  private play(): void {
    this.chunks.push(...this.playback.drain());
  }
}

/** The lines printed for an event: its type, plus the text of what was heard or said. */
export function describeEvent(event: { type: string; text?: unknown }): string[] {
  const lines = [`event ${event.type}`];
  if ((event.type === 'transcript.final' || event.type === 'reply.sentence') && typeof event.text === 'string') lines.push(`  text: ${event.text}`);
  return lines;
}

type ServerEvent = { type: string; [key: string]: unknown };

/** Runs one session: see the file header. Rejects with `KheelonaApiError`, `RelayError` or `UsageError`. */
export async function runRelay(opts: RelayOptions): Promise<RelaySummary> {
  const log =
    opts.log ??
    ((line: string) => {
      console.log(line);
    });
  const replyTimeoutMs = opts.replyTimeoutMs ?? 20_000;
  if (!opts.text && !opts.wavFile) throw new UsageError('Give --text and/or --wav.');
  if (opts.text && Array.from(opts.text).length > 2000) throw new UsageError('--text is at most 2000 characters');
  let wavFrames: Buffer[] | undefined;
  if (opts.wavFile) {
    try {
      wavFrames = wavToFrames(readPcm16Wav(await readFile(opts.wavFile)).samples);
    } catch (error) {
      throw new UsageError(error instanceof Error && 'code' in error ? `Cannot read ${opts.wavFile}.` : (error as Error).message);
    }
  }

  const session = await createSession({
    apiUrl: opts.apiUrl,
    apiKey: opts.apiKey,
    voiceId: opts.voiceId,
    ...(opts.languages ? { languages: opts.languages } : {}),
    ...(opts.interruptions === undefined ? {} : { interruptions: opts.interruptions }),
  });
  const ws = openSocket(session);
  const pause = opts.pausePlayback ?? true;

  const audio = new ReplyAudio(new PausePlayback(pause));
  const queue: Buffer[] = []; // microphone frames waiting to go out; silence fills the gaps
  let wavQueued = false;
  let turnsSent = 0;
  let turnsAnswered = 0;
  let replies = 0;
  let stopPacer = false;
  let pacer: Promise<void> | undefined;
  let timer: NodeJS.Timeout | undefined;
  let finished = false;

  const summary = await new Promise<RelaySummary>((resolve, reject) => {
    const finish = (error?: Error, endReason = 'unknown'): void => {
      if (finished) return;
      finished = true;
      stopPacer = true;
      clearTimeout(timer);
      if (error instanceof RelayError) error.sessionId ??= session.sessionId;
      if (error) reject(error);
      else resolve({ sessionId: session.sessionId, endReason, replies, audioSeconds: audio.pcm().length / 2 / 24_000, outFile: opts.outFile });
    };
    // Nothing from the server for a while (waiting for session.ready, a reply or session.ended): give up, so a copy
    // of this never hangs. Every message from the server restarts the wait.
    const armTimer = (): void => {
      clearTimeout(timer);
      timer = setTimeout(() => {
        finish(new RelayError('no_reply', `Nothing from the server for ${String(replyTimeoutMs / 1000)} s`));
      }, replyTimeoutMs);
    };
    const sendEvent = (event: Record<string, unknown>): void => {
      ws.send(JSON.stringify(event));
    };
    const queueWav = (): void => {
      if (!wavFrames || wavQueued) return;
      wavQueued = true;
      turnsSent += 1;
      queue.push(...wavFrames);
    };
    // Microphone audio at real time for the whole session: your turn's frames, then silence.
    const startPacer = (): void => {
      pacer = sendPaced({
        next: () => {
          const frame = queue.shift();
          if (frame) return frame;
          return SILENCE;
        },
        send: (frame) => {
          if (ws.readyState === WebSocket.OPEN) ws.send(frame);
        },
        stop: () => stopPacer,
      });
    };

    ws.once('open', armTimer);
    ws.on('message', (data: RawData, isBinary: boolean) => {
      armTimer();
      if (isBinary) {
        if (Buffer.isBuffer(data)) audio.push(data);
        return;
      }
      let event: ServerEvent;
      try {
        event = JSON.parse((data as Buffer).toString('utf8')) as ServerEvent;
      } catch {
        return;
      }
      if (typeof event.type !== 'string') return;
      for (const line of describeEvent(event)) log(line);
      audio.onEvent(event);

      switch (event.type) {
        case 'session.ready':
          // Nothing is sent before this: audio sent earlier is dropped.
          if (pause) sendEvent({ type: 'client.capabilities', playback_pause: true });
          if (opts.text) {
            sendEvent({ type: 'input.text', text: opts.text });
            turnsSent += 1;
          } else {
            queueWav();
          }
          startPacer();
          break;
        case 'interrupted':
          audio.interrupted();
          break;
        case 'reply.done':
          replies += 1;
          if (event['reason'] === 'interrupted') break; // the turn goes on with a new reply
          turnsAnswered += 1;
          if (turnsAnswered < turnsSent) break;
          if (!wavQueued && wavFrames) {
            queueWav(); // the typed reply is done: now the WAV
          } else {
            stopPacer = true;
            void (pacer ?? Promise.resolve()).then(() => {
              sendEvent({ type: 'session.end' });
            });
          }
          break;
        case 'error':
          if (event['fatal'] === true) {
            finish(
              new RelayError(typeof event['code'] === 'string' ? event['code'] : 'error', 'The session failed'),
            );
          }
          break;
        case 'session.ended':
          finish(undefined, typeof event['reason'] === 'string' ? event['reason'] : 'unknown');
          break;
        default:
          break;
      }
    });
    ws.on('unexpected-response', (_req, res) => {
      void readRefusal(res).then((error) => {
        finish(error);
      });
    });
    ws.on('close', () => {
      finish(new RelayError('disconnected', 'The connection closed before the session ended'));
    });
    ws.on('error', () => {
      finish(new RelayError('connection_failed', 'The WebSocket failed'));
    });
  }).finally(() => {
    ws.terminate();
  });

  await writeFile(opts.outFile, writeWav(audio.pcm(), 24_000));
  return summary;
}

/** The command line. Returns the exit code: 0 ended normally, 1 API or session error, 2 usage error. */
export async function runCli(
  argv: string[],
  env: Record<string, string | undefined>,
  out: (line: string) => void,
  err: (line: string) => void,
  overrides: { replyTimeoutMs?: number } = {},
): Promise<number> {
  try {
    const { values } = parseArgs({
      args: argv,
      options: {
        text: { type: 'string' },
        wav: { type: 'string' },
        out: { type: 'string', default: 'reply.wav' },
        voice: { type: 'string' },
        languages: { type: 'string' },
        'no-interruptions': { type: 'boolean', default: false },
        'no-pause': { type: 'boolean', default: false },
      },
      strict: true,
    });
    const settings = readEnv(env);
    const summary = await runRelay({
      apiUrl: settings.apiUrl,
      apiKey: settings.apiKey,
      voiceId: values.voice ?? settings.voiceId,
      ...(values.languages ? { languages: values.languages.split(',').map((l) => l.trim()) } : {}),
      ...(values.text ? { text: values.text } : {}),
      ...(values.wav ? { wavFile: values.wav } : {}),
      outFile: values.out,
      ...(values['no-interruptions'] ? { interruptions: false } : {}),
      ...(values['no-pause'] ? { pausePlayback: false } : {}),
      log: out,
      ...(overrides.replyTimeoutMs === undefined ? {} : { replyTimeoutMs: overrides.replyTimeoutMs }),
    });
    out(`session ${summary.sessionId} ended ${summary.endReason} replies ${String(summary.replies)} audio ${summary.audioSeconds.toFixed(1)}s saved ${summary.outFile}`);
    return 0;
  } catch (error) {
    if (error instanceof KheelonaApiError) {
      // No answer at all (status 0): name the failure, like the Python example.
      const code = error.status === 0 ? `api_unreachable_${error.causeName ?? 'Error'}` : error.code;
      err(`error ${code} request_id ${error.requestId ?? '-'}`);
      return 1;
    }
    if (error instanceof RelayError) {
      const session = error.sessionId ? ` session ${error.sessionId}` : '';
      err(`error ${error.code}${session} request_id ${error.requestId ?? '-'}`);
      return 1;
    }
    // A usage mistake (bad flag, missing key, unreadable WAV): one line, exit 2.
    err(error instanceof Error ? error.message : 'Invalid usage.');
    return 2;
  }
}

if (process.argv[1] && pathToFileURL(process.argv[1]).href === import.meta.url) {
  process.exitCode = await runCli(
    process.argv.slice(2),
    process.env,
    (line) => {
      console.log(line);
    },
    (line) => {
      console.error(line);
    },
  );
}
```

**node-server/src/wav.ts**

```ts title="wav.ts"
// Minimal WAV reading and writing for the relay: PCM, 16-bit, mono. No dependencies.

const HINT = 'Convert it with: ffmpeg -i in.wav -ac 1 -ar 16000 -sample_fmt s16 out.wav';

/**
 * Reads a PCM 16-bit mono WAV. `sampleRate` is the rate the file must have: 16000 for the microphone audio the
 * relay sends (the default), 24000 for the reply audio it saves.
 * Throws an Error that names the problem and the ffmpeg command that fixes it.
 */
export function readPcm16Wav(buf: Buffer, sampleRate = 16000): { sampleRate: number; samples: Int16Array } {
  if (buf.length < 12 || buf.toString('ascii', 0, 4) !== 'RIFF' || buf.toString('ascii', 8, 12) !== 'WAVE') {
    throw new Error(`This is not a WAV file. ${HINT}`);
  }
  let fmt: { format: number; channels: number; rate: number; bits: number } | undefined;
  let offset = 12;
  while (offset + 8 <= buf.length) {
    const id = buf.toString('ascii', offset, offset + 4);
    const size = buf.readUInt32LE(offset + 4);
    const body = offset + 8;
    if (id === 'fmt ' && body + 16 <= buf.length) {
      fmt = { format: buf.readUInt16LE(body), channels: buf.readUInt16LE(body + 2), rate: buf.readUInt32LE(body + 4), bits: buf.readUInt16LE(body + 14) };
    } else if (id === 'data') {
      if (!fmt) throw new Error(`The WAV has no fmt chunk before its audio. ${HINT}`);
      const problem = formatProblem(fmt, sampleRate);
      if (problem) throw new Error(`${problem}. ${HINT}`);
      // A streamed WAV can carry an unknown (0xFFFFFFFF) size: take what the file holds, in whole samples.
      const end = Math.min(buf.length, body + size);
      const bytes = end - body - ((end - body) % 2);
      const samples = new Int16Array(bytes / 2);
      for (let i = 0; i < samples.length; i += 1) samples[i] = buf.readInt16LE(body + i * 2);
      return { sampleRate, samples };
    }
    offset = body + size + (size % 2); // chunks are padded to an even size
  }
  throw new Error(`The WAV has no audio data. ${HINT}`);
}

function formatProblem(fmt: { format: number; channels: number; rate: number; bits: number }, rate: number): string | undefined {
  if (fmt.format !== 1) return `The WAV is not plain PCM (format ${String(fmt.format)})`;
  if (fmt.channels !== 1) return `The WAV has ${String(fmt.channels)} channels (${fmt.channels === 2 ? 'stereo' : 'not mono'}); it must be mono`;
  if (fmt.bits !== 16) return `The WAV is ${String(fmt.bits)}-bit; it must be 16-bit`;
  if (fmt.rate !== rate) return `The WAV is ${String(fmt.rate)} Hz; it must be ${String(rate)} Hz`;
  return undefined;
}

/** Wraps raw PCM 16-bit little-endian mono samples in a WAV container. */
export function writeWav(pcm: Buffer, sampleRate: number): Buffer {
  const header = Buffer.alloc(44);
  header.write('RIFF', 0, 'ascii');
  header.writeUInt32LE(36 + pcm.length, 4);
  header.write('WAVE', 8, 'ascii');
  header.write('fmt ', 12, 'ascii');
  header.writeUInt32LE(16, 16);
  header.writeUInt16LE(1, 20); // PCM
  header.writeUInt16LE(1, 22); // mono
  header.writeUInt32LE(sampleRate, 24);
  header.writeUInt32LE(sampleRate * 2, 28);
  header.writeUInt16LE(2, 32);
  header.writeUInt16LE(16, 34);
  header.write('data', 36, 'ascii');
  header.writeUInt32LE(pcm.length, 40);
  return Buffer.concat([header, pcm]);
}
```

**node-server/package.json**

```json title="package.json"
{
  "name": "@kheelona/example-node-server",
  "version": "0.0.0",
  "private": true,
  "description": "Kheelona Voice example: a client-secret mint route (Express) and a server-to-server WebSocket relay. Proprietary; private distribution only.",
  "license": "UNLICENSED",
  "type": "module",
  "engines": {
    "node": ">=24"
  },
  "scripts": {
    "start": "node src/server.ts",
    "relay": "node src/relay.ts"
  },
  "dependencies": {
    "express": "5.3.0",
    "ws": "8.22.0"
  },
  "devDependencies": {
    "@types/express": "5.0.6",
    "@types/ws": "8.18.2"
  }
}
```

The examples are Kheelona's own and are given to partners privately. Contact the Kheelona team for licensing.

---

# Python server example

> A Python mint route and a server-to-server relay, with every file and how to run them.

Source: https://docs.voice.kheelona.com/examples/python-server/

Two small programs for a Python backend:

- **`app.py`** is a mint route for a web page. `POST /api/voice-session` creates a session with your API key and hands the
  page a client secret (the session ID, the client secret and the connection addresses), so the key never reaches a browser.
- **`relay.py`** is a server-to-server relay. It sends a typed turn and/or a WAV file to a voice over the WebSocket,
  prints the events and saves the reply audio. A toy or a phone-line relay works the same way.
- **`kheelona.py`** is the helper both use: `create_session()` with an idempotency key and retries on `429` and `503`.
- **`wavio.py`** reads and writes WAV files with the standard library.

You need Python 3.12 or newer, [uv](https://docs.astral.sh/uv/), and the example folder: the examples you received from the Kheelona team, with the SDK package.

## Settings

| Variable | Default | Meaning |
|---|---|---|
| `KHEELONA_API_KEY` | none (required) | Your API key. Server side only. |
| `KHEELONA_API_URL` | `https://api.voice.kheelona.com` | The API |
| `KHEELONA_VOICE_ID` | `lumi-dragon` | The voice |
| `HOST` / `PORT` | `127.0.0.1` / `5182` | Where the mint route listens |
| `ALLOWED_ORIGIN` | `http://localhost:5180` | The one page origin allowed to call the mint route |

## Run

```sh
cd examples/python-server
uv sync
export KHEELONA_API_KEY=kv_live_…
uv run python app.py                   # the mint route on port 5182

uv run python relay.py --text "Hello! Tell me a short story." --out reply.wav
uv run python relay.py --wav question.wav --out reply.wav --languages en,hi
```

The WAV must be 16 kHz, 16-bit, mono. Convert any recording with
`ffmpeg -i in.wav -ac 1 -ar 16000 -sample_fmt s16 out.wav`. The relay prints events on standard output and errors on
standard error as `error <code> request_id <id>`. See the [server quickstart](https://docs.voice.kheelona.com/get-started/quickstart-server.md).

## Before you copy it

- A request with no `Origin`, a different `Origin`, or a body that is not JSON is refused (403 or 415) before any session
  is created. The route is meant to be same-origin with your page, or behind your own proxy: CORS is not enabled.
- **Add your own sign-in check** where `app.py` says `Authenticate your user here`, and pass that user's opaque ID as
  `end_user_ref`. See [Security](https://docs.voice.kheelona.com/guides/security.md).
- Never print, log or commit the API key.

## The files

**python-server/app.py**

```python title="app.py"
"""A mint route for browsers: POST /api/voice-session creates a session and hands the page a client secret.

Run it:  KHEELONA_API_KEY=... uv run python app.py   (it listens on 127.0.0.1:5182 unless HOST/PORT say otherwise)

The page never sees your API key. It gets a client secret that works once and expires after 60 seconds.
"""

import json
import logging
import os
import re
import sys
from collections.abc import Mapping
from contextlib import asynccontextmanager

import httpx
from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse

from kheelona import KheelonaApiError, create_session

DEFAULT_ORIGIN = "http://localhost:5180"
DEFAULT_API_URL = "https://api.voice.kheelona.com"
MAX_BODY_BYTES = 2048
log = logging.getLogger("kheelona.example")


def listen_address(env: Mapping[str, str]) -> tuple[str, int]:
    """Loopback by default. Binding to another address is your decision, and then you need real authentication."""
    return env.get("HOST", "127.0.0.1"), int(env.get("PORT", "5182"))


# The answers carry a client secret: never cache them, and say that they depend on the Origin header.
NO_STORE = {"Cache-Control": "no-store", "Vary": "Origin"}


def _error(status: int, code: str, headers: dict[str, str] | None = None) -> JSONResponse:
    return JSONResponse({"error": {"code": code}}, status_code=status, headers=headers)


def _languages(body: object) -> list[str] | None:
    """The `languages` of a request body, None if it has none, ValueError if the body is not what we accept."""
    if not isinstance(body, dict) or set(body) - {"languages"}:
        raise ValueError("body")
    languages = body.get("languages")
    if languages is None:
        return None
    if not isinstance(languages, list) or not 1 <= len(languages) <= 11:
        raise ValueError("languages")
    if not all(isinstance(code, str) and re.fullmatch(r"[a-z]{2,3}", code) for code in languages):
        raise ValueError("languages")
    return languages


def create_app(
    env: Mapping[str, str],
    *,
    transport: httpx.AsyncBaseTransport | None = None,
) -> FastAPI:
    """Build the app from environment settings. `transport` is only for tests."""
    api_key = env["KHEELONA_API_KEY"]  # KeyError if missing: main() turns it into a one-line hint
    api_url = env.get("KHEELONA_API_URL", DEFAULT_API_URL).rstrip("/")
    voice_id = env.get("KHEELONA_VOICE_ID", "lumi-dragon")
    allowed_origin = env.get("ALLOWED_ORIGIN", DEFAULT_ORIGIN).strip()
    if allowed_origin.lower() in ("", "null"):
        raise ValueError("ALLOWED_ORIGIN must be your page's origin, for example https://app.example.com")
    client = httpx.AsyncClient(transport=transport, timeout=15.0)

    @asynccontextmanager
    async def lifespan(_app: FastAPI):
        yield
        await client.aclose()

    app = FastAPI(lifespan=lifespan, docs_url=None, redoc_url=None, openapi_url=None)

    @app.post("/api/voice-session")
    async def mint(request: Request) -> JSONResponse:
        # Only your own page may call this: a missing or foreign Origin is refused before anything else.
        if request.headers.get("origin") != allowed_origin:
            return _error(403, "origin_not_allowed")
        # JSON only, so a cross-site HTML form cannot start a session.
        media_type, _, _ = request.headers.get("content-type", "").partition(";")
        if media_type.strip().lower() != "application/json":
            return _error(415, "unsupported_media_type", NO_STORE)
        # Read the body in pieces and stop as soon as it passes the limit, even with no Content-Length.
        raw = bytearray()
        async for chunk in request.stream():
            raw += chunk
            if len(raw) > MAX_BODY_BYTES:
                return _error(413, "payload_too_large", NO_STORE)
        try:
            languages = _languages(json.loads(raw))
        except ValueError:
            return _error(400, "invalid_request", NO_STORE)

        # Authenticate your user here: only signed-in users may start a session.
        # The end_user_ref is that user's opaque id (never a name or an email); this demo uses a fixed one.
        try:
            session = await create_session(
                client,
                api_url=api_url,
                api_key=api_key,
                voice_id=voice_id,
                languages=languages,
                end_user_ref="demo-user",
            )
        except KheelonaApiError as err:
            log.warning("create session failed: status=%s code=%s request_id=%s", err.status, err.code, err.request_id)
            body = {"code": err.code, **({"requestId": err.request_id} if err.request_id else {})}
            if err.status in (429, 503):
                headers = {**NO_STORE, **({"Retry-After": str(err.retry_after)} if err.retry_after else {})}
                return JSONResponse({"error": body}, status_code=err.status, headers=headers)
            return JSONResponse({"error": body}, status_code=502, headers=NO_STORE)
        except httpx.HTTPError as err:
            log.warning("create session failed: %s", type(err).__name__)
            return _error(502, "upstream_unreachable", NO_STORE)

        # Return only what the page needs, nothing else from the API's answer.
        return JSONResponse(
            {
                "sessionId": session["session_id"],
                "clientSecret": session["client_secret"]["value"],
                "endpoints": {
                    "webrtcUrl": session["connect"]["webrtc_url"],
                    "websocketUrl": session["connect"]["websocket_url"],
                },
            },
            status_code=201,
            headers=NO_STORE,
        )

    return app


def __getattr__(name: str) -> FastAPI:
    """`uvicorn app:app` works: the app is built on first use, from the environment."""
    if name == "app":
        return create_app(os.environ)
    raise AttributeError(name)


def main(env: Mapping[str, str] = os.environ) -> int:
    import uvicorn

    try:
        app = create_app(env)
    except KeyError:
        print("Set KHEELONA_API_KEY to your Kheelona Voice API key (server side only).", file=sys.stderr)
        return 2
    host, port = listen_address(env)
    uvicorn.run(app, host=host, port=port, log_level="warning")
    return 0


if __name__ == "__main__":
    sys.exit(main())
```

**python-server/kheelona.py**

```python title="kheelona.py"
"""Create a Kheelona Voice session from your server (POST /v1/sessions).

Call this from your own backend only: it needs your API key, which must never reach a browser or a device.
"""

import asyncio
import re
import uuid
from collections.abc import Sequence

import httpx

MAX_ATTEMPTS = 3
MAX_WAIT_S = 30


class KheelonaApiError(Exception):
    """The API refused the request. Carries the status, error code and request id, never the key."""

    def __init__(self, status: int, code: str, request_id: str | None, retry_after: int | None = None):
        super().__init__(f"Kheelona API error {status} {code} (request_id {request_id or '-'})")
        self.status = status
        self.code = code
        self.request_id = request_id
        self.retry_after = retry_after


async def create_session(
    client: httpx.AsyncClient,
    *,
    api_url: str,
    api_key: str,
    voice_id: str,
    languages: Sequence[str] | None = None,
    end_user_ref: str | None = None,
    interruptions: bool = True,
) -> dict:
    """Create a session and return the API's answer: session_id, client_secret, connect, ...

    One Idempotency-Key covers every attempt, so a retry never creates a second session. A 429 or 503 is
    retried up to 3 times in all, after the seconds in its Retry-After header (1 if it has none).
    """
    body: dict = {"voice_id": voice_id, "languages": list(languages or ["en", "hi"])}
    if end_user_ref:
        body["end_user_ref"] = end_user_ref  # an opaque id of yours, never a name or an email
    if not interruptions:
        body["interruptions"] = False
    headers = {"Authorization": f"Bearer {api_key}", "Idempotency-Key": str(uuid.uuid4())}
    for attempt in range(1, MAX_ATTEMPTS + 1):
        response = await client.post(f"{api_url}/v1/sessions", json=body, headers=headers)
        if response.is_success:
            return response.json()
        error = _api_error(response)
        if response.status_code not in (429, 503) or attempt == MAX_ATTEMPTS:
            raise error
        await asyncio.sleep(min(error.retry_after or 1, MAX_WAIT_S))
    raise AssertionError("unreachable")


def _api_error(response: httpx.Response) -> KheelonaApiError:
    try:
        detail = response.json()["error"]
        code, request_id = _plain(detail["code"]) or "unknown_error", _plain(detail.get("request_id"))
    except (ValueError, KeyError, TypeError):
        code, request_id = "unknown_error", None
    retry_after = response.headers.get("Retry-After", "")
    return KheelonaApiError(response.status_code, code, request_id, int(retry_after) if retry_after.isdigit() else None)


def _plain(value: object) -> str | None:
    """A short identifier made of plain characters, or None: nothing else from an answer is repeated."""
    return value if isinstance(value, str) and re.fullmatch(r"[A-Za-z0-9_.:-]{1,64}", value) else None
```

**python-server/relay.py**

```python title="relay.py"
"""A server-to-server relay: send a typed turn and/or a WAV file to a Kheelona voice, print the events, save the reply.

    KHEELONA_API_KEY=... uv run python relay.py --text "Hello!" --wav question.wav --out reply.wav

Event text (what the voice heard and said) goes only to this terminal. Kheelona stores none of it, and this
example writes nothing but the reply audio file you asked for.
"""

import argparse
import asyncio
import json
import os
import struct
import sys
import time
from collections.abc import Callable, Mapping, Sequence
from dataclasses import dataclass

import httpx
from websockets.asyncio.client import connect
from websockets.exceptions import InvalidStatus, WebSocketException

from kheelona import KheelonaApiError, create_session
from wavio import read_pcm16_wav, write_wav

FRAME_S = 0.020  # one frame every 20 ms
FRAME_BYTES = 640  # 20 ms of 16 kHz, 16-bit mono audio
SILENCE = bytes(FRAME_BYTES)
OUTPUT_RATES = {"pcm16_16k": 16000, "pcm16_24k": 24000, "pcm16_48k": 48000}


class RelayError(Exception):
    """The session failed. `code`, `session_id` and `request_id` are safe to print.

    A realtime `error` event carries no request ID: quote the session ID and the code. An HTTP refusal carries a
    request ID.
    """

    def __init__(self, code: str, request_id: str | None = None, session_id: str | None = None):
        super().__init__(f"{code} (session {session_id or '-'}, request_id {request_id or '-'})")
        self.code = code
        self.request_id = request_id
        self.session_id = session_id


@dataclass
class RelaySummary:
    session_id: str
    end_reason: str
    replies: int
    audio_seconds: float
    out_file: str


@dataclass
class RelayOptions:
    api_url: str
    api_key: str
    voice_id: str = "lumi-dragon"
    languages: Sequence[str] = ("en", "hi")
    text: str | None = None
    wav_path: str | None = None
    out_path: str = "reply.wav"
    interruptions: bool = True
    pause_playback: bool = True  # declare playback_pause and pause the playback buffer on reply.paused
    out: Callable[[str], None] = print
    reply_timeout: float = 20.0  # seconds without anything from the server before giving up
    transport: httpx.AsyncBaseTransport | None = None  # a test hook


class FrameSource:
    """What to send next: the speech in 20 ms frames, then silence for as long as you keep asking."""

    def __init__(self) -> None:
        self._pending = bytearray()

    def load(self, pcm: bytes) -> None:
        self._pending += pcm

    def next(self) -> bytes:
        frame = bytes(self._pending[:FRAME_BYTES])
        del self._pending[:FRAME_BYTES]
        return frame.ljust(FRAME_BYTES, b"\0") if frame else SILENCE


async def pump(send, source: FrameSource, stop: asyncio.Event) -> None:
    """Send one frame every 20 ms, in real time, until `stop` is set.

    Frame k is due at start + k * 20 ms. Sleeping until that deadline (never a fixed 20 ms) keeps the pace
    exact: a late wake-up is made up on the next frame instead of piling up. Send silence while nobody
    speaks: the server's turn detection needs a steady stream to know when a person has finished.
    """
    start = time.monotonic()
    k = 0
    while not stop.is_set():
        delay = start + k * FRAME_S - time.monotonic()
        if delay > 0:
            await asyncio.sleep(delay)
        await send(source.next())
        k += 1
        await asyncio.sleep(0)  # give the receiving side a turn




class PausePlayback:
    """The playback buffer, with pause support.

    Frames wait in `push` until `drain` hands them to the speaker. While the voice has paused its own playback
    (`reply.paused`) `drain` returns nothing and the frames are kept, so nothing is lost. A pause ends at the first of
    `reply.resumed`, `interrupted` (the buffer is flushed), `reply.done` or the next `reply.started`. A pause, resume
    or done for any reply other than the one that is playing (or one that has already ended) is stale and ignored.
    """

    def __init__(self, enabled: bool = True) -> None:
        self.enabled = enabled  # False when the relay did not declare playback_pause
        self.paused = False
        self.dropped = 0  # frames thrown away by `interrupted` or by the end of the session
        self._queue: list[bytes] = []
        self._current: str | None = None

    def on_event(self, event: dict) -> None:
        kind = event.get("type")
        if kind == "reply.started":
            self._current = event.get("reply_id")  # a new reply ends any pause
            self.paused = False
        elif kind == "reply.paused":
            if self.enabled and self._current is not None and event.get("reply_id") == self._current:
                self.paused = True
        elif kind == "reply.resumed":
            if event.get("reply_id") == self._current:
                self.paused = False
        elif kind in ("interrupted", "session.ended"):  # flush: what was queued will never play
            self.dropped += len(self._queue)
            self._queue = []
            self.paused = False
            self._current = None  # that reply is over: a pause naming it is stale
        elif kind == "reply.done":
            if event.get("reply_id") == self._current:
                self.paused = False
                self._current = None

    def push(self, frame: bytes) -> None:
        self._queue.append(frame)

    def drain(self) -> list[bytes]:
        """The frames to play now: none while paused."""
        if self.paused:
            return []
        frames, self._queue = self._queue, []
        return frames




class Reply:
    """Collects the reply audio and counts finished replies."""

    def __init__(self, out: Callable[[str], None], pause: bool = True) -> None:
        self.out = out
        self.playback = PausePlayback(enabled=pause)
        self.pcm = bytearray()
        self.done = 0  # every reply.done, interrupted ones too
        self.answered = 0  # replies that finished a turn
        self.max_seq = 0  # highest reply_seq seen
        self.drop_upto = 0  # frames at or below this reply_seq belong to an interrupted reply
        self.output_rate = 24000
        self.session_id = ""
        self.ended: str | None = None

    def on_binary(self, data: bytes) -> None:
        """Reply audio: an 8-byte header (kind, flags, sentence, reply_seq) then PCM16 samples."""
        if len(data) < 8:
            return
        kind, _flags, _sentence, seq = struct.unpack_from("<BBHI", data)
        if kind != 1:  # 1 = reply audio
            return
        self.max_seq = max(self.max_seq, seq)
        if seq <= self.drop_upto:
            return  # a late frame of a reply that was interrupted
        self.playback.push(data[8:])
        self._play()

    def on_event(self, event: dict) -> None:
        kind = event.get("type")
        self.out(f"event {kind}")
        self.playback.on_event(event)
        self._play()  # a resume, a done or a new reply lets the held frames out
        if kind == "session.ready":
            self.session_id = event.get("session_id", self.session_id)
            self.output_rate = OUTPUT_RATES.get(event.get("audio", {}).get("output_format"), 24000)
        elif kind in ("transcript.final", "reply.sentence"):
            self.out(f"  text: {event.get('text', '')}")
        elif kind == "interrupted":
            self.drop_upto = self.max_seq  # flush what is queued, drop the rest of this reply
        elif kind == "reply.done":
            self.done += 1
            if event.get("reason") != "interrupted":  # an interrupted reply is followed by a new one
                self.answered += 1
        elif kind == "error" and event.get("fatal"):
            raise RelayError(str(event.get("code", "error")))
        elif kind == "session.ended":
            self.ended = str(event.get("reason", "unknown"))


    def _play(self) -> None:
        # A real player would hand these frames to the speaker; this example saves them.
        for frame in self.playback.drain():
            self.pcm += frame


async def _receive(ws, reply: Reply, timeout: float) -> None:
    """Handle one message from the server; give up after `timeout` seconds without one."""
    try:
        message = await asyncio.wait_for(ws.recv(), timeout)
    except TimeoutError:
        raise RelayError("no_reply") from None
    if isinstance(message, bytes):
        reply.on_binary(message)
    else:
        reply.on_event(json.loads(message))


def _refusal(err: InvalidStatus) -> RelayError:
    try:
        detail = json.loads(err.response.body or b"")["error"]
        return RelayError(str(detail["code"]), detail.get("request_id"))
    except (ValueError, KeyError, TypeError):
        return RelayError(f"http_{err.response.status_code}")


def _http(opts: RelayOptions) -> httpx.AsyncClient:
    """The HTTP client for creating the session, with a 15 second timeout."""
    return httpx.AsyncClient(transport=opts.transport, timeout=15.0)


async def run_relay(opts: RelayOptions) -> RelaySummary:
    speech = read_pcm16_wav(opts.wav_path)[1] if opts.wav_path else b""
    reply = Reply(opts.out, pause=opts.pause_playback)

    async with _http(opts) as http:
        session = await create_session(
            http,
            api_url=opts.api_url,
            api_key=opts.api_key,
            voice_id=opts.voice_id,
            languages=opts.languages,
            interruptions=opts.interruptions,
        )
    secret = session["client_secret"]["value"]
    try:
        # Servers send no Origin header (the websockets library adds none).
        async with connect(
            session["connect"]["websocket_url"],
            subprotocols=["kheelona.v1"],
            additional_headers={"Authorization": f"Bearer {secret}"},
        ) as ws:
            # Send nothing until the server says session.ready: audio sent before it is dropped.
            while not reply.session_id:
                await _receive(ws, reply, opts.reply_timeout)
            if opts.pause_playback:  # tell the voice this client can pause and resume its playback buffer
                await ws.send(json.dumps({"type": "client.capabilities", "playback_pause": True}))
            source = FrameSource()
            stop = asyncio.Event()
            turns = 0
            if opts.text:  # the typed turn goes first, and its reply is awaited before any audio is sent
                await ws.send(json.dumps({"type": "input.text", "text": opts.text}))
                turns += 1
            pumping = asyncio.create_task(pump(ws.send, source, stop))
            try:
                while reply.answered < turns:
                    await _receive(ws, reply, opts.reply_timeout)
                if speech:
                    source.load(speech)  # real time, 20 ms frames, then silence until the reply is done
                    turns += 1
                    while reply.answered < turns:
                        await _receive(ws, reply, opts.reply_timeout)
            finally:
                stop.set()
                await asyncio.gather(pumping, return_exceptions=True)  # the first error wins
            await ws.send(json.dumps({"type": "session.end"}))
            while reply.ended is None:
                await _receive(ws, reply, opts.reply_timeout)
    except InvalidStatus as err:
        raise _refusal(err) from None
    except RelayError as err:
        err.session_id = err.session_id or reply.session_id or None
        raise
    except (WebSocketException, OSError) as err:
        raise RelayError(f"connection_failed_{type(err).__name__}", session_id=reply.session_id or None) from None

    write_wav(opts.out_path, bytes(reply.pcm), reply.output_rate)
    seconds = len(reply.pcm) / 2 / reply.output_rate
    summary = RelaySummary(reply.session_id, reply.ended, reply.done, seconds, opts.out_path)
    opts.out(
        f"session {summary.session_id} ended {summary.end_reason} replies {summary.replies} "
        f"audio {seconds:.1f}s saved {summary.out_file}"
    )
    return summary


def main(argv: Sequence[str] | None = None, env: Mapping[str, str] = os.environ) -> int:
    parser = argparse.ArgumentParser(description="Talk to a Kheelona voice from a script and save the reply audio.")
    parser.add_argument("--text", help="a typed turn (the voice answers it first)")
    parser.add_argument("--wav", help="a 16-bit mono 16 kHz WAV file, sent as a spoken turn")
    parser.add_argument("--out", default="reply.wav", help="where to save the reply audio (default reply.wav)")
    parser.add_argument("--voice", default=env.get("KHEELONA_VOICE_ID", "lumi-dragon"))
    parser.add_argument("--languages", default="en,hi", help="comma-separated language codes (default en,hi)")
    parser.add_argument("--no-interruptions", action="store_true", help="the voice is never cut off")
    parser.add_argument(
        "--no-pause", action="store_true", help="do not declare playback_pause (the voice never pauses)"
    )
    args = parser.parse_args(argv)
    if not args.text and not args.wav:
        parser.error("give --text, --wav or both")
    if args.text and len(args.text) > 2000:
        parser.error("--text is at most 2000 characters")
    api_key = env.get("KHEELONA_API_KEY")
    if not api_key:
        print("Set KHEELONA_API_KEY to your Kheelona Voice API key (server side only).", file=sys.stderr)
        return 2
    if args.wav:
        try:
            read_pcm16_wav(args.wav)
        except (ValueError, OSError) as err:
            print(err, file=sys.stderr)
            return 2
    opts = RelayOptions(
        api_url=env.get("KHEELONA_API_URL", "https://api.voice.kheelona.com").rstrip("/"),
        api_key=api_key,
        voice_id=args.voice,
        languages=[code.strip() for code in args.languages.split(",") if code.strip()],
        text=args.text,
        wav_path=args.wav,
        out_path=args.out,
        interruptions=not args.no_interruptions,
        pause_playback=not args.no_pause,
    )
    try:
        asyncio.run(run_relay(opts))
    except KheelonaApiError as err:
        print(f"error {err.code} request_id {err.request_id or '-'}", file=sys.stderr)
        return 1
    except RelayError as err:
        session = f" session {err.session_id}" if err.session_id else ""
        print(f"error {err.code}{session} request_id {err.request_id or '-'}", file=sys.stderr)
        return 1
    except httpx.HTTPError as err:
        print(f"error api_unreachable_{type(err).__name__} request_id -", file=sys.stderr)
        return 1
    return 0


if __name__ == "__main__":
    sys.exit(main())
```

**python-server/wavio.py**

```python title="wavio.py"
"""Read and write the WAV files the relay uses (standard library only)."""

import wave
from pathlib import Path

HINT = "Convert it with: ffmpeg -ac 1 -ar 16000 -sample_fmt s16 -i in.wav out.wav"


def read_pcm16_wav(path: str | Path) -> tuple[int, bytes]:
    """Return (sample_rate, raw PCM bytes) of a 16-bit, mono, 16 kHz WAV, or raise ValueError saying what is wrong."""
    try:
        with wave.open(str(path), "rb") as w:
            channels, width, rate = w.getnchannels(), w.getsampwidth(), w.getframerate()
            problems = []
            if channels != 1:
                problems.append("stereo" if channels == 2 else f"{channels} channels")
            if width != 2:
                problems.append(f"{width * 8}-bit samples")
            if rate != 16000:
                problems.append(f"a sample rate of {rate} Hz")
            if problems:
                found = " and ".join(problems)
                raise ValueError(f"{path}: the WAV must be mono, 16-bit, 16 kHz, but it has {found}. {HINT}")
            return rate, w.readframes(w.getnframes())
    except (wave.Error, EOFError) as err:
        raise ValueError(f"{path}: not a PCM WAV file ({err}). {HINT}") from err


def write_wav(path: str | Path, pcm: bytes, rate: int) -> None:
    """Write raw 16-bit mono PCM as a WAV file."""
    with wave.open(str(path), "wb") as w:
        w.setnchannels(1)
        w.setsampwidth(2)
        w.setframerate(rate)
        w.writeframes(pcm)
```

**python-server/pyproject.toml**

```toml title="pyproject.toml"
[project]
name = "kheelona-example-python-server"
version = "0.1.0"
description = "Kheelona Voice example: create a session, mint client secrets for a browser, and relay audio over the WebSocket."
readme = "README.md"
license = { text = "Proprietary" }
requires-python = ">=3.12"
dependencies = [
    "fastapi==0.143.0",
    "httpx==0.28.1",
    "uvicorn==0.54.0",
    "websockets==17.2",
]

[dependency-groups]
dev = ["pytest==9.1.1", "ruff==0.17.0"]

[tool.uv]
# A sample you copy from, not a package: nothing to build or install.
package = false
# Supply-chain guard: resolve only releases at least 14 days older than 2026-10-30, the day these examples were written.
exclude-newer = "2026-10-16T00:00:00Z"

[tool.ruff]
line-length = 120
target-version = "py312"

[tool.ruff.lint]
select = ["E", "F", "I", "UP", "B", "SIM"]

[tool.pytest.ini_options]
testpaths = ["tests"]
pythonpath = ["."]
```

The examples are Kheelona's own and are given to partners privately. Contact the Kheelona team for licensing.

---

# Browser example

> A web page that talks to a voice with the browser SDK, and the small backend that creates its session. Every file, and how to run it.

Source: https://docs.voice.kheelona.com/examples/browser-basic/

A web page that talks to a Kheelona voice with the [browser SDK](https://docs.voice.kheelona.com/guides/browser-sdk.md), and the small backend it needs.
Copy it as a starting point. The [browser quickstart](https://docs.voice.kheelona.com/get-started/quickstart-browser.md) walks through it.

- **`public/main.js`** is a plain ES module with no build step. Start creates a session through your backend, then calls
  `KheelonaVoice.connect`. The listeners are attached **after** `await connect()`. Captions are set with `textContent`
  only. End calls `session.end()` and the text box calls `sendText`. If the browser refuses to play the voice, a
  **Tap to allow sound** button calls `session.resumeAudio()` directly in its click handler.
- **`public/controls.js`** wires the **Mute** and **Interrupt** buttons.
- **`public/errors.js`** holds the wording of the error line.
- **`server.ts`** is the whole backend, with only the Node standard library. It serves the page at `/`, the SDK's files at
  `/sdk/` and the route `POST /api/voice-session`, all from one origin, so there is no CORS. Your API key stays in this
  process.

You need Node 24 or newer, and the example folder and the SDK package: the examples you received from the Kheelona team, with the SDK package.

## Settings

| Variable | Default | Meaning |
|---|---|---|
| `KHEELONA_API_KEY` | none (exit 2 with a hint when missing) | Your API key. Server side only. |
| `KHEELONA_API_URL` | `https://api.voice.kheelona.com` | The API |
| `KHEELONA_VOICE_ID` | `lumi-dragon` | The voice |
| `HOST` / `PORT` | `127.0.0.1` / `5180` | Where the page and the backend listen |
| `ALLOWED_ORIGIN` | `http://localhost:5180` | The one page origin allowed to call the route (blank or `null` is refused at start) |
| `KHEELONA_REALTIME_ORIGINS` | `https://api.voice.kheelona.com wss://api.voice.kheelona.com` | The origins of the `connect` URLs, separated by spaces. The page's Content Security Policy allows connections to these and to nothing else. |

## Run

```sh
cd examples/browser-basic
npm install   # the SDK package file from the Kheelona team is already beside package.json
export KHEELONA_API_KEY=kv_live_…
npm start
```

Open `http://localhost:5180` and click **Start talking**. Open the page as `localhost`, which is what `ALLOWED_ORIGIN`
expects. If you serve it from another origin, set `ALLOWED_ORIGIN` to match, and add that origin in your console. See
[Security](https://docs.voice.kheelona.com/guides/security.md).

## Before you copy it

- **Add your own sign-in check** where `server.ts` says `Authenticate your user here`, and pass that user's opaque ID as
  `end_user_ref`.
- The route refuses a request whose `Origin` is missing or not `ALLOWED_ORIGIN` (403) and anything but JSON (415). It
  returns the session ID, the client secret and the connection addresses, and nothing else.
- Every response carries a strict Content Security Policy. See [Security](https://docs.voice.kheelona.com/guides/security.md#a-content-security-policy-for-your-page).
- The page shows the error code and, when it is retryable, "try again in N s". A retry creates a new session.

## The files

**browser-basic/server.ts**

```ts title="server.ts"
// The example's whole backend: it serves the page, the SDK files and the mint route from ONE origin (so no CORS), and
// it holds your API key. Run it with `npm start`, then open http://localhost:5180 (see Settings on the example's docs page).
import { readFile, realpath, stat } from 'node:fs/promises';
import { createServer as createHttpServer, type IncomingMessage, type Server, type ServerResponse } from 'node:http';
import { dirname, extname, resolve, sep } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';

const HERE = dirname(fileURLToPath(import.meta.url));
const REQUEST_TIMEOUT_MS = 15_000;

/** The settings (see Settings on the example's docs page). */
export interface BrowserEnv {
  apiKey: string;
  apiUrl: string;
  voiceId: string;
  /** The one page origin that may call the mint route. */
  allowedOrigin: string;
  /** Origins the page may connect to (the `connect` URLs the API returns), space separated, for the CSP. */
  realtimeOrigins: string;
  host: string;
  port: number;
}

/** Where the files come from. Tests pass their own folders. */
export interface ServerOptions {
  publicDir?: string;
  sdkDir?: string;
}

/** Reads the settings; throws an Error with a one-line hint when something is missing or wrong (exit code 2). */
export function readEnv(env: Record<string, string | undefined>): BrowserEnv {
  const apiKey = env['KHEELONA_API_KEY'];
  if (!apiKey) throw new Error('KHEELONA_API_KEY is not set. Create an API key in the Kheelona console and export it.');
  const allowedOrigin = (env['ALLOWED_ORIGIN'] ?? 'http://localhost:5180').trim();
  if (allowedOrigin === '' || allowedOrigin.toLowerCase() === 'null') {
    throw new Error("ALLOWED_ORIGIN must be your page's origin, for example https://app.example.com");
  }
  const realtimeOrigins = (env['KHEELONA_REALTIME_ORIGINS'] ?? 'https://api.voice.kheelona.com wss://api.voice.kheelona.com').trim();
  // The value goes into a response header: plain origins only.
  if (!/^[A-Za-z][A-Za-z0-9+.-]*:\/\/[^\s;,'"]+(?: [A-Za-z][A-Za-z0-9+.-]*:\/\/[^\s;,'"]+)*$/.test(realtimeOrigins)) {
    throw new Error('KHEELONA_REALTIME_ORIGINS must be origins separated by spaces, for example https://host wss://host');
  }
  const port = Number(env['PORT'] ?? '5180');
  if (!Number.isInteger(port) || port < 0 || port > 65535) throw new Error('PORT must be a port number.');
  return {
    apiKey,
    apiUrl: env['KHEELONA_API_URL'] || 'https://api.voice.kheelona.com',
    voiceId: env['KHEELONA_VOICE_ID'] || 'lumi-dragon',
    allowedOrigin,
    realtimeOrigins,
    host: env['HOST'] || '127.0.0.1',
    port,
  };
}

/** The strict policy the page runs under: this origin's files, and the Kheelona realtime origins to connect to. */
export function contentSecurityPolicy(realtimeOrigins: string): string {
  return [
    "default-src 'self'",
    "script-src 'self'",
    "style-src 'self'",
    `connect-src 'self' ${realtimeOrigins}`,
    "img-src 'self'",
    "media-src 'self'",
    "object-src 'none'",
    "base-uri 'self'",
    "form-action 'self'",
    "frame-ancestors 'none'",
  ].join('; ');
}

const TYPES: Readonly<Record<string, string>> = {
  '.html': 'text/html; charset=utf-8',
  '.js': 'text/javascript; charset=utf-8',
  '.css': 'text/css; charset=utf-8',
  '.map': 'application/json; charset=utf-8',
};

function send(res: ServerResponse, csp: string, status: number, type: string, body: string | Buffer, headers: Record<string, string> = {}): void {
  res.writeHead(status, {
    'Content-Type': type,
    'Content-Security-Policy': csp,
    'X-Content-Type-Options': 'nosniff',
    'Referrer-Policy': 'no-referrer',
    'Cache-Control': 'no-store',
    ...headers,
  });
  res.end(body);
}

const sendJson = (res: ServerResponse, csp: string, status: number, body: unknown, headers: Record<string, string> = {}): void => {
  send(res, csp, status, 'application/json', JSON.stringify(body), headers);
};

/**
 * Maps a request path to a file inside `root`, or `undefined`. The path is decoded first, then resolved, and refused
 * unless the result is still inside `root`: `..`, encoded dots and slashes, backslashes and NUL never get out.
 */
async function fileInside(root: string, requestPath: string): Promise<string | undefined> {
  let decoded: string;
  try {
    decoded = decodeURIComponent(requestPath);
  } catch {
    return undefined;
  }
  if (decoded.includes('\0') || decoded.includes('\\')) return undefined;
  const base = resolve(root);
  const file = resolve(base, `.${decoded}`);
  if (!file.startsWith(base + sep)) return undefined;
  if (!(extname(file) in TYPES)) return undefined;
  try {
    // A symlink must not lead out either: compare real paths. The root itself may be a link (pnpm links packages),
    // so it is resolved first and is the one explicitly trusted folder.
    const realFile = await realpath(file);
    if (!realFile.startsWith((await realpath(base)) + sep)) return undefined;
    return (await stat(realFile)).isFile() ? realFile : undefined;
  } catch {
    return undefined;
  }
}

async function serveStatic(req: IncomingMessage, res: ServerResponse, csp: string, dirs: Required<ServerOptions>): Promise<void> {
  const path = (req.url ?? '/').split('?')[0] ?? '/';
  const isSdk = path.startsWith('/sdk/');
  const file = await fileInside(isSdk ? dirs.sdkDir : dirs.publicDir, isSdk ? path.slice('/sdk'.length) : path === '/' ? '/index.html' : path);
  if (file === undefined) {
    sendJson(res, csp, 404, { error: { code: 'not_found' } });
    return;
  }
  send(res, csp, 200, TYPES[extname(file)] ?? 'application/octet-stream', req.method === 'HEAD' ? '' : await readFile(file));
}

/** The mint route: create the session with your API key, hand the page only what it needs. */
async function mintSession(env: BrowserEnv, req: IncomingMessage, res: ServerResponse, csp: string): Promise<void> {
  // Same-origin by design (no CORS headers). Only your own page may call this route: a request with no Origin, or
  // another one, never reaches Kheelona. JSON only, so a cross-site HTML form cannot spend your key.
  if (req.headers.origin !== env.allowedOrigin) {
    sendJson(res, csp, 403, { error: { code: 'origin_not_allowed' } });
    return;
  }
  if (!req.headers['content-type']?.toLowerCase().startsWith('application/json')) {
    sendJson(res, csp, 415, { error: { code: 'unsupported_media_type' } });
    return;
  }
  // Authenticate your user here: only signed-in users may start a session.
  // The request body is never read (this route takes no input), so its size and Content-Length don't matter.
  // One attempt only: to retry on 429/503, reuse one Idempotency-Key as examples/node-server does.
  const upstream = await fetch(`${env.apiUrl.replace(/\/+$/, '')}/v1/sessions`, {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${env.apiKey}`,
      'Content-Type': 'application/json',
      'Idempotency-Key': globalThis.crypto.randomUUID(),
    },
    body: JSON.stringify({
      voice_id: env.voiceId,
      languages: ['en', 'hi'],
      end_user_ref: 'demo-user', // replace with the signed-in user's opaque ID
    }),
    signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS),
  });
  if (!upstream.ok) {
    const body = (await upstream.json().catch(() => ({}))) as { error?: { code?: unknown; request_id?: unknown } };
    const code = plain(body.error?.code) ?? 'unknown_error';
    const requestId = plain(body.error?.request_id);
    console.error(`error ${code} request_id ${requestId ?? '-'}`);
    const retryable = upstream.status === 429 || upstream.status === 503;
    const retryAfter = upstream.headers.get('retry-after');
    sendJson(res, csp, retryable ? upstream.status : 502, { error: { code, ...(requestId ? { requestId } : {}) } }, retryable && retryAfter && /^\d+$/.test(retryAfter) ? { 'Retry-After': retryAfter } : {});
    return;
  }
  const created = (await upstream.json()) as {
    session_id: string;
    client_secret: { value: string };
    connect: { webrtc_url: string; websocket_url: string };
  };
  // Return only what the page needs: never the API key, and nothing else from the answer.
  sendJson(res, csp, 201, {
    sessionId: created.session_id,
    clientSecret: created.client_secret.value,
    endpoints: { webrtcUrl: created.connect.webrtc_url, websocketUrl: created.connect.websocket_url },
  });
}

/** A short identifier made of plain characters, or undefined: nothing else from an answer is repeated. */
function plain(value: unknown): string | undefined {
  return typeof value === 'string' && /^[A-Za-z0-9_.:-]{1,64}$/.test(value) ? value : undefined;
}

/** Creates the server (not listening yet): the page, the SDK files and the mint route on one origin. */
export function createServer(env: BrowserEnv, options: ServerOptions = {}): Server {
  const dirs: Required<ServerOptions> = {
    publicDir: options.publicDir ?? resolve(HERE, 'public'),
    sdkDir: options.sdkDir ?? resolve(HERE, 'node_modules/@kheelona/voice/dist'),
  };
  const csp = contentSecurityPolicy(env.realtimeOrigins);
  return createHttpServer((req, res) => {
    void (async () => {
      const path = (req.url ?? '/').split('?')[0];
      if (path === '/api/voice-session') {
        if (req.method !== 'POST') {
          sendJson(res, csp, 405, { error: { code: 'method_not_allowed' } }, { Allow: 'POST' });
          return;
        }
        await mintSession(env, req, res, csp);
      } else if (req.method === 'GET' || req.method === 'HEAD') {
        await serveStatic(req, res, csp, dirs);
      } else {
        sendJson(res, csp, 405, { error: { code: 'method_not_allowed' } }, { Allow: 'GET, HEAD' });
      }
    })().catch((error: unknown) => {
      // Only the kind of failure: an error message can echo a request.
      console.error(`error ${error instanceof Error ? error.name : 'unexpected'} request_id -`);
      if (!res.headersSent) sendJson(res, csp, 502, { error: { code: 'internal' } });
      else res.end();
    });
  });
}

if (process.argv[1] && pathToFileURL(process.argv[1]).href === import.meta.url) {
  let env: BrowserEnv;
  try {
    env = readEnv(process.env);
  } catch (error) {
    console.error((error as Error).message);
    process.exit(2);
  }
  createServer(env).listen(env.port, env.host, () => {
    console.log(`page on http://${env.host === '127.0.0.1' ? 'localhost' : env.host}:${String(env.port)} (page origin ${env.allowedOrigin})`);
  });
}
```

**browser-basic/public/index.html**

```html title="index.html"
<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <title>Kheelona Voice: browser example</title>
    <link rel="stylesheet" href="/style.css" />
  </head>
  <body>
    <main>
      <h1>Talk to a Kheelona voice</h1>
      <p class="status">State: <strong id="state">not connected</strong></p>
      <div class="row">
        <button id="start" type="button">Start talking</button>
        <button id="end" type="button" disabled>End</button>
        <button id="allow-sound" type="button" hidden>Tap to allow sound</button>
      </div>
      <div class="row">
        <button id="mute" type="button" aria-pressed="false" disabled>Mute</button>
        <button id="interrupt" type="button" disabled>Interrupt</button>
      </div>
      <p id="error" role="alert" hidden></p>
      <div id="captions" aria-live="polite"></div>
      <div class="row">
        <label for="say">Or type a message</label>
        <input id="say" type="text" maxlength="2000" autocomplete="off" disabled />
        <button id="send" type="button" disabled>Send</button>
      </div>
    </main>
    <script type="module" src="/main.js"></script>
  </body>
</html>
```

**browser-basic/public/main.js**

```js title="main.js"
/* global document, fetch */
// The page: mint a session on your server, connect with the SDK, show what is said.
// A plain ES module, no build step. The CSP allows only this origin: no inline scripts or styles.
import { KheelonaVoice } from '/sdk/index.js';
import { describeError, MintError } from '/errors.js';
import { wireControls } from '/controls.js';

const $ = (id) => document.getElementById(id);
let session = null;

async function connectToVoice() {
  // Your server creates the session with your API key. The page only ever gets a single-use client secret.
  const res = await fetch('/api/voice-session', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: '{}',
  });
  if (!res.ok) {
    const body = await res.json().catch(() => ({}));
    const retryAfter = Number(res.headers.get('Retry-After'));
    throw new MintError(body.error?.code ?? 'mint_failed', res.status, retryAfter > 0 ? retryAfter * 1000 : undefined);
  }
  const { sessionId, clientSecret, endpoints } = await res.json();
  // Call this from a click: the browser starts the audio and the microphone only after a user gesture.
  // A failed connect needs a new session (a new secret), so every attempt mints again.
  return KheelonaVoice.connect({ sessionId, clientSecret, endpoints, workletBaseUrl: '/sdk/' });
}

function listen(voice) {
  // Attach after `await connect()`: nothing is missed, and `state` first gives the current state.
  voice.on('state', (state) => {
    $('state').textContent = state;
  });
  voice.on('transcript', ({ text }) => addCaption('You', text));
  voice.on('replySentence', ({ text }) => addCaption('Voice', text));
  voice.on('error', (e) => {
    if (e.code === 'audio_unavailable') {
      // The browser refused to play the voice: ask for a tap (see the "Tap to allow sound" button).
      $('allow-sound').hidden = false;
    } else {
      showError(e);
    }
  });
  voice.on('ended', () => finish());
}

// Captions are set with textContent only: what people and the voice say is never read as HTML.
function addCaption(who, text) {
  const line = document.createElement('p');
  line.className = who === 'You' ? 'you' : 'voice';
  line.textContent = `${who}: ${text}`;
  $('captions').append(line);
}

function showError(e) {
  // Show the code (and when to try again); never log error.cause, it can quote connection details.
  $('error').textContent = describeError(e);
  $('error').hidden = false;
}

function setRunning(running) {
  $('start').disabled = running;
  $('end').disabled = !running;
  $('say').disabled = !running;
  $('send').disabled = !running;
  $('mute').disabled = !running;
  $('interrupt').disabled = !running;
}

function finish() {
  session = null;
  setRunning(false);
  $('allow-sound').hidden = true;
  // The next session starts unmuted.
  $('mute').setAttribute('aria-pressed', 'false');
  $('mute').textContent = 'Mute';
  $('state').textContent = 'ended';
}

$('start').addEventListener('click', async () => {
  $('error').hidden = true;
  setRunning(true);
  $('state').textContent = 'connecting';
  try {
    session = await connectToVoice();
    listen(session);
  } catch (e) {
    showError(e);
    finish();
    $('state').textContent = 'not connected';
  }
});

wireControls(() => session, { muteButton: $('mute'), interruptButton: $('interrupt') });

$('end').addEventListener('click', async () => {
  if (session) await session.end();
});

function sendTyped() {
  const text = $('say').value.trim();
  if (!session || text === '') return;
  // The caption comes back as a `transcript` event, like a spoken turn's.
  session.sendText(text);
  $('say').value = '';
}
$('send').addEventListener('click', sendTyped);
$('say').addEventListener('keydown', (e) => {
  if (e.key === 'Enter') sendTyped();
});

// A tap is the gesture the browser needs: call resumeAudio() directly in the handler, with no await before it.
$('allow-sound').addEventListener('click', () => {
  session?.resumeAudio();
  $('allow-sound').hidden = true;
});
```

**browser-basic/public/controls.js**

```js title="controls.js"
// Mute and interrupt buttons for a live session. Kept apart from main.js so it can be tested without a browser.

/** Wires a mute toggle and an interrupt button. `getSession` returns the live session, or null when there is none. */
export function wireControls(getSession, { muteButton, interruptButton }) {
  muteButton.addEventListener('click', () => {
    const session = getSession();
    if (!session) return;
    // The button holds the state: aria-pressed is "true" while the microphone is muted.
    const muted = muteButton.getAttribute('aria-pressed') !== 'true';
    session.mute(muted);
    muteButton.setAttribute('aria-pressed', String(muted));
    muteButton.textContent = muted ? 'Unmute' : 'Mute';
  });
  interruptButton.addEventListener('click', () => {
    // Stops the voice now, as if the person had started speaking.
    getSession()?.interrupt();
  });
}
```

**browser-basic/public/errors.js**

```js title="errors.js"
// What the page shows for an error. Kept apart from main.js (which imports the SDK) so it can be tested alone.

/** The mint route refused or failed: carries what the page needs to explain it. */
export class MintError extends Error {
  constructor(code, status, retryAfterMs) {
    super(code);
    this.name = 'MintError';
    this.code = code;
    this.retryable = status === 429 || status === 503;
    this.retryAfterMs = retryAfterMs;
  }
}

/** One line for the person: the code, a friendlier hint for a refused microphone, and "try again in N s" if it helps. */
export function describeError(e) {
  const code = typeof e?.code === 'string' ? e.code : 'error';
  const wait = e?.retryable && e?.retryAfterMs ? `, try again in ${String(Math.ceil(e.retryAfterMs / 1000))} s` : '';
  if (code === 'microphone_unavailable') {
    return `The microphone is blocked or missing. Allow it in the browser, then try again (${code})`;
  }
  return `Something went wrong (${code})${wait}`;
}
```

**browser-basic/public/style.css**

```css title="style.css"
:root { color-scheme: light dark; font-family: system-ui, sans-serif; }
body { margin: 0; }
main { max-width: 40rem; margin: 0 auto; padding: 1rem 16px 3rem; }
.row { display: flex; flex-wrap: wrap; gap: 0.5rem; align-items: center; margin: 1rem 0; }
button { font: inherit; padding: 0.5rem 1rem; }
input { font: inherit; flex: 1 1 12rem; padding: 0.5rem; }
#error { padding: 0.5rem 0.75rem; border: 2px solid currentColor; border-radius: 0.25rem; }
#captions p { margin: 0.4rem 0; }
#captions .you { opacity: 0.75; }
```

The examples are Kheelona's own and are given to partners privately. Contact the Kheelona team for licensing.

---

# curl example

> Create a session with one curl command. The script, the settings and what the answer holds.

Source: https://docs.voice.kheelona.com/examples/curl/

The shortest way to see the API answer: one `POST /v1/sessions` with curl.

```sh
export KHEELONA_API_KEY=kv_live_…
./create-session.sh
```

It prints the session: the `session_id`, a single-use `client_secret`, the `connect` addresses, the pinned voice and the
limits. Give the secret to a client within its lifetime, about a minute. The Node, Python and browser examples are
clients.

## Settings

| Variable | Default | Meaning |
|---|---|---|
| `KHEELONA_API_KEY` | none (the script exits with a hint when missing) | Your API key |
| `KHEELONA_API_URL` | `https://api.voice.kheelona.com` | The API |
| `KHEELONA_VOICE_ID` | `lumi-dragon` | The voice |

## Notes

- The key goes to curl on **standard input** (`-H @-`), never as an argument, so it is not in a process list or in your
  shell history. It needs curl 7.76 or newer and `uuidgen`.
- `Idempotency-Key` is one fresh UUID per session. Repeating a request with the same key returns the same session with a
  new client secret. See [Sessions](https://docs.voice.kheelona.com/concepts/sessions.md).
- The answer holds a client secret. Don't paste it into chat, tickets or logs.
- To end a session that you created and never connected: `POST /v1/sessions/{id}/end` with the same `Authorization`
  header.

## The file

**curl/create-session.sh**

```bash title="create-session.sh"
#!/usr/bin/env bash
# Creates a Kheelona Voice session with curl and prints the answer (JSON: session_id, client_secret, connect, ...).
#   KHEELONA_API_KEY=... ./create-session.sh
# The key reaches curl on its standard input, never on the command line, so it is in no process list.
# The answer holds a single-use client secret: treat it like a password, and never log it.
set -euo pipefail

: "${KHEELONA_API_KEY:?Set KHEELONA_API_KEY to your API key (from the console)}"

API_URL="${KHEELONA_API_URL:-https://api.voice.kheelona.com}"
VOICE_ID="${KHEELONA_VOICE_ID:-lumi-dragon}"

# One Idempotency-Key per logical session: repeating the request with the same key returns the same session.
printf 'Authorization: Bearer %s\n' "$KHEELONA_API_KEY" | curl -sS --fail-with-body \
  -H @- \
  -H 'Content-Type: application/json' \
  -H "Idempotency-Key: $(uuidgen)" \
  -X POST "$API_URL/v1/sessions" \
  -d "{\"voice_id\": \"$VOICE_ID\", \"languages\": [\"en\", \"hi\"]}"
echo
```

The examples are Kheelona's own and are given to partners privately. Contact the Kheelona team for licensing.

---

# @kheelona/voice

Source: https://docs.voice.kheelona.com/reference/sdk/readme/

The Kheelona Voice browser SDK: talk to a Kheelona voice from a web page, over WebRTC with a WebSocket fallback.

Start with [KheelonaVoice.connect](https://docs.voice.kheelona.com/reference/sdk/classes/kheelonavoice.md#connect), which returns a [VoiceSession](https://docs.voice.kheelona.com/reference/sdk/interfaces/voicesession.md). See the Browser SDK guide
(`/guides/browser-sdk/`) for a complete walkthrough with code.

## Classes

- [KheelonaVoice](https://docs.voice.kheelona.com/reference/sdk/classes/kheelonavoice.md)
- [KheelonaVoiceError](https://docs.voice.kheelona.com/reference/sdk/classes/kheelonavoiceerror.md)

## Interfaces

- [ConnectOptions](https://docs.voice.kheelona.com/reference/sdk/interfaces/connectoptions.md)
- [SessionErrorInfo](https://docs.voice.kheelona.com/reference/sdk/interfaces/sessionerrorinfo.md)
- [SessionEvents](https://docs.voice.kheelona.com/reference/sdk/interfaces/sessionevents.md)
- [SessionReadyEvent](https://docs.voice.kheelona.com/reference/sdk/interfaces/sessionreadyevent.md)
- [VoiceSession](https://docs.voice.kheelona.com/reference/sdk/interfaces/voicesession.md)

## Type Aliases

- [ClientEvent](https://docs.voice.kheelona.com/reference/sdk/type-aliases/clientevent.md)
- [Emotion](https://docs.voice.kheelona.com/reference/sdk/type-aliases/emotion.md)
- [ErrorCode](https://docs.voice.kheelona.com/reference/sdk/type-aliases/errorcode.md)
- [LanguageStage](https://docs.voice.kheelona.com/reference/sdk/type-aliases/languagestage.md)
- [SdkErrorCode](https://docs.voice.kheelona.com/reference/sdk/type-aliases/sdkerrorcode.md)
- [ServerEvent](https://docs.voice.kheelona.com/reference/sdk/type-aliases/serverevent.md)
- [SessionState](https://docs.voice.kheelona.com/reference/sdk/type-aliases/sessionstate.md)

## Variables

- [ERROR\_CODES](https://docs.voice.kheelona.com/reference/sdk/variables/error_codes.md)