Server events

Server events: sent by Kheelona Voice to the client, as JSON text frames. Every event has a type field.

Every server event also carries:

Field	Type	Required	Description	Rules
event_id	EventId	yes	Server-generated ID.	pattern ^evt_[0-9A-HJKMNP-TV-Z]{26}$
ts	string	yes	When the event was created: RFC 3339 UTC with milliseconds.	format date-time

Event	When
session.ready	The connection is accepted.
state	The conversation state changed.
input.speech_started	Voice activity detection heard speech.
input.speech_stopped	The turn detector decided the turn is complete.
transcript.final	What was heard (live text; never stored).
reply.started	The voice began a reply.
reply.sentence	A sentence’s audio started being delivered. text has no emotion tags (live text; never stored).
reply.done	The reply finished.
interrupted	The person talked over the voice. The client MUST flush queued reply audio immediately.
reply.paused	Only for a client that declared playback_pause (client.capabilities), and only for a reply it saw start: the voice paused its own playback because the person started to speak. The client SHOULD pause its playback buffer now. Not a listening cue. The pause ends at the first of reply.resumed, interrupted, reply.done or the next reply.started.
reply.resumed	Only after reply.paused: the pause ended and the same reply plays on. The client resumes its playback buffer.
safety	A safety rule fired. Category only, never text.
session.warning	A limit is close: 60 s before max duration, 20 s before the idle timeout.
error	A problem. If fatal, the session ends next.
session.ended	The last event before the connection closes.
pong	Answer to ping.

session.ready

The connection is accepted.

Field	Type	Required	Description	Rules
session_id	SessionId	yes	The session.	pattern ^ses_[0-9A-HJKMNP-TV-Z]{26}$
voice	VoiceInfo	yes	The voice and the version this session pinned.
languages	array of Lang	yes	The session’s declared languages (default ["en","hi"]).
audio	AudioInfo	yes	Audio formats in each direction.
limits	Limits	yes	Duration and idle limits.
interruptions	boolean	yes	Whether the person can interrupt the voice.

state

The conversation state changed.

Field	Type	Required	Description	Rules
value	StateValue	yes	The new state.	one of listening, thinking, speaking

input.speech_started

Voice activity detection heard speech.

Field	Type	Required	Description	Rules
turn	integer	yes	The turn number.	format uint32; min 0

input.speech_stopped

The turn detector decided the turn is complete.

Field	Type	Required	Description	Rules
turn	integer	yes	The turn number.	format uint32; min 0

transcript.final

What was heard (live text; never stored).

Field	Type	Required	Description	Rules
turn	integer	yes	The turn number.	format uint32; min 0
text	string	yes	The transcript.
language	Lang	yes	The language it was heard in.	one of en, hi, mr, bn, as, gu, pa, ta, te, kn, ml
language_beta	boolean	yes	True for every language that is not supported (beta or alpha; kept for older clients).
language_stage	LangStage	yes	The language’s stage: supported, beta or alpha.	one of supported, beta, alpha

reply.started

The voice began a reply.

Field	Type	Required	Description	Rules
turn	integer	yes	The turn being answered.	format uint32; min 0
reply_id	ReplyId	yes	The reply.	pattern ^rep_[0-9A-HJKMNP-TV-Z]{26}$

reply.sentence

A sentence’s audio started being delivered. text has no emotion tags (live text; never stored).

Field	Type	Required	Description	Rules
reply_id	ReplyId	yes	The reply.	pattern ^rep_[0-9A-HJKMNP-TV-Z]{26}$
index	integer	yes	The sentence index within the reply (also in the audio frame header).	format uint16; min 0; max 65535
text	string	yes	The sentence as caption text.
emotion	Emotion	yes	The emotion it’s spoken with.	one of excited, happy, giggly, calm, soft, sad, surprised

reply.done

The reply finished.

Field	Type	Required	Description	Rules
reply_id	ReplyId	yes	The reply.	pattern ^rep_[0-9A-HJKMNP-TV-Z]{26}$
reason	ReplyDoneReason	yes	Why it finished.	one of completed, interrupted, safety, error
sentences_spoken	integer	yes	Sentences actually spoken.	format uint16; min 0; max 65535

interrupted

The person talked over the voice. The client MUST flush queued reply audio immediately.

Field	Type	Required	Description	Rules
reply_id	ReplyId	yes	The interrupted reply.	pattern ^rep_[0-9A-HJKMNP-TV-Z]{26}$
heard_ms	integer	yes	Milliseconds of it the person heard.	format uint32; min 0

reply.paused

Only for a client that declared playback_pause (client.capabilities), and only for a reply it saw start: the voice paused its own playback because the person started to speak. The client SHOULD pause its playback buffer now. Not a listening cue. The pause ends at the first of reply.resumed, interrupted, reply.done or the next reply.started.

Field	Type	Required	Description	Rules
reply_id	ReplyId	yes	The paused reply.	pattern ^rep_[0-9A-HJKMNP-TV-Z]{26}$

reply.resumed

Only after reply.paused: the pause ended and the same reply plays on. The client resumes its playback buffer.

Field	Type	Required	Description	Rules
reply_id	ReplyId	yes	The resumed reply.	pattern ^rep_[0-9A-HJKMNP-TV-Z]{26}$

safety

A safety rule fired. Category only, never text.

Field	Type	Required	Description	Rules
turn	integer	yes	The turn.	format uint32; min 0
stage	SafetyStage	yes	Where the rule fired.	one of input, output, provider
category	string	yes	What kind of content fired, as a stable snake_case label: from the classifier violent, illegal, sexual, pii, self_harm, unethical, political, copyright, jailbreak, unknown, unsafe (no category named); from the blocklists also profanity, hate, romance, scary, dangerous_activity, substances, personal_data. A free string: new labels may be added, so treat an unknown one as “other”.
action	SafetyAction	yes	What the voice did.	one of redirected, sentence_dropped

session.warning

A limit is close: 60 s before max duration, 20 s before the idle timeout.

Field	Type	Required	Description	Rules
code	WarningCode	yes	Which limit.	one of max_duration_soon, idle_soon
seconds_left	integer	yes	Seconds until it’s reached.	format uint32; min 0

error

A problem. If fatal, the session ends next.

Field	Type	Required	Description	Rules
code	ErrorCode	yes	The error code.	one of invalid_request, invalid_audio, unauthenticated, secret_expired, forbidden, origin_not_allowed, voice_not_found, session_not_found, capacity_exhausted, partner_limit_reached, rate_limited, provider_unavailable, internal, not_found, conflict, unavailable
message	string	yes	A human-readable explanation (no conversation content).
retryable	boolean	yes	Whether retrying can help.
fatal	boolean	yes	Whether the session ends next.

session.ended

The last event before the connection closes.

Field	Type	Required	Description	Rules
reason	EndReason	yes	Why it ended.	one of client_ended, partner_ended, max_duration, idle_timeout, disconnected, server_shutdown, provider_failure, suspended, service_stopped, error
duration_ms	integer	yes	Connected time in milliseconds.	format uint64; min 0
turns	integer	yes	Turns in the session.	format uint32; min 0

pong

Answer to ping.

No fields besides type.

Types

AudioFormat

PCM16 little-endian mono at one of three sample rates.

Value	Description
pcm16_16k	16,000 Hz (the input default).
pcm16_24k	24,000 Hz (the output default).
pcm16_48k	48,000 Hz.

AudioInfo

Audio formats chosen at session creation.

Field	Type	Required	Description	Rules
input_format	AudioFormat	yes	Client → server audio.	one of pcm16_16k, pcm16_24k, pcm16_48k
output_format	AudioFormat	yes	Server → client audio.	one of pcm16_16k, pcm16_24k, pcm16_48k

Emotion

The emotion a sentence is spoken with.

Value	Description
excited	Excited
happy	Happy: the first sentence’s default when the model gives no tag.
giggly	Giggly: only when the person is laughing.
calm	Calm
soft	Soft
sad	Sad
surprised	Surprised

EndReason

Why a session ended.

Value	Description
client_ended	The client sent session.end.
partner_ended	The partner called POST /v1/sessions/{id}/end.
max_duration	The maximum duration was reached.
idle_timeout	Nobody spoke for the idle timeout.
disconnected	The connection dropped.
server_shutdown	The server is shutting down.
provider_failure	Three provider failures in a row.
suspended	The partner was suspended mid-session.
service_stopped	The voice service was switched off.
error	Another error.

ErrorCode

A machine-readable error code.

Value	Description
invalid_request	Fix the request.
invalid_audio	A malformed audio frame; the frame is dropped.
unauthenticated	Missing or invalid API key or client secret.
secret_expired	The client secret is older than 60 s or already used: create a new session.
forbidden	The key lacks the scope, or the partner is suspended.
origin_not_allowed	The web origin isn’t allowed for this partner.
voice_not_found	Unknown or retired voice, or not enabled for this partner.
session_not_found	No such session.
capacity_exhausted	The whole service is full; retry after the hinted seconds.
partner_limit_reached	The partner is at its maximum concurrent sessions.
rate_limited	Too many API calls.
provider_unavailable	A processing step failed. Non-fatal for one turn; fatal after 3 failures in a row.
internal	Our bug; the request ID finds it.
not_found	REST only: the resource doesn’t exist, or isn’t visible to your API key (a resource that belongs to someone else is not_found, so its existence isn’t revealed).
conflict	REST only: the request conflicts with the current state of the resource; change the request and retry.
unavailable	REST only: the API is temporarily unavailable; retry later.

EventId

string, pattern ^evt_[0-9A-HJKMNP-TV-Z]{26}$.

Lang

A supported language, as its ISO 639-1 code. Odia is not supported.

Value	Description
en	English
hi	Hindi
mr	Marathi (beta)
bn	Bengali (beta)
as	Assamese (alpha)
gu	Gujarati
pa	Punjabi (beta)
ta	Tamil
te	Telugu (beta)
kn	Kannada (beta)
ml	Malayalam (alpha)

LangStage

How far a language is tested: supported, beta or alpha.

Value	Description
supported	Tested end to end.
beta	Works, with more recognition mistakes.
alpha	Early: spoken turns are often not recognised.

Limits

Session limits.

Field	Type	Required	Description	Rules
max_duration_s	integer	yes	Maximum session length in seconds.	format uint32; min 0
idle_timeout_s	integer	yes	Idle timeout in seconds.	format uint32; min 0

ReplyDoneReason

Why a reply finished.

Value	Description
completed	Every sentence was spoken.
interrupted	The person talked over it.
safety	A safety rule ended it.
error	A provider failed.

ReplyId

string, pattern ^rep_[0-9A-HJKMNP-TV-Z]{26}$.

SafetyAction

What the voice did when a safety rule fired.

Value	Description
redirected	Spoke the voice’s fallback line instead.
sentence_dropped	Dropped the sentence.

SafetyStage

Where a safety rule fired.

Value	Description
input	On the person’s turn, before the reply model.
output	On a reply sentence, before speech.
provider	Reported by a provider.

SessionId

string, pattern ^ses_[0-9A-HJKMNP-TV-Z]{26}$.

StateValue

Conversation state.

Value	Description
listening	Waiting for the person.
thinking	From the end of the person’s turn until the first reply audio.
speaking	Reply audio is being delivered.

VoiceInfo

The voice a session uses.

Field	Type	Required	Description	Rules
id	VoiceSlug	yes	Public voice ID, e.g. lumi-dragon.	pattern ^[a-z][a-z0-9-]{2,39}$
version	integer	yes	The pinned version (from 1).	format uint32; min 0
name	string	yes	Display name.

VoiceSlug

string, pattern ^[a-z][a-z0-9-]{2,39}$.

WarningCode

Which limit a session.warning is about.

Value	Description
max_duration_soon	Maximum duration in 60 s.
idle_soon	Idle timeout in 20 s.