Latency

The target

Our target is that the voice typically answers within 3 seconds of the person finishing, counted from the end of their speech to the first sound of the reply. We aim to keep the 95th percentile, nineteen replies in twenty, under 4.5 seconds. Kheelona measures this on the service and reports it.

This is a target, not a guarantee. It is not in any contract, and a single reply can be slower. During the partner pilot Kheelona is best effort.

What is in those seconds

From the end of speech to the first audio, the service:

1. decides the person has finished (a short wait to make sure they are not only pausing);
2. turns the speech into text;
3. checks the text for safety;
4. writes the first sentence of the reply;
5. checks that sentence;
6. speaks it.

Everything after the first sentence happens while the first sentence plays, so a long reply does not make you wait longer.

What changes it

• Distance. Kheelona’s servers are in India. A person far from India has a longer network trip each way, and so a longer wait. The text of each reply is written in a US region of our cloud provider, which adds a little time, or, for a voice set up for India, in Mumbai, where how soon a voice starts depends on how it is set up. See Data and privacy.
• The first sentence. A reply that begins with a long sentence takes longer to start than one that begins with a short one.
• Load. When many people are talking at once, replies can be slower. When Kheelona is full, a new session is refused with capacity_exhausted instead. See Limits.
• Your connection. A slow or unstable network between the person and Kheelona adds to the wait on either transport.

Measure it yourself

Time the gap between input.speech_stopped and the first reply.sentence on your page:

• In the browser SDK, listen to the raw event stream for both events, and subtract the receive times.
• On a WebSocket, do the same with the events as they arrive.

This is your own measure. It leaves out the service’s wait for the end of speech and includes your network each way, so it is not the same number as the target. Use it to compare, over time and between devices. Typed turns have no speech and no input.speech_stopped.

Keep it feeling quick

• Show the thinking state so the person sees that something is happening.
• Start playing at once: the SDK does.
• Don’t add your own waiting before you connect. Create the session just before the person starts, not long before: an unused client secret expires after about a minute.