Captions

Kheelona sends you the text of the conversation as it happens, so you can show captions or drive an animation. It sends it live and does not keep it. See Live text for the events.

What you get

What	Event	Browser SDK event
What the person said	transcript.final, once the turn is understood	transcript
What the voice says	reply.sentence, one sentence at a time, as that sentence starts to play	replySentence

A person’s text arrives once per turn, after they have finished. There is no word-by-word text while they speak. The language field says which language was heard. languageStage is supported, beta or alpha, and languageBeta is true for the last two, where recognition is less reliable. Show a spoken turn in that case as an approximation, and label it “Beta” or “Alpha”.

A reply’s sentences come one after another, each as its audio starts. The index counts from 0. A sentence’s emotion is how it is spoken. It is a hint for an animation, such as a face, and it is not text: the text has no tags in it. Today the values are happy, excited, sad, giggly and soft: happy is the voice’s normal warm delivery, and the others mark a sentence spoken with more excitement, more sadness, a laugh or a hushed voice. Older values such as calm and surprised are no longer sent. New emotion names can appear, so treat one you don’t know as neutral.

Show them safely

Set captions as text, never as HTML. What the person says and what the voice replies are never trusted markup. The example does this with textContent:

public/main.js

// Captions are set with textContent only: what people and the voice say is never read as HTML.
function addCaption(who, text) {
  const line = document.createElement('p');
  line.className = who === 'You' ? 'you' : 'voice';
  line.textContent = `${who}: ${text}`;
  $('captions').append(line);
}

When the voice is cut off

A sentence is shown when it starts to play. If the person interrupts, the sentence may have been heard only in part. The interrupted event has heardMs, and the reply then ends. You may mark the last caption as cut off, for example with an ellipsis.

Typed turns

A typed turn (sendText) also gets a transcript event, with the text you sent. Show captions from that event and you treat spoken and typed turns the same way. Don’t add the typed text to the captions yourself as well, or it shows twice.

What not to do

• Don’t store captions without a reason and a notice. Kheelona keeps no conversation text, so the captions you receive are the only copy. What you do with them is your responsibility, including telling your users and getting consent, especially for children. See the data and privacy policy.
• Don’t send captions to third parties you have not told your users about.
• Don’t log them in your server logs by accident. Treat them like any other personal conversation.

Next

• Browser SDK: the events.
• Server events reference: every field.