Safety levels

Every voice has a safety level, set by Kheelona: child or general. You can see it on the Voices page of your console and in GET /v1/voices. You cannot change it, and the instructions of a session cannot switch it off.

The levels

	child	general
What the person says	Checked in every category	Checked for the most serious content
What the voice is about to say	Every sentence, in every category	Checked for the most serious content
Meant for	Voices for children	Voices for older users

The child level is the strictest. It checks each sentence of the voice’s reply before it is spoken, not only what the person says. Use a child voice for a child.

A typed turn (input.text) goes through the same checks. On a child voice, what is said aloud is checked a little more strictly than typed text, because speech recognition can change the words.

Enforced outside the prompt

Safety does not depend on asking the voice nicely. The checks run outside the voice’s instructions, as separate steps on the text of the turns, in all 11 languages. The voice’s instructions do not control them, and your instructions cannot remove them. You don’t switch them on, and you can’t switch them off.

When the main check is slow

Kheelona’s safety rules have more than one part. If the part that judges the meaning of a sentence cannot answer in time, Kheelona’s own safety rules still apply to that turn or sentence. That check is then less thorough, and the turn is marked as such on Kheelona’s side. You get no event for it. Don’t assume a turn without a safety event was checked in every way.

What happens when a check acts

There are two places a check can act, and two things that can happen.

• On what the person said (stage input): the voice does not answer it. It says a short, friendly line in the person’s language that changes the subject. The safety event has the action redirected, and the reply ends with reply.done and the reason safety.
• On what the voice was about to say (stage output): that sentence is dropped and never spoken. If nothing had been said yet, the voice says its friendly line instead and the reply ends. Otherwise the reply ends after the sentences already spoken. The safety event has the action sentence_dropped.
• A third stage, provider, is for a block reported by the system that writes the reply. You may never see it.

A gentle line for a child who needs a grown-up

On a child voice, when what the person says is about hurting themselves, sexual content or violence (this covers a child saying that they are being hurt), the voice does not change the subject. It answers with a calm, caring line that tells the child to talk to a grown-up they trust, in their language, and says it is still there to play. It is the same in every child voice. The event is the same safety event with the category self_harm, sexual or violent.

Harmless replies are sometimes stopped

In every language, the checks sometimes stop a harmless reply, a few in every hundred in our tests. The voice then says its short friendly line, or ends the reply early. The checks sometimes mistake a harmless reply for a harmful one, and stop it.

On a child voice, a harmless question asked aloud is occasionally answered with the gentle “talk to a grown-up you trust” line instead of the friendly one. This is rare, and it happens in every language.

Beta languages

Of the beta and alpha languages, Telugu, Kannada and Malayalam have less precise safety checks, and they stop a few harmless sentences. On a child voice, such a sentence may get the gentle “talk to a grown-up you trust” reply instead of an answer. Otherwise the voice changes the subject, or drops a sentence of its reply. The checks stay on, even when that stops a harmless sentence. We measure them again when they improve.

The safety event

{ "type": "safety", "turn": 3, "stage": "input", "category": "violent", "action": "redirected" }

Field	Values
turn	The turn number.
stage	input, output or provider.
category	A short label for the kind of content. It is never the words themselves.
action	redirected or sentence_dropped.

One event is sent per block, with one category. If several apply, the care categories self_harm, sexual and violent come first. The labels today are:

• violent, illegal, sexual, pii, self_harm, unethical, political, copyright, jailbreak, unknown;
• profanity, hate, romance, scary, dangerous_activity, substances, personal_data;
• unsafe, when nothing more specific applies;
• unchecked, when a check could not be completed and the text was stopped to be safe.

The set can grow. Treat a label you don’t know as “other”. Kheelona does not publish the checks themselves.

What to do with the event

The event is yours to use. For example: log it with the session ID, tell a parent or a moderator, or show a note in the page. Kheelona does not keep the conversation, so the event is your record. In the browser SDK it is the safety event. On a WebSocket it is the safety message. See Live text events and the server events reference.

If you decide that something in your product needs a human, such as a child who may be in danger, that is your responsibility. Plan who reads these events before you go live.