Summary: Building a bilingual AI avatar for a UAE government library, native Arabic speakers caught three things our tests missed: the register, the pronunciation and the mouth. Formal Arabic with a voweled Emirati greeting, a speech-to-text check on every Arabic TTS phrase, and a geometric fix to the jaw took the demo to 20 correct answers out of 20.
“She says Salm.”
That was the first comment from an Arabic-speaking reviewer on our library assistant. The avatar is called Salma. Her own voice was getting her name wrong, and nobody on the build side had heard it.
The brief looked simple. A government library wanted an assistant that greets visitors in Arabic and English, answers questions about the collection from the catalogue, and appears on screen as a person rather than a chat window. Once it worked, native speakers took it apart, and they were right every time.
Pick one Arabic register and keep it
Our first prompt asked the model for a light Emirati dialect in conversation and Modern Standard Arabic for facts. It sounded like a sensible compromise. One reviewer called the result “almost formal Arabic with some local words”. Another called it “Google Translate”.
Part of the problem is the voice. No Azure prebuilt voice speaks real Emirati dialect. The ar-AE voices have an Emirati accent, but they were trained mostly on formal Arabic, so dialect words come out wrong. A word such as أدوّر was mispronounced until we replaced it with ابحث.
What held up was formal Arabic throughout, with a fixed, fully voweled Emirati greeting and goodbye. With that register the full evaluation passed 20 of 20 questions on the live system on 12 October.
A real dialect voice is possible. It means a Custom Neural Voice trained on an Emirati speaker: a Limited Access application, recorded consent, studio sessions and several weeks. We now offer that as a persona option for a funded project, not as a demo feature.
Check Arabic TTS by listening with a machine
You cannot wait for a native speaker every time you change a sentence. So we let speech-to-text do the listening: synthesise the phrase with the Arabic TTS voice, transcribe it back, and compare.
That turned “she says Salm” into a measured problem within minutes. The ar-QA-AmalNeural voice rendered سلمى as “سلم” in every vowelling we tried. Written as سَلْما with a final alif, both Amal and ar-AE-FatimaNeural were transcribed back as سلمى. IPA phoneme tags fixed it for Fatima but not for Amal, so plain respelling turned out to be more reliable than SSML phonemes across voices.
Dates and page numbers needed the same care. The voice read ٧٧١ م as “metre” until the spoken text said ميلادي.
The pattern we kept is a voice-only rewrite step. The screen shows normal text; the voice receives voweled or respelled forms. Rewrite whole phrases, never single words: كتب عن can mean “books about” or “he wrote about”, and a word-level fix picks the wrong one. Let the model add vowel marks only where a word is ambiguous.
The mouth: why the teeth did not move
The second complaint was harder: “missing upper teeth and a defective lower jaw”.
We had bought a commercial scanned character and retargeted it to the skeleton that browser lip-sync libraries expect. Scanned characters move the lower teeth and tongue with a jaw bone. The retargeting dropped that bone, and the mouth shapes moved only the lips and chin. When we measured, the teeth and tongue moved by exactly zero in every mouth shape, including “jaw open”. The lips opened over teeth that stayed put.
No texture or shader fixes that. The fix is geometric:
- Swing the lower teeth and tongue as one rigid piece about a hinge about 12.8 cm behind the chin, up to 13.7 degrees at full jaw open, scaled by how far the chin travels in each shape.
- Lower the upper teeth by 2.2 mm.
- For each viseme, lift the upper lip until the right amount of tooth shows: about 2 to 5 mm on open vowels, less on rounded ones.
- Darken the inside of the mouth at runtime.
Close-ups after the fix show both rows of teeth, no gum, and no tooth through closed lips. For any bought avatar, check whether the teeth move in “jaw open” before you do anything else.
Keep the voice and the data in the UAE
For a government buyer, where the voice is generated matters as much as how it sounds. The Azure neural voices we used run in UAE North and send viseme events, which drive the lip-sync. Third-party voices may sound more local, but they process outside the UAE and do not send visemes, so the mouth has to be driven from the audio instead.
The answers themselves come from the library’s own catalogue through retrieval, and the supporting services run in-country. Where a model must also run in-country, the cost changes: we wrote about that in why a sovereign AI assistant is a fixed monthly fee.
What to check before you commission an AI avatar
- Ask which Arabic register it will speak, and hear it from a native speaker in your own organisation before anyone sees a demo.
- Ask how pronunciation is tested. “We listened to it” does not scale; a speech-to-text round trip does.
- Ask to see a close-up of the mouth on open vowels.
- Ask where the voice is generated and whether it sends visemes.
Frequently asked questions
Which Arabic voice should a UAE government AI avatar use?
Formal Arabic in an ar-AE voice, with a fixed, fully voweled Emirati greeting and goodbye. In our tests light dialect read as neither dialect nor formal Arabic to native reviewers.
Can an AI avatar speak real Emirati dialect?
Yes, with a custom neural voice trained on an Emirati speaker. It needs a Limited Access approval, recorded consent and several weeks of work, so it belongs in a funded project rather than a demo.
How do you check Arabic TTS pronunciation without a native speaker?
Synthesise the phrase, transcribe it back with speech-to-text and compare. Fix problems by respelling whole phrases for the voice only, and confirm the result with a native speaker at the end.
Why do an avatar’s teeth stay still when it talks?
Retargeting a scanned character to a browser lip-sync skeleton usually drops its jaw bone, so only the lips and chin move. The fix is to move the lower teeth and tongue as a rigid piece about a hinge behind the chin.
Next step
If you are planning an Arabic AI assistant or avatar, we can show you this one and how it answers from your own content. See our enterprise AI work, or get in touch.
