A bilingual AI avatar for government services that never leaves the UAE

Summary: A government programme wants one presenter who speaks Arabic and English, on the website and in the mobile app, with the voice and the words never leaving the UAE. The first demo runs on an avatar service hosted abroad, and security says no. What follows is how to build the avatar from three in-region parts: a 3D persona rendered in the visitor’s browser, lip-sync from Azure Speech viseme events, and one custom voice for both languages hosted in UAE North.

The brief fit on a sticky note. “One face. Speaks Arabic and English. Web and the app. Two to three seconds to answer.” The programme owner at a federal entity in the UAE had stuck it to the whiteboard in the first workshop, and for a week it looked like the easy part of the project. Avatars are a product category now. You pick one.

The demo went well, too. A polished 3D character, natural lip movement, a warm voice. Then the security lead asked where the character was rendered and where the speech was synthesised, and the vendor’s answer was a data centre on another continent. The clarifications for this programme required every component that touched a visitor’s voice or words to run inside the entity’s own Azure environment. The demo was over.

We’d been through a version of this before. The avatar service we’d used for a client demo the year before had the same problem, so we already knew the answer wasn’t a different product. It was taking the avatar apart.

An avatar is three things pretending to be one

Strip the marketing and a talking avatar is a face, a mouth and a voice. Where each one runs is a separate decision.

Once you see it that way, “which avatar vendor” stops being the question. The question is which of the four parts you can source in-region today, and how you fill the gap for the rest.

Render the face on the visitor’s device

The persona is a 3D character rendered in the browser with PlayCanvas, an open-source engine that runs on WebGPU with a WebGL2 fallback. The same component embeds in the entity’s existing mobile app. There’s no render farm. A visitor’s phone does the work a hosted avatar service would otherwise do on a GPU somewhere you can’t point to on a map.

Two practical consequences. The look, dress and framing of the character are the customer’s decision, approved in UX design, and we can start from persona designs already produced for another UAE government AI trainer. And until the approved persona is modelled, a generic character with the same mouth rig fronts the agents, so user testing starts on schedule instead of waiting for art.

For a flagship kiosk or launch, a photoreal Unreal Engine MetaHuman can be pixel-streamed from GPU virtual machines in Azure UAE North. That’s the pattern we run for digital-twin clients. It costs GPU hours per session, which is why it’s an option, not the default.

Lip-sync from viseme events, and what Arabic gets

Azure neural text-to-speech can return, alongside the audio, a stream of viseme events. Microsoft’s viseme documentation describes 22 viseme IDs, each a mouth position for a group of phonemes, delivered with an audio offset in ticks of 100 nanoseconds. The browser plays the audio and moves the character’s mouth on those timestamps. No lip-sync model runs anywhere; the shapes come with the sound.

For a bilingual persona, one row in the docs decides the design. The same service offers richer outputs, blend shapes for 3D faces and SVG for 2D, but the language support table lists every Arabic locale, ar-AE included, with viseme ID only. Blend shapes are listed for English and a set of European locales; SVG is en-US only. So the design rigs the mouth on viseme IDs for both languages and treats blend shapes as an enhancement where an English utterance can use them. If someone promises you blend-shape lip-sync in Arabic on Azure today, ask them for the locale row.

One voice for two languages, hosted in Dubai

The persona needs one voice, or it stops being one persona. Azure’s professional custom voice can be trained on a consented speaker’s recordings and, with the cross-lingual feature, made to speak a second language in the same voice. The training runs in a Microsoft custom-voice region; the resulting voice is hosted in UAE North, so at runtime no visitor audio or text leaves the country. Custom voice is also gated behind Microsoft’s limited-access approval, which goes in the dependency calendar with a date.

One more locale detail. On the language support page, ar-AE custom voices are listed without the cross-lingual feature, while ar-EG and ar-SA are listed as cross-lingual sources and targets. So a Gulf-Arabic speaker recorded for the persona is trained under a source locale that supports the transfer, then carried into English. It’s a footnote in the docs and a week of confusion if you find it late.

Until the custom voice is approved and trained, the standard neural voices for ar-AE (Fatima and Hamdan) and an en-US voice stand in. Nobody waits for the voice to hear the agents.

Two to three seconds, in-region

The sticky note said two to three seconds, and the tempting way to hit it is a real-time voice API that does everything in one call. When we checked in September 2026, Voice Live in UAE North ran only on global model deployments, and the text-to-speech avatar service wasn’t offered in the region at all. Both would have handed the visitor’s words to infrastructure outside the residency clause.

So the pipeline is the classic one, done carefully: streaming speech recognition in UAE North, the agent turn, streaming synthesis with viseme events, and the first audio played while the rest of the answer is still being generated. Barge-in lets the visitor interrupt. The target is measured in performance testing at the agreed concurrency, and a text-chat mode using the same agents is there from day one for anyone who’d rather type. When Microsoft lands the native services in the region, they get adopted under the support contract, with the customer’s approval, and nothing above them changes.

What the programme owner got

One face on web and mobile. Arabic and English from one voice. Captions, keyboard navigation, right-to-left and left-to-right layouts to WCAG 2.1 AA, because a government avatar that only works for people who can hear it isn’t finished. And a security lead who could name the region for every component, including the visitor’s own phone.

The sticky note is still on the whiteboard. It turned out to be a good spec, once we stopped shopping for it. If you’re specifying a bilingual assistant for the public and the residency clause is already written, talk to 10ⁿ Tech before the first vendor demo.

Related resources

• Data residency on Azure UAE North depends on the deployment type, not the region name
• The AI virtual trainer: one-to-one training at scale, honestly assessed
• Digital twins and pixel streaming
• Self-hosted cloud AI

Photo credits: Helena Lopes and Jukka Aalho on Unsplash.