AI starts speaking Uzbek dialects with human-like accuracy

This solution not only overcame many complex linguistic and language-related barriers but also marked a significant step in developing high-quality speech technology tailored to local needs. The project began in 2024 when Aiphoria partnered with one of Uzbekistan's leading banks to create fully automated, AI-powered voice agents. A dedicated team of the bank’s specialists was assembled and actively involved in the process.

As part of the project, real-life conversations in Uzbek — including regional dialects, accents, and code-switching with Russian — were taken into account to develop a comprehensive solution based on Automatic Speech Recognition (ASR), Text-to-Speech (TTS), and Large Language Models (LLMs). Special focus was given to accurately understanding speech, using anonymized audio data collected from the bank's call center. Labeling and normalization processes were carried out based on real user queries and open-source data.

Linguists, editors, and call center employees participated in the annotation process. With their input, a database of over 7,000 normalization rules was created. Special attention was given to banking-specific terminology in the lexicon — for example, terms like “annuity payments” and “ATM.”

To enable lifelike interaction, over 100 hours of scripted text were recorded by professional voice actresses in various styles. Three distinct voice tones — friendly, neutral, and firm — were developed for different communication scenarios. These were used to train neural network-based speech synthesis models, which were later evaluated by seasoned call center staff.

As a result, AI agents like the Collection Agent were deployed in the bank. These agents now handle up to 40% of incoming calls and operate ten times more efficiently than human staff. They do not require salaries, never fall ill, and are always ready to work. Most importantly, they are capable of engaging in highly effective, natural conversations.

This project stands out not only as a cutting-edge technological solution but also as one of the first successful examples of AI localization in Uzbekistan — potentially marking a historic milestone.