On October 1, Naomi Moneypenny, senior director of product development at Microsoft Foundry Models division, announced three new voice-focused models within Microsoft Foundry, the company's application and agent model catalog. The release aims to address a fundamental challenge in conversational AI: eliminating the awkward pauses that plague voice interactions.

"A voice agent has to do more than hear a request and produce a reply. It has to keep the interaction moving while it listens, reasons, uses tools, and speaks. A delay at either end can turn a conversation into a series of awkward pauses," writes Moneypenny.

The three models now available in Microsoft Foundry are MAI-Transcribe-2-Streaming, which converts live speech to text as it occurs; MAI-Voice-2.1, a multilingual text-to-speech model designed for expressive output; and MAI-Voice-2.1-Flash, a variant optimized for responsive, high-volume voice deployments.

Partials drive incremental speech recognition

Rather than waiting for a speaker to complete an utterance, the technology generates initial hypotheses—called partials in the incremental speech recognition field—within the "low hundreds of milliseconds" of receiving audio. The model refines these partials as additional context arrives, eventually settling on a final transcript once the speaker finishes.

A live transcription experience can surface words almost as quickly as they are spoken.

Naomi Moneypenny

This capability enables customer-service systems to begin processing a caller's intent mid-sentence, allows voice assistants to prepare tool calls faster, and supports live transcription that keeps pace with natural speech. Moneypenny positions MAI-Voice-2.1-Flash as particularly suited for live support agents, voice assistants, and applications requiring spoken responses, while MAI-Voice-2.1 remains the choice when expressive quality takes priority over speed and cost efficiency.

According to Moneypenny's announcement, MAI-Transcribe-2-Streaming ranks first for accuracy on both partial and final transcripts on the Artificial Analysis leaderboard.

Time for voice to speak up?

Will Bodewes, CEO of voice-agent platform company Phonely, views Microsoft's market entry as a positive signal. "Voice AI is ready for prime time, if Microsoft's entry into the market is any indication," Bodewes tells The New Stack.

Microsoft has maintained a presence in speech technology for years through Azure Speech, which now ships the new MAI models. "For an AI developer, that's an opportunity to build on that momentum, so expect more players to follow suit. But the model is only the beginning. Voice agents need to learn from every call and measurably improve over time. For companies to see real results, their voice AI must be ready for prime time in theory and in practice," Bodewes explains.

From an infrastructure perspective, the advancement is significant. Bodewes notes that during the early cloud-speech era—spanning from the mid-2010s through services like Google Cloud Speech-to-Text API, Azure Speech Services, and Amazon Transcribe—requests could queue behind thousands of others, resulting in 30-second delays that typically caused call disconnections.

In the early days, a speech request to a hosted model could sit behind a thousand others for 30 seconds, which, on a phone call, is a hang-up.

Will Bodewes

Phonely addressed this by building routing infrastructure to direct traffic to the least-loaded endpoints. "Owning the model meant owning the queue, and that's how we got to 120-millisecond responses," Bodewes says. He also emphasizes the importance of enforcing critical business logic in code rather than relying solely on the language model. "Every sentence you add to a prompt adds another chance of failure. So we enforce the critical steps in code. If an agent needs a name, email, and date of birth before it issues a refund, the codebase guarantees it. Leaving that to the LLM is how you ship incidents."

The front end (speech) isn't the hard part

Amit Kiran Rege, a machine learning researcher at Phonely, acknowledges the positive developments but cautions that deeper complexities exist. "Microsoft shipped the two ends of the voice agent pipeline, speech in and speech out, which tells you it's betting on a cascade effect for contact centers, the same as we did. But in our work, we found the ends aren't the hard part. The hard part is the silence in the middle where the model is deciding whether a caller is done or taking a breath before the next word," Rege explains.

When Phonely scaled voice services to millions of calls, the team discovered an unexpected consequence: reducing latency in the data pipeline increased the frequency of agent interruptions.

Microsoft shipped the two ends of the voice agent pipeline, speech in and speech out … But in our work, we found the ends aren't the hard part.

Amit Kiran Rege

"Two-thirds of the time our agent decided to jump in too early, and nobody heard it, because the caller resumed before the first syllable and we canceled the reply. Cut your latency in half, and those become audible interruptions. Thus, the lesson is that latency and turn-taking have to improve together," Rege warns.

Talking our way into future interfaces

Microsoft envisions developers using these models and tools to build customer-service agents, multilingual assistants that recognize spoken language and respond in a consistent voice, interactive learning experiences that transition between languages and speakers, and voice-driven interfaces with live captioning.