AI Voice Agents with Amazon Connect and AWS Bedrock
What it actually takes to put a speech-to-speech agent into a real contact center — bidirectional streaming, tool calling, and human fallback, not just a demo call.
By Naeem Akhtar · 8 min read
Why speech-to-speech changed what's possible here
Older voice-agent architectures chained together three separate models: speech-to-text, an LLM for reasoning, and text-to-speech to respond — each hop adding latency and a chance to lose prosody or interrupt handling. Amazon Nova Sonic, a speech-to-speech foundation model on Bedrock, collapses that into a single model invoked over one bidirectional streaming connection, handling speech understanding and generation together. That's what makes a phone call feel like a conversation instead of a series of request-response exchanges with a bot.
Bidirectional streaming matters more than it sounds — it's what lets the agent handle interruptions, backchannel naturally ("mm-hmm", brief pauses), and respond with low enough latency that callers don't default to hanging up and calling back during business hours.
The architecture: Connect, Bedrock, and a business logic layer
A production voice agent isn't just a model — it's a small stack, each piece doing one job:
- Amazon Connect — telephony, call routing, and compliance-grade call recording. This is the part of the stack that already has to be reliable at carrier scale, so it's the right layer to own that.
- AWS Bedrock (Nova Sonic + Claude) — the conversational layer. Nova Sonic handles the audio; a Claude-powered reasoning step (directly or via Bedrock AgentCore) handles what to say and what to do with what the caller said.
- AWS Lambda — extracts structured intake details from the conversation and calls out to business systems: a case management platform, a CRM, a scheduling API.
- Amazon S3 — full call recording and transcription storage, for compliance review and for catching drift in what the agent is qualifying or routing over time.
We used exactly this stack — Bedrock, Amazon Connect, Nova Sonic, and Lambda — to build a multi-tenant AI voice intake agent for a law firm that was losing qualified leads to voicemail and long hold times, including all after-hours coverage. The system now answers 100% of client intake calls day or night, handling roughly ten calls a day, and the firm has seen a 30% increase in lead conversion since it went live.
Tool calling and knowledge grounding
A voice agent that can only talk isn't useful for intake, scheduling, or support — it needs to check a caller's case status, look up availability, or verify eligibility criteria against real data mid-conversation. Nova Sonic supports function calling and knowledge grounding through retrieval, meaning the agent can pull structured facts into its response rather than guessing or making the caller repeat information it should already have access to.
Design note
Keep the tool-calling layer separate from the conversational model — a Lambda function that validates and executes an action, rather than letting the model call external systems directly, gives you a place to enforce business rules and log every action independently of what the model "decided" to do.Multi-agent and multi-tenant patterns
For a system serving more than one client or handling more than one type of call, session segmentation and a multi-agent design keep the architecture from becoming a single tangled prompt trying to handle every case. A well-structured multi-agent setup on Bedrock AgentCore keeps the business logic layer intact per tenant — routing rules, qualification criteria, and integrations stay isolated per client even though the underlying infrastructure is shared. This needs to be a decision made at design time; bolting multi-tenancy onto a single-client build later means re-architecting the routing and state model, not just adding a config flag.
Human fallback and the audit trail aren't optional
Every production voice agent needs a defined path for when the conversation goes somewhere the agent shouldn't handle alone — a distressed caller, a genuinely ambiguous case, an explicit request for a human. That's a design decision made up front, not a gap discovered after a bad call. Combined with full call logging and transcription to S3, it means every conversation can be reviewed, and drift in what the agent treats as a qualified lead versus not gets caught early rather than showing up as a slow decline in conversion months later.
Frequently asked questions
What's the difference between Amazon Nova Sonic and older voice bot architectures?
Nova Sonic is a speech-to-speech model that handles understanding and generation in one model over a bidirectional streaming connection. Older architectures chained separate speech-to-text, LLM, and text-to-speech steps together, which added latency and lost natural turn-taking at each handoff. Nova Sonic's single-model streaming approach is what makes the conversation feel closer to a real phone call.
Does Amazon Connect handle the AI reasoning, or just the phone call?
Amazon Connect handles telephony, call routing, and compliance recording. The conversational reasoning happens in AWS Bedrock (via Nova Sonic and a Claude-powered reasoning layer), with a Lambda layer in between to extract structured data and call business systems. Each layer stays swappable rather than being one monolithic system.
Can one AI voice agent system serve multiple clients with different rules?
Yes, with a multi-tenant architecture designed from the start — per-tenant configuration, qualification criteria, and integrations, built on a shared infrastructure layer. This needs to be a design-time decision; retrofitting multi-tenancy onto a system built for a single client typically means re-architecting the routing and state model.
What happens when the AI voice agent can't handle a call?
A production deployment needs a defined human fallback path for edge cases — ambiguous situations, explicit requests for a human, or conversations outside what the agent is scoped to handle. Combined with full call transcription and logging, this keeps a person in the loop for the cases that need one, without requiring a human on every call.