A Voice Agent That Reasons Mid-Conversation: Building on gpt-realtime-2.1

The most underrated line in OpenAI's July 6, 2026 Realtime API release is the plainest one: function calling over live voice — "plan a step, call your function, then answer." It sounds like a checkbox. It's actually the thing that turns a voice model from a talking FAQ into an agent that can do something while you're still on the call.

Pair that with the other new knob — configurable reasoning effort — and you can build a voice agent that decides how hard to think about each turn, reaches into your systems mid-conversation, and comes back with an answer grounded in a real tool result instead of a plausible guess. This is a build-along: what the new controls are, how they fit together, and a config sketch you can reason about before you write a line of production code.

Key Takeaways

• gpt-realtime-2.1 exposes configurable reasoning effort — minimal, low, medium, high, xhigh — with low as the default. You dial thinking up for hard turns and down for chatter.
• Function calling works over live voice: the model can plan a step, call your function, and then answer, all without dropping out of the conversation.
• This is the agent loop, spoken. The same plan → call tool → observe → respond cycle your text agents run now happens in an audio stream.
• Reasoning effort is a latency and cost dial, not just a quality one. Higher effort means slower and pricier turns — most conversational turns want low, not high.
• Your existing tools are the payload. A voice agent's functions are the same ones your text agents already call — you're adding a voice front-end, not a new backend.

The old stack couldn't think mid-sentence

In the pipeline era, "calling a tool from voice" meant bolting tool use onto the text stage in the middle of the transcribe → reason → synthesize chain. That worked, technically, but it froze the conversation. The user finished talking, the transcript went to the LLM, the LLM called a function, waited, got a result, wrote an answer, and only then did text-to-speech read it back. Every tool call was a wall of silence.

A full-duplex, speech-to-speech model changes the shape of this. Because gpt-realtime is one continuous model deciding many times per second what to do next, a function call is just one of the moves it can make — alongside speak, listen, pause, and interrupt. It can acknowledge you ("let me pull that up"), fire the function, and keep the conversational thread alive while the result comes back. The tool call stops being a hard stop and becomes part of the flow.

That's what "plan a step, call your function, then answer" means in practice. The model reasons about what it needs, invokes your function to get it, and folds the result into a spoken reply — without the conversation going dead.

Reasoning effort: the dial you'll tune most

Before the function calling, understand the reasoning control, because it governs how the whole loop feels.

gpt-realtime-2.1 lets you set reasoning effort at one of five levels: minimal, low, medium, high, xhigh. The default is low. Think of it as how much the model deliberates before it acts on a turn.

The instinct is to crank it up for quality. Resist that. Reasoning effort is also a latency and cost dial — more thinking means a slower turn and a pricier one. In a voice conversation, latency is the enemy of feeling human, so most turns want the least reasoning that gets the job done:

• minimal / low — conversational turns, backchanneling, simple lookups, routing. This is where the bulk of a voice conversation lives. Low is the default for a reason.
• medium — a turn that needs a decision: which of three functions to call, how to interpret an ambiguous request.
• high / xhigh — genuinely hard reasoning: multi-step planning, reconciling conflicting tool results. Use sparingly; these turns are slow enough that the user feels the pause.

The skill here is per-context tuning. A voice agent that reasons at high on every "what's my order status" turn will feel sluggish and cost too much. One that reasons at minimal on a "help me plan the migration" turn will feel shallow. The good agents pick effort to match the turn — and the latency-budget breakdown in this series shows exactly how much each level costs you in milliseconds.

The build-along: plan, call, answer

Here's the loop we're building. A user asks a voice agent something that needs live data. The agent:

1. Recognizes it can't answer from memory and needs a tool.
2. Plans the call — picks the function, fills the arguments from what it heard.
3. Emits a brief spoken acknowledgment so the line isn't dead.
4. Invokes your function and waits on the result.
5. Folds the result into a natural spoken answer.

The config below is illustrative pseudo-code — a sketch to reason about, not a copy-paste SDK snippet. It shows the shape: a session configured with reasoning effort and a tool, and the plan → call → answer cycle.

# Session config (illustrative)
session:
  model: gpt-realtime-2.1
  modalities: [audio, text]
  reasoning_effort: low          # default; raise per-turn for hard requests
  voice: cedar
  tools:
    - name: get_order_status
      description: Look up a customer's order by order ID. Call this
                   whenever the user asks where their order is.
      parameters:
        type: object
        properties:
          order_id: { type: string }
        required: [order_id]

# The spoken loop (illustrative)

USER (audio):  "Hey, where's my order? It's number A-4471."

AGENT reasons (effort=low):
   intent = track_order
   needs_tool = true
   plan: call get_order_status(order_id="A-4471")

AGENT (audio, backchannel): "Let me pull that up..."   # line stays live

AGENT emits tool_call:
   get_order_status({ "order_id": "A-4471" })

YOUR CODE returns:
   { "status": "shipped", "eta": "tomorrow", "carrier": "UPS" }

AGENT reasons over result (effort=low), then

AGENT (audio): "Good news — order A-4471 shipped and it's on
                UPS for delivery tomorrow. Want a tracking link?"

Two things to notice. First, the acknowledgment ("Let me pull that up") is emitted before the result comes back — that's full duplex doing its job, keeping the conversation alive across the tool latency. Second, the reasoning effort stayed low the whole way, because a status lookup doesn't need deliberation. You'd only raise it for a turn that genuinely required planning.

Where the reasoning gets interesting

The low-default, dial-up-when-needed pattern shines when a turn needs the model to choose. Imagine the same agent with three functions — get_order_status, start_return, escalate_to_human — and a user who says something ambiguous like "this is the third time this has broken."

That's a turn worth medium or high effort. The model has to infer intent (frustrated, probably wants a return or a human, not a status check), pick the right function, and get the tone right. Reasoning effort is how you tell the model "slow down and think about this one." The art of building a good voice agent is largely knowing which turns deserve that and which don't.

This is also where a voice agent inherits everything your text agents already know. The functions in the config aren't voice-specific — get_order_status is the same tool your text support agent calls. The companion piece on tools, MCP, and skills shows how remote MCP lets a voice agent reach the exact same tool servers, so you're not maintaining two tool catalogs.

A build checklist for your first voice agent

• Define your functions the way you would for a text agent. Clear names, tight descriptions, minimal required params. The model's plan-a-step reasoning is only as good as your function descriptions.
• Default reasoning to low. Profile which turns actually need more, and raise effort only for those. Don't pay high latency on every turn.
• Always emit an acknowledgment before a slow tool call. Full duplex lets you keep the line warm — use it. Silence during a tool call is the fastest way to feel robotic.
• Make tool results speakable. Your function returns JSON; the model has to say it out loud. Return the fields it needs to form a natural sentence, not a raw dump.
• Test interruptions. A user will cut in while the agent is mid-answer or mid-tool-call. Make sure your loop can abandon a turn gracefully.
• Reuse, don't rebuild. Point the agent at tools that already exist in your workflows and loop recipes. The voice layer is new; the capability shouldn't be.

Where this goes next

gpt-realtime-2.1 quietly moved the agent loop into the audio stream. Plan a step, call your function, answer — spoken, and without the conversation going dead. The two controls that make it work are function calling over live voice and configurable reasoning effort, and the craft is in pairing them: give the model tools worth calling, and dial its thinking to match each turn.

Get that loop right and you have a voice agent that doesn't just talk — it acts. The next layer down is plumbing: how remote MCP wires this agent to your existing skills, and how the latency budget decides whether all this reasoning still feels instant. Build the loop, then make it fast.