Measuring turn detection in OpenAI’s Realtime API

A caller is reading out a phone number and pauses after “four one five.” Before they go on, the model starts answering, and what it says is the strange part: it politely tells the caller to finish. The model could tell the caller wasn’t done. The system that decides whose turn it is had already handed it the floor. (The caller in that recording is a synthetic voice, as is every caller in this post. The model on the other end is real.)

That failure starts with one decision every realtime voice product makes many times a second: is this person done talking? Decide too early and the system cuts them off. Decide too late and they wait in silence. A capable model still feels rude or slow when this boundary is wrong.

OpenAI’s Realtime API offers two turn detectors. server_vad is a timer. Once it sees silence_duration_ms of quiet (500 by default), it ends the turn. semantic_vad instead runs a classifier over the speech and waits longer when the utterance sounds unfinished. Its eagerness can be low, medium, high or auto. The documentation names the controls. It does not show what happens when someone pauses to remember the rest of a phone number.

I built a harness that streams speech in real time, records both sides on one clock, and reconstructs when the model’s audio would reach a listener. Then I ran the pause sweep. Four results:

1. Server VAD ends your turn at any pause over ~0.6 s, and usually you never find out. The server commits the half-sentence, starts a reply, then cancels it when you continue. From about 1.5 s, you start hearing the model cut in. A half-read phone number and a finished question look the same to the timer.
2. Semantic VAD rarely cuts into an unfinished sentence, but its waits have hard ceilings of roughly 2, 4 and 8 seconds. With low eagerness, the 90th-percentile wait after a finished sentence was 8.7 s.
3. After a pause, semantic VAD appears to judge only the last fragment. “Can you tell me what the capital of Australia is?” gets a reply in 1.6 s. Put a 1.5-second pause before “Australia is?” and it waits 9.1 s — the same as sending “Australia is?” on its own.
4. “Mm-hm” stops the model every time. 56 of 56 backchannels and interruptions stopped playback in under 0.2 s. With semantic VAD, a listener’s “yeah” is followed by 4–5 seconds of silence.

I also got three parts of the measurement wrong before the results were ready, including a harness bug that made the model look silent when it had replied. I describe those mistakes near the end.

What prior benchmarks already tell us

Turn-taking is a busy research area in 2026. This experiment sits alongside useful prior work:

• TurnBench (Sesame, 2026) scores end-of-turn detectors on 30 hours of real two-person conversation, and already shows OpenAI’s two detectors at opposite extremes: Server VAD catches 95.5% of turn ends but fires on 52.5% of non-ends; Semantic VAD fires on 1.8% but catches only 30.3%. (arXiv:2608.25218)
• Full-Duplex-Bench (v1–v3) plays recorded users to speaking systems and scores pause handling, backchannels and interruptions. (arXiv:2503.04721)
• OpenBenchmarks dials voice agents over real phone lines and found vendor-reported latency roughly 490 ms lower than what the call recording shows. (method)

I could not find a controlled sweep of a production API’s own turn settings that aligned server events with audio at the listener’s ear. That alignment exposes behavior the aggregate benchmarks do not: silent turn splits, timeout ceilings, and the fragment effect.

How I measured it

Each trial opens a fresh session and sends synthetic speech in real time. The harness breaks the speech into 20 ms frames and sends each frame when a microphone would have produced it.

The return path needs a different clock. A realtime server can send audio bytes faster than a speaker can play them, so arrival time tells me when the client received a chunk, not when the listener heard it. I place each chunk on an emulated playout timeline. When the user interrupts, the client follows OpenAI’s recommendation: it flushes playback and tells the server how much audio actually played.

Every trial leaves the same evidence: a stereo recording with the user on the left and model on the right, server events with arrival times, and the exact session config. Before calling the API, I calibrated the harness against a fake system with known timing. Its measured response gaps had a constant +10 ms offset with zero spread, which is the speech detector’s resolution. The harness, every trial’s events and every recording are on GitHub.

The main experiment is a pause sweep. I split twelve sentences in two. Six stop mid-clause (“I’d like to book a table for… four people at seven tonight”), three are dictation (“my phone number is four one five… five five five, zero one nine two”), and three sound complete before the pause (“I need to change my reservation.… It’s under the name Jordan.”). Two synthetic voices speak each sentence with 0.3, 0.6, 0.9, 1.5 or 2.5 seconds of silence between the halves, against four settings. I cut both halves from one continuous take so the first half keeps its unfinished prosody, then verified every cut by transcribing both halves. A network outage interrupted the sweep partway through; I re-ran the affected trials and dropped six where the harness itself fell behind real time, which leaves 463.

Server VAD: the silent split

The default is a 500 ms silence timer. In practice, the server’s speech detector adds a little hangover, and the turn ends about 590 ms after sound stops. Pauses longer than that ended the turn in 88 of 89 trials.

The gap between the two lines is the hidden failure. At pauses of 0.6 and 0.9 seconds, the server ends the turn and starts generating, but the first audio takes another ~0.7 s to arrive. By then the user is speaking again, so the server cancels the reply before a sample plays. None of those 44 trials produced a single audible word. The user hears an ordinary pause; the session records two turns and a discarded response.

Past about 1.5 seconds, the reply reaches the speaker before the user resumes, and the split becomes audible. That is what happened in the phone-number trial at the top of this post. The server ended the turn after “four one five,” and the model said:

“I’m here and listening. If you’re still reading it out, go ahead and finish, and I’ll respond when you’re done.”

The reply shows that the model understood the user had more to say. The timer had already handed it the floor. That timer has no access to meaning: it cut in about as often on unfinished clauses (17 of 56) and dictation (11 of 27) as on first halves that sounded complete (11 of 29).

Semantic VAD: patient, until it isn’t

Semantic VAD fixes this where it matters most. On unfinished clauses it spoke early in only 1 or 2 of about 57 trials per setting, against 17 of 56 for server VAD. It still cut in on dictation and on first halves that sound complete, almost always after pauses of 1.5 s or more: 13 to 20 trials per setting out of about 116. That is defensible after “I need to change my reservation,” and less so halfway through a phone number. The bigger cost appears when the user really is finished:

The replies fall into two clusters: a normal response around 1.5 s and a long tail. The server events explain why. Semantic VAD commits your audio almost as quickly as server VAD does (about 650 ms after you stop, against 590 ms); the added delay comes while the classifier decides whether the utterance is complete. If it decides the user is still going, it waits until a ceiling set by eagerness, and the ceilings are sharp: 95% of these waits ended within 2.0 s on high, and 99% within 3.95 s on auto and 7.95 s on low. OpenAI does not document these numbers; they come from the observed event timing.

It judges the last fragment

The surprising part was which trials reached the ceiling. “Can you tell me what the capital of Australia is?” is a finished question. Add a pause before “Australia is?” and semantic VAD waits as though it is unfinished. I ran the same sentence three ways, each in a fresh session.

Time from the last word to the first audible reply. On auto the same pattern holds with the ~4 s ceiling: 1.5–1.6 s whole, 5.2–5.4 s for the fragment and the split. Every condition is in the run data.

When the second half sounds like a fragment on its own, the split sentence waits exactly as long as that fragment. The intact sentence returns quickly. A self-contained second half such as “four people at seven tonight” is fast either way. Voice also matters: the classifier treated “Seattle this weekend?” alone as finished in one voice and unfinished in the other.

I cannot inspect the classifier, so “judges the last fragment” remains an inference from behavior. It predicts every sentence in the sweep. The practical consequence is that the users most likely to pause mid-sentence, including people thinking, reading something out, or speaking a second language, are exactly the ones most likely to hit the eight-second wait.

Where the second goes

The server reports when it decides the turn is over. That event lets me separate each reply into time spent deciding and time spent producing:

The first audio takes about 0.7 s to produce on every setting. Nearly all the difference comes before that, while the turn detector waits. For this system, optimizing generation would leave most of the setting-dependent latency untouched.

“Mm-hm” stops it cold

The pause sweep tests when the model starts talking. A second experiment tests when it stops. I asked for a long, step-by-step wifi troubleshooting walkthrough. Then, 2.5 s after the model became audible, the user said one of seven things. Four were backchannels, the brief signals listeners use to mean keep going: “Mm-hmm,” “Yeah,” “Okay,” and “Right, right.” Three were actual interruptions: “Wait, stop,” “Sorry, what was the first step again?” and “Yeah, but what if I don’t know the router password?”

The model stopped every time, 56 of 56, in under 0.2 s (median 0.12 s). The system sees the same event for a backchannel and “wait, stop”: speech started. Part of that is the client: it follows the recommended pattern and flushes playback on every speech event. The event itself carries no hint of what was said, so a client that wants to treat “mm-hm” differently has to wait for more evidence, such as how long the speech lasts or what the transcript says.

One behavior is right: after the pause, the model resumes where it was cut off. At the moment of interruption, the user had heard only the introduction, so “First, …” is the correct next line. I initially called this a restart. The playout timeline showed that I was wrong about how much audio the user had heard.

What I got wrong along the way

I caught three mistakes in my measurement before they reached the charts. Each came from a shortcut that looked reasonable at the time.

1. My repeats weren’t independent. My first sweep ran four sentences three times each. The three repeats came back almost identical (9.1, 9.2, 9.1 s) because I sent byte-for-byte identical audio to a classifier that is close to deterministic. What I had called twelve “trials” per point were really four. The sweep here uses twelve sentences, two voices, and no repeats.

2. A harness bug made the model look silent. The harness decided that the model had finished by watching audio onsets, including onsets in audio still queued for playback. When an interruption flushed the queue, onsets from audio that never played remained on the list, and the trial ended early. Three trials appeared to stop without a reply. The model had replied; my harness had stopped waiting. A regression test now covers the bug, and I re-ran every affected trial.

3. The synthetic voice paused in the wrong place. I made an unfinished first half by synthesizing the whole sentence with a pause marker, then cutting at the silence. For “book a table for… four people,” the voice inserted its pause after “four people.” That left only “at seven tonight” as the “second half.” I now transcribe every split and re-synthesize it until the cut lands on the intended word.

Next: a model with no turn detector

Three days before these runs, OpenAI released GPT-Live-1, a full-duplex model with no turn settings. It decides for itself when to listen and speak. On the same tests, it listens through “mm-hm” and replies faster than any setting here. It also takes over a second to stop after “wait, stop.” I’ll make that comparison in the next post.