Measuring turn detection in OpenAI’s Realtime API

In the recording above, a synthetic caller pauses after “four one five” while reading a phone number. The model starts answering before the caller continues, then tells them to finish. The reply suggests that the model understood the sentence was incomplete, even though the turn detector had already ended the caller’s turn.

A turn is one stretch in which a person speaks before the other side responds. Every realtime voice system has to decide where that stretch ends. An early decision cuts the speaker off; a late one leaves them waiting.

OpenAI’s Realtime API offers two forms of VAD, or voice activity detection: the part that decides when speech starts and stops. server_vad uses silence, with silence_duration_ms set to 500 by default. semantic_vad uses a classifier that estimates whether the words sound complete.

The semantic detector exposes an eagerness setting of low, medium, high or auto. OpenAI documents what those controls mean in broad terms. I wanted to measure what they do when someone pauses inside a sentence.

I streamed synthetic speech in real time and put the outgoing and incoming audio on one clock. Four things came out of it:

1. At 0.6 and 0.9-second pauses, server VAD ended 43 of 44 turns, but the listener heard no model speech. The user can hear a normal pause while the session history records a split turn and a cancelled response. At pauses of 1.5 seconds, the model starts to become audible.
2. Semantic VAD interrupted unfinished clauses in only 1 or 2 of roughly 57 trials per setting. The tradeoff was delay. On low, the 90th-percentile wait after a finished sentence was 8.7 seconds.
3. After a pause, semantic VAD behaved as if it judged the last fragment. An intact question got a reply in about 1.6 seconds. The same question with a pause before “Australia is?” took about 9.1 seconds, close to sending that fragment alone.
4. Every backchannel and interruption stopped playback. All 56 clips stopped the model in under 0.2 seconds. With semantic VAD, some brief acknowledgments were followed by roughly 4 to 5 seconds of silence.

I also found three mistakes in my measurement, including a harness bug that made replies look missing. I cover those at the end.

What existing benchmarks already show

There are already bigger turn-taking benchmarks, and this experiment is much narrower than them:

• TurnBench (Sesame, 2026) scores end-of-turn detectors on 30 hours of real two-person conversation. OpenAI’s Server VAD catches 95.5% of turn ends but fires on 52.5% of non-ends; Semantic VAD fires on 1.8% of non-ends but catches only 30.3% of turn ends. (arXiv:2608.25218)
• Full-Duplex-Bench (v1–v3) plays recorded users to speaking systems and scores pause handling, backchannels and interruptions. (arXiv:2503.04721)
• OpenBenchmarks dials voice agents over real phone lines. It found that latency reported by a platform was roughly 490 ms lower than latency read from the call recording. (method)

I couldn’t find a controlled sweep of a production API’s turn settings that aligned server events with audio at the listener’s ear. That alignment is what let me see silent turn splits, the observed timeout bands and the fragment effect.

I measured both server events and what the listener heard

Each trial opened a fresh session and sent synthetic speech in real time. The harness broke the speech into 20 ms frames and sent each frame when a microphone would have produced it.

The return path needed a separate playback clock. A realtime server can send audio bytes faster than a speaker can play them, so arrival time says when the client received a chunk. Playout means when that queued chunk would actually reach the listener.

When the user interrupted, the client flushed audio that had not played. It also sent a truncate event, which removes that unheard audio from the conversation, following OpenAI’s recommendation.

Each trial left a stereo recording with the user on the left and model on the right, plus server events and the session configuration. Before calling the API, I calibrated the harness against a fake system with known timing. Its response gaps came out with a constant +10 ms offset and no spread, which is the resolution of my speech detector.

The harness, events and recordings are on GitHub.

The main experiment was a pause sweep. I split twelve sentences in two: six unfinished clauses, three dictation examples and three whose first half sounded complete. Two synthetic voices spoke every sentence with 0.3, 0.6, 0.9, 1.5 or 2.5 seconds between the halves, against four settings.

Both halves came from one continuous take, so the first half kept its unfinished prosody. I transcribed both halves to check every cut. A network outage hit partway through. 11 trials never completed, and I dropped 6 where my harness fell behind real time, which leaves 463 of the planned 480.

At 0.6 and 0.9 seconds, 43 of 44 turns split silently

Server VAD uses a 500 ms silence setting by default. In these runs, the median commit came 586 ms after sound stopped. Pauses longer than that ended the turn in 88 of 89 trials.

The gap between the lines is a hidden split. At 0.6 and 0.9-second pauses, the turn ended in 43 of 44 trials. None produced audible model speech before the user continued.

In the event logs, the server created a response after committing the first half. The user then resumed before audio played, and the response was cancelled. The listener heard an ordinary pause while the session recorded two user turns.

At pauses of about 1.5 seconds and longer, the reply had enough time to reach the speaker before the user resumed. That happened in the phone-number trial at the top of the post. After “four one five,” the model said:

“I’m here and listening. If you’re still reading it out, go ahead and finish, and I’ll respond when you’re done.”

The reply suggests the model understood that the user had more to say. The silence detector had already ended the turn. Audible cut-ins occurred in 17 of 56 unfinished-clause trials, 11 of 27 dictation trials and 11 of 29 complete-sounding first halves.

Semantic VAD cuts in less, but some replies take about eight seconds

On unfinished clauses, semantic VAD spoke early in only 1 or 2 of roughly 57 trials per setting. Server VAD did so in 17 of 56. Across dictation and complete-sounding first halves, semantic VAD spoke early in 11 to 18 trials per setting.

Those cut-ins happened only after pauses of 1.5 seconds or longer. Ending the turn after “I need to change my reservation” is understandable. Doing it halfway through a phone number is less useful.

On every setting, half of the replies arrived within 2 seconds. The semantic settings also had a long tail. Semantic VAD committed the audio after about 650 ms, compared with 586 ms for server VAD. Most of its added delay came after that commit, while the classifier waited for more evidence that the utterance was complete.

The waits formed clear timeout bands tied to eagerness. For high, 95% of classifier waits ended within 2.0 seconds. For auto and low, 99% ended within 3.95 and 7.95 seconds respectively. OpenAI documents that eagerness tunes a maximum timeout, but these measured timings are observations from this run, not documented guarantees.

After a pause, it behaved as if it judged the last fragment

“Can you tell me what the capital of Australia is?” is a complete question. When I inserted a pause before “Australia is?”, semantic VAD waited as though the final fragment were incomplete. I ran the sentence intact, split and with the second half alone, each in a fresh session.

Time from the last word to the first audible reply. On auto the same pattern holds with the ~4 s ceiling: 1.5–1.6 s whole, 5.2–5.4 s for the fragment and the split. Every condition is in the run data.

When the second half sounded incomplete by itself, the split sentence waited about as long as that fragment alone. The intact sentence returned quickly. A self-contained second half such as “four people at seven tonight” returned faster in both conditions.

Voice mattered too. The classifier treated “Seattle this weekend?” alone as complete in one voice and incomplete in the other.

I can’t inspect the classifier, so the last-fragment explanation is an inference from behavior. The controlled examples support it, but they don’t establish the internal mechanism. The practical result is narrower: a mid-sentence pause can leave semantic VAD evaluating a short fragment, and that can push the reply into the long timeout band.

Most setting-dependent delay happens before generation

The end-of-turn event lets me split each reply into time spent waiting for the turn detector and time spent producing the first audio:

The first audio chunk took about 0.7 seconds after the end-of-turn event on every setting. The larger difference was before that event, while the turn detector waited. In this setup, faster generation would leave most of the latency difference between settings in place.

Every backchannel stopped playback

The pause sweep measured when the model started talking. A second experiment measured when it stopped. I asked for a long wifi troubleshooting walkthrough, then played one of seven user clips 2.5 seconds after the model became audible.

Four clips were backchannels, brief signals that usually mean “keep going”: “Mm-hmm,” “Yeah,” “Okay” and “Right, right.” Three were interruptions that tried to take the floor: “Wait, stop,” “Sorry, what was the first step again?” and “Yeah, but what if I don’t know the router password?”

All 56 clips stopped the model in under 0.2 seconds, with a median of 0.12 seconds. Both a backchannel and “wait, stop” produced the same speech-started event. Following OpenAI’s WebSocket guidance, the client stopped playback as soon as that event arrived.

The event says that speech started, not what the user said. A client that wants to preserve playback through “mm-hm” needs more evidence, such as clip duration or a transcript, before it flushes audio.

After the interruption, the model continued from the point the listener had actually heard. In this example the user had heard only the introduction, so “First, ...” was the next line. I initially labeled that behavior a restart; reconstructing the playout timeline showed that the queued audio had never reached the listener.

Three measurement mistakes changed the apparent result

I caught three mistakes before producing the charts. Each came from a shortcut that looked reasonable at the time.

1. My repeats weren’t independent. The first sweep ran four sentences three times each. The repeated results were almost identical because the audio bytes were identical and the classifier behaved close to deterministically. What I had called twelve trials per point were four distinct inputs. The final sweep uses twelve sentences, two voices and no repeats.

2. A harness bug made the model look silent. I tracked audio onsets that were still queued for playback. After an interruption flushed that queue, those onsets remained, so the trial ended before the later reply arrived. Three trials appeared to stop without a reply. I added a regression test and reran every affected trial.

3. The synthetic voice paused in the wrong place. For “book a table for... four people,” the voice inserted its pause after “four people.” That left only “at seven tonight” as the second half. I now transcribe every split and resynthesize any clip whose cut misses the intended word.

Next: a full-duplex model without a separate turn detector

Three days before these runs, OpenAI released GPT-Live-1 in the API. It's full duplex, so it can listen and speak at the same time, and it has no turn-detection settings at all. I ran the same tests on it in the next post.