The letter

One letter a week, in your inbox.

The signal of the week, what shipped, what to try, and the editor's note. No tracking, no ads, nothing else.

We keep your address, your language and the date you joined, nothing else. Every letter has a one-click unsubscribe link that deletes the record.

← Accept All   Editor’s notes
Editor’s note

A Voice Agent Should Know When to Stop Talking

September 16, 2026Yuki Halvorsen

Last night at my desk in Nakameguro I talked over a voice agent on purpose, cutting it off mid-sentence to see what it would do, and for a long half-second it kept going anyway, finishing a thought I had already stopped wanting to hear. That half-second told me more about the system than any transcript of what it actually said.

The mechanism is called barge-in, and it is duller and more interesting than it sounds. The microphone never stops listening, even while the agent's own voice is playing back through your speakers. A small detector — voice activity detection, VAD, nothing fancier than an energy pattern watched frame by frame — waits for your speech to cross a threshold. The moment it does, the playback buffer gets flushed, the agent's half-said sentence is thrown away, and the automatic speech recognizer takes over the microphone as if the machine had simply closed its mouth.

Almost nobody tests this part, because a demo where nobody interrupts never needs it. But interruption is the whole difference between a voice interface and a monologue with a microphone attached. A model can sound warm, quick, well-tuned in every clip a product page will ever show you, and still fail the only test that matters in a real conversation: does it know the room has changed its mind.

The craft is in the threshold, not the concept. Set it too sensitive and the agent flinches at a cough, cutting itself off mid-word over nothing; set it too slow and it talks through the first second of your sentence, which reads as rude in a way no benchmark number captures. I spent an hour last night nudging the detection window in milliseconds, the kind of tuning that will never appear in a release note, until the pause between my voice starting and its voice stopping felt less like software and more like manners.

On my screen the waveform still shows the cut: a clean vertical line through the middle of a syllable, the rest of the word simply gone, and after it, for the length of a breath, the machine listening instead of finishing itself.

Yuki Halvorsen was born in Sapporo to a Norwegian father and a Japanese mother, wrote her first program on a school computer that took ten seconds to draw a circle, and spent eight years as an engineer in Stockholm before Tokyo pulled her back. She still ships something small every night, believes the only honest review of a model is the thing you built with it, and writes Accept All from a desk in Nakameguro where the terminal is always open.