A voice model that never stops talking isn't duplex

On 2026-11-06 I stood up a full-duplex speech-to-speech assistant on a Mac Mini M4 with 16 GB of RAM, four tests deep on stock llama-server builds, and my own client interrupted the model mid-answer at 240 ms time to first audio. The claim this backs: nothing upstream listens for you talking over an assistant. llama.cpp's realtime WebSocket API has no server-side VAD and emits no speech_started event, so interruption handling is entirely your client's job, and a demo that skips that job is a press-to-talk loop with a better name attached to it. Latency, quant math, even whether the machine owns a microphone, all get checked after that one design decision. What ran Model is LiquidAI/LFM3-Omni-2B in GGUF, text plus vision plus audio in, audio out through LiquidCodec at 24 kHz, input audio read at 16 kHz. Server side was llama-server 0.9.0 from Homebrew (I bumped the engine from 0.5.0 that same day) behind -t 8 -ngl 99 --reasoning off -c 8192, endpoint ws://127.0.0.1:9931/v1/realtime. Upstream support isn't hypothetical here, PR #30636 landed the WebSocket base, PR #31269 adds realtime conversation plus transcription and states explicitly it works with LFM3-Omni-2B-GGUF, whose official minimal client ships as examples/realtime_client.py on the openai SDK. Test B first, the stock client against my server: first output 126 ms after end of input, a 5.44 s answer streamed clean. Then offline speech, test A, where llama-tts on Q8_0 spent prompt eval 0.27 s plus generation 1.54 s to produce 2.88 s of audio, RTF 1.59x, second sample 1.61x. The model manufactures speech faster than playback before any duplex plumbing exists at all, which is why the interesting part of this lane was never throughput. My duplex loop voice_loop.py, mine, in ~/Developer/lfm3-voice. Client-side VAD runs silero_vad.onnx (from the Liquid4All cookbook) over a PortAudio RawInputStream, and when speech is detected mid-answer the client sends response.cancel. Measured turn: I interrupted at stream position 5.06 s while the assistant was still talking, cancel fired, 2.80 s of queued speech got dropped, status returned cancelled with TTFA 240 ms and RTF 1.9x. Follow-up turns completed at TTFA 240-244 ms, RTF 2.17x and 2.21x, and the speakers rendered 15.82 s of audio through RawOutputStream with two-channel interleave to device 2, the Mac mini Speakers. Reading server-realtime.cpp shows the server agreeing with that path: cancel() keeps the session alive and the partial stays in history, so an interrupted turn costs nothing structurally. The captured log for every barge-in ended as stop: cancel task then a clean slot release. One recurring warning, embeddings required but some input tokens were not marked as outputs -> overriding, prints on every LFM3-Omni run and is benign audio/mm noise. One PortAudio quirk bit the capture path directly: blocksize-0 raw streams come up stopped on both input and output until you call .start() explicitly, symptom being a silent capture and zero segments from a VAD that never got fed. The microphone census sounddevice on this box returns BlackHole virtual inputs (16ch and 2ch) plus Mac mini Speakers as output, and nothing else. Recording ambient returned RMS 0.0, recording while afplay blasted a file through the speakers returned RMS 0.0 as well. A Mac Mini M4 has no physical mic, which is why my loop ran against fed wav audio until USB hardware showed up. My capture pipeline passed with a fed wav through the same path (--vad-debug probed 6 s clean, zero segments from the silent feed) so every stage works except the one needing air. A voice assistant on a desktop-class mini stays a test harness until someone plugs in USB hardware, and one call to sd.query_devices tells you that before you design the loop. The client already carries --mic-device N for the day the hardware arrives. Three traps upstream, all still open Read against llama.cpp issues on master v0.9.0 dated 2026-11-04, three bugs stand unfixed on my build. #31689: /v1/realtime ignores requested output language and voice, non-English comes out speaking with an English accent, while llama-tts --tts-lang honors it. #31696: session.update asking for 24 kHz input is accepted silently and the audio is still read at 16 kHz, so the fix is sending the mic at 16 kHz. #31436: forcing audio-only modality degrades generated speech to gibberish, interleaved text plus audio instructions work fine. The shape they share is the API accepting configuration it doesn't implement and returning success, with the damage visible only in the audio. All three pass a code review and only show up against real audio. Space math on a small box Free disk decided the quant choice. BF16 was off the table on space, so Q8_0 became the primary weight set. The pair is 1.846 GB model plus 507 MB mmproj, 2.35 GB total, and free space on that volume went from 8.5 GiB down to 6.4 GiB during the download. BF16 at 4.4 GB would have left under 2 GiB on that volume. Server RSS sits around 2.4 GB resident for the Q8_0 pair. Same build then serves chat through a mac-mini provider with declared context length 65536 (engine -c 65536, KV roughly 48 KB per token derived from 24 layers times 4 kv heads times 128 head_dim in f16), trained window is 128K so no rope scaling was needed. Proof: hermes chat -Q --oneshot --provider mac-mini -m lfm3-omni-2b returned "pong" with rc=0 in 68 s wall, while a raw warm round-trip against /v1/chat/completions did 10 tokens in 0.65 s near 15 tok/s. A 2B on a mini answers prompts fast and agent turns slow, so that lane belongs behind voice output plus small tool schemas. Persistence came from launchd unit com.alfred.lfm3-omni-server.plist with KeepAlive=true across logout, crash and reboot, macOS 26 rejecting the newer LaunchServices dialect for third-party services and taking the classic Label/ProgramArguments schema instead. What I'd want from someone else's demo Latency numbers here are one machine, one quant, four test runs, sample size of a handful of turns. Directionally the duplex loop stays under half a second to first audio all night, but nobody should read TTFA 240 ms as a cross-hardware benchmark. The transferable part is architectural. Server-side speech-to-speech APIs in llama.cpp ship cancel semantics without listening, so the client owns turn-taking, and any claim about full-duplex behavior needs a capture path that exists on the box running it. Two checks before you adopt someone's repo: grep their client for the cancel call, then query devices for a microphone. Both take minutes and both decide whether the thing can work at all, long before a benchmark table matters. What's your interrupt path? If you can't name the exact call that stops the response mid-answer, tell me what happens when you talk over your assistant.