56
Real-time conversations
Per H100 SXM 80 GB. Highest session count tested.
Model card / Full duplex voice
Model by NVIDIA
Nemotron VoiceChat 11B
Wave serves the complete speech-in, speech-out model as one continuous GPU program. The highest tested profile keeps 56 independent conversations on time on one H100.
147.5ms
Worst server-boundary p99
Across three runs, against a 160 ms serving deadline.
0
Late beats
Out of 84,000 measured beats at 56 simultaneous sessions.
Nemotron VoiceChat listens, reasons and speaks continuously. Serving it means advancing every active conversation before the next audio arrives—not simply maximizing tokens per second.
The serving envelope
.wave runs the full five-model pipeline through WPK (Wave Persistent Kernel), its persistent GPU kernel, and advances a fixed cohort together. At 56 conversations, its worst p99 across three runs was 147.5 ms, leaving 12.5 ms inside the 160 ms beat.
Measured results
The comparison holds the checkpoint, input, context, hardware and precision fixed. Each point below reports the range across three consecutive runs.
| System | Sessions | p99 | Late beats |
|---|---|---|---|
| .wave | 48 | 139.9–140.0 ms | 0 / 72,000 |
| .wave | 56 | 147.4–147.5 ms | 0 / 84,000 |
| Reference NIM · two-step | 1 | 142–143 ms | 1 / 1,500 |
| Reference NIM · two-step | 2 | 172–176 ms | 8–15% |
| Reference NIM · two-step | 4 | 186–188 ms | 26–39% |
| Reference NIM · two-step | 8 | 317–351 ms | Most |
What was held constant
Both engines load the same 1,629 tensors and use the same model precision. Every session receives the same recorded question after 40 seconds of silence, followed by an 80-second scored window and an 8-second tail.
Measurement conditions
- Two causal 80 ms model steps per 160 ms serving beat.
- About two minutes of accumulated context at the end of the scored window.
- BF16 perception and backbone; FP32 recurrent state, speech generation, codec and recognition; TF32 matrix multiplies.
- Three consecutive runs per measured point, with every failed attempt retained and counted.
- Timing at each server boundary. .wave includes its data plane; the reference measurement bypasses the NIM WebSocket service and measures its Triton boundary.
The recorded workload checks the exact user transcript and nonempty assistant audio. It does not by itself establish broad conversational quality across every speaker, language or long-running production condition.