The result in brief
56 live conversations, sharing one GPU and staying on schedule.
Nemotron VoiceChat 11B listens and speaks at the same time. Keeping a conversation natural means processing each new piece of audio on time, across the complete five-model pipeline.
The .wave inference engine processes conversations together, shares model-weight reads and keeps coordination on the GPU. On one H100, it served 56 concurrent sessions with no late beats observed across three runs.
Against the reference NIM serving configuration, the measured density gives 56× more sessions per GPU and approximately 98% lower GPU cost per conversation at equal GPU price and utilization. All 84,000 measured session-beats at 56 conversations completed within the deadline.
01 / The challenge
Capacity that keeps pace with a conversation.
A live voice service must keep every active session synchronized with incoming audio. Adding users is useful only while their responses still arrive on time. In this workload, two model steps form a 160 ms beat.
The supplied NIM configuration processes the model through several stages. Coordination between those stages and repeated work for separate sessions consume part of the time available. At two sessions, it missed 8–15% of beat deadlines across the three runs.
For the operator of a voice service, that limits the number of conversations that share each GPU. More hardware is then needed to support the same traffic.
02 / The solution
Run the whole pipeline together.
.wave integrates the model into an execution program that processes a group of conversations together. Model weights are reused across sessions, conversation state stays ready between beats, and the GPU advances through the pipeline without returning to the CPU for each operation.
This makes more of each beat available for serving conversations. The engine handles execution, session state and scheduling while maintaining the model’s audio cadence.
03 / Business impact
More conversations from the same GPU hour.
Capacity per GPU
The highest tested .wave profile serves 56 concurrent conversations on one H100, compared with the one-session reference baseline. This increases the volume of live sessions that can share a given GPU allocation.1
GPU cost per conversation
At the same hourly GPU price and utilization, dividing the GPU hour among 56 sessions reduces the GPU-cost contribution per conversation by 98.2%, rounded to 98% in the headline.2
GPU cost per conversation-hour
Same GPU-hour price and utilization.
Conversations stay on time
At 56 concurrent conversations, every measured beat completed within its 160 ms deadline across three runs. At two sessions, the reference stack missed 8–15% of deadlines. This determines usable capacity: how many conversations can share a GPU while keeping their audio on schedule.3
84,000 measured session-beats
56 conversations · three runs
04 / Under the hood
A compiled program that stays on the GPU.
At the core of the .wave engine is WPK, the Wave Persistent Kernel. The engine’s compiler generates the graph, schedule and memory plan from Python Tensor IR. WPK executes all five models as one persistent GPU program, with shared weight reads and coordination between operations kept on the device.
Profiles are compiled for a model, session capacity and context length. The technical article explains the compiler, memory layout, worker coordination and experiment in detail.
Read the WPK technical articleMeasurement notes
- Density. Nemotron VoiceChat 11B on one H100 SXM 80 GB, compared with the pinned reference NIM container in its shipping two-step configuration. The comparison is 56 .wave sessions against the one-session reference baseline. At one session, the reference recorded one late request in 1,500; .wave recorded none in 84,000 session-beats at 56 sessions. The reference therefore does not meet the same strict zero-late-beat criterion.
- GPU cost. Calculated as 1 − 1/56 = 98.2%, assuming equal GPU-hour price and utilization, with the tested capacity filled. This concerns the GPU-cost contribution per conversation, rather than a measured reduction in an entire service bill.
- Deadlines. A beat contains two 80 ms model steps and has a 160 ms deadline. At 56 sessions, .wave recorded zero late beats out of 84,000 session-beats across three runs; server-side p99 was 147.4–147.5 ms. The reference recorded 8–15% late beats at two sessions, with p99 of 172–176 ms. The zero count describes the measured runs.
Three consecutive runs per measured point, with the same model weights, precision and recorded inputs, and approximately two minutes of context. NIM timing is measured at the Triton boundary, bypassing its WebSocket service; .wave timing includes its data plane. Client transport and audio playout are excluded.
Measurement data