Nemotron VoiceChat 11B

About the model

Nemotron VoiceChat 11B is NVIDIA’s English model for real-time spoken conversation. It understands incoming speech and generates its spoken reply as the conversation unfolds, so the user can interrupt or add information while the assistant is talking.

Alongside the audio, it streams transcripts of what the user says and what it answers.

NVIDIA model card

InteractionFull duplex

Listens and speaks at once, so callers can interrupt mid-sentence.

OutputSpeech and text

Spoken replies with live transcripts of both speakers.

Parameters11B

Speech recognition, dialogue and speech synthesis in one model.

Key capabilities

Listen while speaking

Process incoming speech while generating a spoken response.

Natural conversation timing

Handle interruptions and the hand-offs between listening and speaking.

Speech and text output

Generate spoken replies together with their text.

Live transcripts

Receive user and assistant transcripts as the conversation happens.

API usage

Set DOTWAVE_API_KEY on your server, then open a session. The Python and TypeScript guides cover audio capture, playback and transcripts.

Model ID: nemotron-voicechat
Endpoint: wss://api.dotwave.ai/v1/live/sessions

import asyncio, json, os
import websockets

LIVE_URL = "wss://api.dotwave.ai/v1/live/sessions"

async def main():
    async with websockets.connect(
        LIVE_URL,
        additional_headers={"Authorization": f"Bearer {os.environ['DOTWAVE_API_KEY']}"},
    ) as ws:
        # session.start must be the first message, or the socket refuses it.
        await ws.send(json.dumps({
            "type": "session.start",
            "session": {
                "model": "nemotron-voicechat",
                "audio": {
                    "input": {
                        "format": {"type": "audio/pcm", "rate": 24000}
                    },
                    "output": {
                        "format": {"type": "audio/pcm", "rate": 24000},
                        "encoding": "base64",
                    },
                },
            },
        }))
        # Send session.input_audio.append and handle
        # session.output_audio.delta events here.

asyncio.run(main())

import WebSocket from 'ws';

const ws = new WebSocket('wss://api.dotwave.ai/v1/live/sessions', {
  headers: { Authorization: `Bearer ${process.env.DOTWAVE_API_KEY}` },
});

// session.start must be the first message, or the socket refuses it.
ws.on('open', () => {
  ws.send(JSON.stringify({
    type: 'session.start',
    session: {
      model: 'nemotron-voicechat',
      audio: {
        input: { format: { type: 'audio/pcm', rate: 24000 } },
        output: {
          format: { type: 'audio/pcm', rate: 24000 },
          encoding: 'base64',
        },
      },
    },
  }));
});

ws.on('message', raw => {
  const event = JSON.parse(raw.toString());
  if (event.type === 'session.output_audio.delta') {
    playPcm24k(Buffer.from(event.delta, 'base64'));
  }
});

curl https://api.dotwave.ai/v1/live/client_secrets \
  -H "Authorization: Bearer $DOTWAVE_API_KEY" \
  -H "Content-Type: application/json" \
  --data '{
    "expires_after": {
      "anchor": "created_at",
      "seconds": 60
    },
    "session": {
      "type": "live",
      "model": "nemotron-voicechat",
      "audio": {
        "input": {
          "format": {"type": "audio/pcm", "rate": 24000}
        },
        "output": {
          "format": {"type": "audio/pcm", "rate": 24000},
          "encoding": "base64"
        }
      }
    }
  }'

Pricing and limits

Rate $0.005 / minute

Introductory price, billed while a session runs.

Credit $10 to start

Included at sign-up. No card required.

Limits 120 s per session

Sessions use the model’s built-in persona and its one voice, aria. Custom instructions, text input and tools are not supported yet; a session that asks for them is refused.

Public beta: events, limits and the introductory price may change before general availability.

Data handling: audio and transcripts are not retained.

Applications & use cases

Customer-support voice agents

Let callers ask questions, interrupt and clarify details while the assistant responds.

Tutoring & English conversation practice

Build spoken exercises and practice conversations that adapt as the learner speaks.

Interactive characters & companions

Give virtual characters a spoken interface that can listen and respond throughout a conversation.

How .wave keeps this model on time at scale: read the case study