NeMo-Speech.cpp

NeMo-Speech.cpp is NVIDIA's official solution for local speech inference, providing day-0 support for the latest models from NVIDIA NeMo Speech, with native inference powered by ggml.

NeMo-Speech.cpp is blazing fast and built for real-time streaming. Speech recognition transcribes audio in 160 ms chunks up to 67× faster than real time, and speech synthesis generates speech up to 60× faster than real time with its first audio in under 10 ms. Both stay faster than real time even on a CPU.

Streaming speech recognition with Nemotron Speech Streaming 0.6B (Q8_0), 160 ms chunks:

Streaming speech synthesis with MagpieTTS Multilingual (Q8_0), 186 ms audio chunks:

See BENCHMARK.md for the methodology and more results.

Install the nemo-speech CLI for the detected platform and backend:

On Linux or macOS, run:

curl -fsSL https://github.com/NVIDIA/NeMo-Speech.cpp/raw/main/scripts/install.sh | sh

Open a new shell after installation so the updated user PATH takes effect.

On Windows, run from PowerShell:

irm https://github.com/NVIDIA/NeMo-Speech.cpp/raw/main/scripts/install.ps1 | iex

Open a new PowerShell window after installation so the updated user PATH takes effect.

The installer downloads the prebuilt archive for the latest release, checks it against the SHA-256 checksum published with the release, and builds from source when no archive is available for your platform. Pass --source (-Source on Windows) to always build from the main branch. A source build requires Git, CMake 3.26 or newer, Ninja, a C++17 compiler, SentencePiece development files, and the toolchain required by the selected backend, if any. See Installation for platform-specific prerequisites and options.

Transcribe a local WAV file. On first use, the CLI downloads the pinned default Nemotron 3.5 GGUF from Hugging Face and verifies its size and SHA-256:

nemo-speech transcribe /path/to/audio.wav

Source checkouts can use test_files/asr/wav/test/jfk.wav as a smoke-test input.

The same command can transcribe the default microphone on builds that include live capture:

nemo-speech transcribe --live

Run nemo-speech model list to see defaults, short names, and which command uses each model. For example, nemo-speech pull nemotron-en downloads the English-only model ahead of time, and --model nemotron-en selects it. Local GGUF paths continue to work without downloading anything. The CLI selects an available backend and handles common mono or stereo PCM WAV sample rates automatically. See the CLI model guide and model conversion for custom checkpoints.

The CLI is the primary interface. Run nemo-speech --help to see the capabilities included in your build. The CLI guide covers model selection, GPU controls, directory transcription, subtitles, diarization, translation, synthesis, structured output, and benchmarking. The VoiceChat guide covers building and serving the realtime pipeline.

Start the same runtime as a local HTTP service and open the playground:

nemo-speech serve \
  --asr-model nemotron-3.5 \
  --open

The server binds to http://127.0.0.1:8080 by default. Its transcription and speech routes expose documented OpenAI-compatible subsets, alongside realtime WebSocket transcription and realtime VoiceChat when its model is loaded. A separately built riva_server binary provides the ASR, TTS, and translation gRPC interfaces. See the server guide when you are ready to integrate either frontend.

Release archives include stable C headers, shared libraries, and an exported CMake package. An installed application can link only the capability it uses:

find_package(NeMoSpeech REQUIRED COMPONENTS ASR)
target_link_libraries(my_app PRIVATE NeMoSpeech::ASR)

See native SDK integration for in-process C/C++ usage, or client integration for OpenAI SDK, curl, and Riva-compatible gRPC usage.

Requires CMake 3.26 or newer, Ninja, C and C++17 compilers, SentencePiece development files, and the toolchain required by the selected backend, if any. For a CUDA ASR and TTS server with the playground:

git submodule update --init llama.cpp third_party/cpp-httplib
scripts/configure.sh cuda-server
cmake --build --preset cuda-server

The configuration helper validates required submodules and applies the pinned ggml patch series for CUDA builds. CPU, Metal, Vulkan, server, component, Windows, and container instructions are in Build from source.

NVIDIA-authored code is released under the Apache License 2.0, with the project copyright notice in NOTICE. Third-party components retain their respective terms; see Third-Party Notices. Release archives also include these files under share/licenses/nemo-speech/.

External contributions are welcome. See CONTRIBUTING.md for the contribution terms and Developer Certificate of Origin sign-off process.