speech-language-model

LLaMA-Omni is a low-latency and high-quality end-to-end speech interaction model built upon Llama-3.1-8B-Instruct, aiming to achieve speech capabilities at the GPT-4o level.

[ICLR 2025] SOTA discrete acoustic codec models with 40/75 tokens per second for audio language modeling

A Survey of Spoken Dialogue Models (60 pages)

AAAI 2025: Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Model

[ICCV 2025] Official Implementation for "Lyra: An Efficient and Speech-Centric Framework for Omni-Cognition"

SlamKit is an open source tool kit for efficient training of SpeechLMs. It was used for "Slamming: Training a Speech Language Model on One GPU in a Day"

Code for DeSTA2.5-Audio, general-purpose LALM

A single-layer, streaming codec model providing SOTA audio quality and discrete tokens designed for superior downstream modelability.

Code and model for ICASSP 2025 Paper "Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Data"

Streamable Text-to-Speech model using a language modeling approach, without vector quantization

Ultra-low-bitrate Speech Codec for Speech Language Modeling Applications

[EMNLP 2026 Main] Speaking While Listening: a survey and empirical audit of full-duplex spoken dialogue systems — L0–L3 architectural hierarchy, T×I×R interaction ontology, and a five-state decision machine, with a curated list of models, datasets, and benchmarks.

A high-performance, universal serving framework for any-to-any models.

Survey of audio language models

The official code for the SALMon🍣 benchmark (ICASSP 2025 - Oral)

[IEEE OJSP'26, IEEE SLT'24] "Speaker-Disentangled Chunk-Wise Regression for Syllabic Tokenization"

A collections of audio codecs with a standardized API

a fully open-source implementation of a GPT-4o-like speech-to-speech video understanding model.

End-to-end spoken language understanding toolkit: a small trainable neural audio codec with RVQ, joint intent-and-slot models over speech features, streaming partial intents, a speech-LLM instruction data pipeline, and turn-level evaluation with reproducible reports. NumPy core with a CPU-only torch extra; synthetic data only, fully offline.

[EMNLP 2026] FlexiSLM: A Spoken Language Model with Dynamic and Controllable Frame Rates