LiveKit Inference

Overview

LiveKit Inference provides access to many of the best models and providers for voice agents, including STT, LLM, TTS, and realtime models from OpenAI, Google, AssemblyAI, Deepgram, Cartesia, Fish Audio, Inworld, and more. LiveKit Inference is included with LiveKit Cloud and doesn't require additional plugins. It's zero data retention by default. Your prompts, audio, and model outputs are never stored or used to train models. To learn more about supported models and configuration options, see the LLM, STT, TTS, and realtime topics.

By default, LiveKit Inference routes requests to model providers without regional restrictions. To restrict requests to models served from your project's data region, such as for EU data residency, enable region restriction.

To learn more about LiveKit Inference, see the blog post Introducing LiveKit Inference: A unified model interface for voice AI .

For LiveKit Inference models, use the inference module classes in your AgentSession:

from livekit.agents import AgentSession, inference

session = AgentSession(
    stt=inference.STT(
        model="deepgram/flux-general",
        language="en"
    ),
    llm=inference.LLM(
        model="google/gemma-4-31b-it",
    ),
    tts=inference.TTS(
        model="fishaudio/s2.1-pro",
        voice="fa4c9eb3dccc4806b382b40d61c6b10a",
    ),
)

import { AgentSession, inference } from '@livekit/agents';

const session = new AgentSession({
    stt: new inference.STT({
        model: "deepgram/flux-general",
        language: "en"
    }),
    llm: new inference.LLM({
        model: "google/gemma-4-31b-it",
    }),
    tts: new inference.TTS({
        model: "fishaudio/s2.1-pro",
        voice: "fa4c9eb3dccc4806b382b40d61c6b10a",
    }),
});

String descriptors

As a shortcut, you can pass a model descriptor string directly instead of using the inference classes. This is a convenient way to get started quickly.

from livekit.agents import AgentSession

session = AgentSession(
    stt="assemblyai/universal-3-6-pro:en",
    llm="google/gemma-4-31b-it",
    tts="fishaudio/s2.1-pro:fa4c9eb3dccc4806b382b40d61c6b10a",
)

import { AgentSession } from '@livekit/agents';

const session = new AgentSession({
    stt: "assemblyai/universal-3-6-pro:en",
    llm: "google/gemma-4-31b-it",
    tts: "fishaudio/s2.1-pro:fa4c9eb3dccc4806b382b40d61c6b10a",
});

For detailed parameter references and model-specific options, see the individual model guides for LLM, STT, and TTS.

Authentication

LiveKit Inference uses your LiveKit Cloud credentials. You don't need a separate API key for each model provider. Set LIVEKIT_API_KEY and LIVEKIT_API_SECRET in your .env file, or pass api_key and api_secret directly to an inference class. In Node.js, these parameters are apiKey and apiSecret.

Zero data retention

LiveKit Inference is zero data retention (ZDR) by default. Your prompts, audio, and model outputs pass through only to generate a response. Neither LiveKit nor the underlying model providers log, store, or train on your data. This applies to every LLM, STT, TTS, and realtime model, on every plan, with no configuration required.

Region restriction

By default, LiveKit Inference can route requests to a model provider in any region. To keep inference requests within your project's data region, enable region restriction.

For new projects with agents hosted in the EU, region restriction is enabled by default. To enable it for an existing project, sign in to LiveKit Cloud  and select Project → Settings. In the Data and privacy section, set Inference region restriction to Enabled.

When enabled, LiveKit Inference only routes requests to models with an endpoint in your project's data region. For a project with the EU data region, this limits inference to models served from the EU. Models without an endpoint in your data region are shown as unavailable in the Agent Builder.

Region restriction applies only to models routed through LiveKit Inference. Provider plugins send requests directly to the provider, so you must configure the provider's regional endpoint separately. To configure EU data residency across the rest of your stack, see EU data residency.

Data residency

Some models offer a dedicated endpoint in the EU, in addition to the global endpoint. A checkmark in the EU endpoint column of the model tables means the model has an EU endpoint, which lets you keep inference requests in the EU with region restriction.

Models

The following tables list all models currently available through LiveKit Inference.

Large language models (LLM)

Browse the available large language models by model family.

A checkmark in the EU endpoint column means the model has a dedicated EU endpoint, in addition to the global endpoint. To learn more, see Data residency.

Hosted by LiveKit

DeepSeek

Gemini

Kimi

OpenAI

SpaceXAI

Speech-to-text (STT)

Browse the available speech-to-text models by model family.

A checkmark in the EU endpoint column means the model has a dedicated EU endpoint, in addition to the global endpoint. To learn more, see Data residency.

AssemblyAI

Deepgram

Cartesia

Gemini

Inworld

OpenAI

Speechmatics

SpaceXAI

Text-to-speech (TTS)

Browse the available text-to-speech models by model family.

A checkmark in the EU endpoint column means the model has a dedicated EU endpoint, in addition to the global endpoint. To learn more, see Data residency.

Cartesia

Deepgram

Fish Audio

Gemini

Gradium

Inworld

Rime

SpaceXAI

Realtime models

Browse the available realtime models by provider. Realtime models, also called speech-to-speech models, consume and produce speech directly. They replace the cascaded STT-LLM-TTS pipeline. To learn more about how they differ, see the realtime models overview. Each model is either turn-based or full-duplex.

A checkmark in the EU endpoint column means the model has a dedicated EU endpoint, in addition to the global endpoint. To learn more, see Data residency.

Gemini Live API

OpenAI GPT-Live

OpenAI Realtime API

SpaceXAI Grok Voice Agent API

To use a realtime model, pass an instance of the RealtimeModel class from the inference module to the llm argument of your AgentSession. The model ID needs the provider prefix.

from livekit.agents import AgentSession, inference

session = AgentSession(
    llm=inference.RealtimeModel(model="openai/gpt-realtime-2.1"),
)

import { AgentSession, inference } from '@livekit/agents';

const session = new AgentSession({
    llm: new inference.RealtimeModel({ model: "openai/gpt-realtime-2.1" }),
});

Turn detection and interruption

LiveKit Inference also provides the models that the Agents framework uses for turn-taking:

• The audio turn detector predicts when the user has finished speaking.
• Adaptive interruption handling distinguishes true interruptions from backchanneling while the agent is speaking.

Billing

LiveKit Inference billing is based on usage. Discounted rates are available on the Scale plan. Custom rates are available on the Enterprise plan. Billing for realtime models varies by model. LiveKit Inference bills some by input and output tokens, and others by the minute. For rates, see Realtime models  on the pricing page. The turn detection and interruption models are included at no extra cost for agents deployed to LiveKit Cloud. For self-hosted agents, usage beyond a monthly development allowance is billed per request on the Ship and Scale plans. On the Build plan, the session falls back to v1-mini and VAD-based interruption detection at the limit. For pricing information, see the LiveKit Inference pricing page . For more information on quotas, limits, and billing for LiveKit Inference, see the following topics.