AI Gateway Tool Use and Function Calling
Connect a model to functions in your application, such as a weather lookup or a database query. The model chooses a tool and supplies arguments; your application decides how to execute it.
Set AI_GATEWAY_API_KEY and install your SDK or API client. Choose a model with tool-use support.
- Define the tool name, description, and argument schema.
- Send the tool definitions with the conversation.
- Validate the requested arguments and execute the matching application function.
- Return the result using the original tool-call ID.
- Continue the conversation until the model answers or your step limit stops the loop.
The examples below request a weather lookup. The Python beta runs an example function that returns fixed weather data. Other tabs print the requested function call so you can inspect it before execution.
Define a tool schema so the model can request a function call. The TypeScript and HTTP examples print the requested call; your application validates the arguments, executes the function, and sends a tool result. The Python beta example uses ai.Agent to execute the example function and continue the conversation automatically.
Tool-call and tool-result fields differ between Chat Completions, Messages, and Responses. With AI SDK 7, add an execute function and stopWhen: isStepCount(3) to run a bounded tool loop.
tools.ts
import { generateText, tool } from 'ai';
import { z } from 'zod';
const { toolCalls } = await generateText({
tools: {
get_weather: tool({
description: 'Get the weather for a city.',
inputSchema: z.object({ city: z.string() }),
}),
},
toolChoice: 'required',
model: "anthropic/claude-sonnet-5",
prompt: "What is the weather in San Francisco?",
});
console.log(toolCalls);tools_ai.py
import asyncio
import ai
@ai.tool
async def get_weather(city: str) -> str:
"""Get the weather for a city (example data)."""
return f"The example weather in {city} is sunny, 64F."
async def main():
model = ai.get_model("anthropic/claude-sonnet-5")
messages = [ai.user_message("What is the weather in San Francisco?")]
agent = ai.Agent(tools=[get_weather])
async with agent.run(model, messages) as stream:
async for event in stream:
if isinstance(event, ai.events.TextDelta):
print(event.chunk, end="", flush=True)
print()
asyncio.run(main())tools-chat.ts
import OpenAI from 'openai';
const client = new OpenAI({
apiKey: process.env.AI_GATEWAY_API_KEY,
baseURL: 'https://ai-gateway.vercel.sh/v1',
});
const response = await client.chat.completions.create({
tools: [
{
type: 'function',
function: {
name: 'get_weather',
description: 'Get the weather for a city.',
parameters: {
type: 'object',
properties: {
city: {
type: 'string',
},
},
required: ['city'],
additionalProperties: false,
},
},
},
],
tool_choice: 'required',
model: 'anthropic/claude-sonnet-5',
messages: [
{
role: 'user',
content: 'What is the weather in San Francisco?',
},
],
});
console.log(response.choices[0]?.message.tool_calls);tools_chat.py
import os
from openai import OpenAI
client = OpenAI(
api_key=os.environ["AI_GATEWAY_API_KEY"],
base_url="https://ai-gateway.vercel.sh/v1",
)
response = client.chat.completions.create(
tools=[{"type": "function", "function": {"name": "get_weather", "description": "Get the weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"], "additionalProperties": False}}}],
tool_choice="required",
model="anthropic/claude-sonnet-5",
messages=[{"role": "user", "content": "What is the weather in San Francisco?"}],
)
print(response.choices[0].message.tool_calls)tools-chat.sh
curl --fail-with-body https://ai-gateway.vercel.sh/v1/chat/completions \
-H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "anthropic/claude-sonnet-5",
"messages": [
{
"role": "user",
"content": "What is the weather in San Francisco?"
}
],
"tools": [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the weather for a city.",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string"
}
},
"required": [
"city"
],
"additionalProperties": false
}
}
}
],
"tool_choice": "required"
}'tools-messages.ts
import Anthropic from '@anthropic-ai/sdk';
const client = new Anthropic({
apiKey: process.env.AI_GATEWAY_API_KEY,
baseURL: 'https://ai-gateway.vercel.sh',
});
const response = await client.messages.create({
tools: [
{
name: 'get_weather',
description: 'Get the weather for a city.',
input_schema: {
type: 'object',
properties: {
city: {
type: 'string',
},
},
required: ['city'],
additionalProperties: false,
},
},
],
tool_choice: {
type: 'any',
},
model: 'anthropic/claude-sonnet-5',
messages: [
{
role: 'user',
content: 'What is the weather in San Francisco?',
},
],
max_tokens: 1024,
});
console.log(response.content);tools_messages.py
import os
from anthropic import Anthropic
client = Anthropic(
api_key=os.environ["AI_GATEWAY_API_KEY"],
base_url="https://ai-gateway.vercel.sh",
)
response = client.messages.create(
tools=[{"name": "get_weather", "description": "Get the weather for a city.", "input_schema": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"], "additionalProperties": False}}],
tool_choice={"type": "any"},
model="anthropic/claude-sonnet-5",
messages=[{"role": "user", "content": "What is the weather in San Francisco?"}],
max_tokens=1024,
)
print(response.content)tools-messages.sh
curl --fail-with-body https://ai-gateway.vercel.sh/v1/messages \
-H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
-H "Content-Type: application/json" \
-H "anthropic-version: 2023-06-01" \
-d '{
"model": "anthropic/claude-sonnet-5",
"messages": [
{
"role": "user",
"content": "What is the weather in San Francisco?"
}
],
"max_tokens": 1024,
"tools": [
{
"name": "get_weather",
"description": "Get the weather for a city.",
"input_schema": {
"type": "object",
"properties": {
"city": {
"type": "string"
}
},
"required": [
"city"
],
"additionalProperties": false
}
}
],
"tool_choice": {
"type": "any"
}
}'tools-responses.ts
import OpenAI from 'openai';
const client = new OpenAI({
apiKey: process.env.AI_GATEWAY_API_KEY,
baseURL: 'https://ai-gateway.vercel.sh/v1',
});
const response = await client.responses.create({
tools: [
{
type: 'function',
name: 'get_weather',
description: 'Get the weather for a city.',
parameters: {
type: 'object',
properties: {
city: {
type: 'string',
},
},
required: ['city'],
additionalProperties: false,
},
strict: true,
},
],
tool_choice: 'required',
model: 'anthropic/claude-sonnet-5',
input: 'What is the weather in San Francisco?',
});
console.log(response.output);tools_responses.py
import os
from openai import OpenAI
client = OpenAI(
api_key=os.environ["AI_GATEWAY_API_KEY"],
base_url="https://ai-gateway.vercel.sh/v1",
)
response = client.responses.create(
tools=[{"type": "function", "name": "get_weather", "description": "Get the weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"], "additionalProperties": False}, "strict": True}],
tool_choice="required",
model="anthropic/claude-sonnet-5",
input="What is the weather in San Francisco?",
)
print(response.output)tools-responses.sh
curl --fail-with-body https://ai-gateway.vercel.sh/v1/responses \
-H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "anthropic/claude-sonnet-5",
"input": "What is the weather in San Francisco?",
"tools": [
{
"type": "function",
"name": "get_weather",
"description": "Get the weather for a city.",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string"
}
},
"required": [
"city"
],
"additionalProperties": false
},
"strict": true
}
],
"tool_choice": "required"
}'For tool definitions and execution loops, see the AI SDK tool-calling guide and the Python beta tools guide.
Use the tool-call ID returned by the model, including when it requests several calls in one response. Preserve the response items required by the chosen API when building the next turn:
| Format | Tool request | Tool result and continuation |
|---|---|---|
| AI SDK 7 | toolCalls | Add execute to the tool and stopWhen: isStepCount(3) for a bounded loop. See the SDK tool-loop example. |
| AI SDK for Python (beta) | ai.tool functions registered with ai.Agent | The agent executes the registered functions and continues the conversation. See Python tool calling. |
| Chat Completions | Assistant tool_calls | Append the assistant message, then a role: 'tool' message with tool_call_id. See Chat tool calling. |
| Messages API | tool_use content block | Return a user message containing tool_result with tool_use_id. See Messages tool calling. |
| Responses / OpenResponses | function_call output item | Return function_call_output with call_id. See Responses tools and OpenResponses tools. |
For production tools, authorize access in your application and validate arguments before executing a function. Set an application step limit so repeated calls cannot run indefinitely. Handle tool errors explicitly and keep credentials out of tool results.
Provider-executed tools such as web search use provider-specific definitions and behavior. Follow their guides instead of treating them as application functions.
To connect a coding agent or inspect model choices from the CLI, use the current Vercel CLI:
terminal
npx vercel ai-gateway models list
npx vercel ai-gateway setupSee coding agents for supported clients and setup instructions.
Last updated September 13, 2026
Was this helpful?