Repository navigation
.NET: [Feature]: Accept AIJsonSchemaCreateOptions for the structured-output schema of AIAgent.RunAsync<T> #9245
Description
Problem
AIAgent.RunAsync<T> creates the response schema with ChatResponseFormat.ForJsonSchema<T>(serializerOptions) and then overwrites AgentRunOptions.ResponseFormat with it (AIAgentStructuredOutput.cs L127-L134; ChatClientAgent's overloads that take ChatClientAgentRunOptions forward to it, ChatClientAgentCustomOptions.cs L167-L232). An application cannot pass the AIJsonSchemaCreateOptions (a TransformSchemaNode) that it can already pass to AIFunctionFactory for its tools, and a ResponseFormat it sets itself is replaced, as ADR 0036 intends: RunAsync<T> owns the schema. This request is for a way to influence how it creates that schema, not to bypass it.
For a type serialized by a custom JsonConverter (a strongly typed ID, a value object such as an IBAN), the schema then says true for the property and {} for the items of a list of them: it does not even say the value is a string.
Use Case and Evidence
Found while adding Microsoft.Extensions.AI support to AdCodicem.ValueObjects, a library of value objects serialized by their own converters, and checking how its schemas reach an agent's structured output. An Order record holds an Iban, a struct written as a JSON string of 15 to 34 characters by its own converter, and a List<Iban>. The application has a TransformSchemaNode that describes Iban as {"type":"string","minLength":15,"maxLength":34,"pattern":"..."}. The repro calls RunAsync<Order> on a ChatClientAgent over a scripted IChatClient that records the ChatOptions.ResponseFormat it receives.
Output (Microsoft.Agents.AI 1.24.0, Microsoft.Extensions.AI 10.10.0, Microsoft.Extensions.AI.Abstractions 10.10.1, .NET 10.0.12, Windows 11 x64; same output on Ubuntu 24.04 x64):
agent.RunAsync<Order>(...): format the IChatClient receives
{"$schema":"https://json-schema.org/draft/2020-12/schema","type":"object","properties":{"account":true,"quantity":{"type":"integer"},"alternates":{"type":"array","items":{}}},"required":["account","quantity","alternates"]}
Result.Account = FR7630006000011234567890189
agent.RunAsync<Order>(..., options: new AgentRunOptions { ResponseFormat = wanted }): format the IChatClient receives
{"$schema":"https://json-schema.org/draft/2020-12/schema","type":"object","properties":{"account":true,"quantity":{"type":"integer"},"alternates":{"type":"array","items":{}}},"required":["account","quantity","alternates"]}
is it the caller's format? False
Wanted: AIJsonUtilities.CreateJsonSchema(typeof(Order), ..., inferenceOptions: schemaOptions)
{"$schema":"https://json-schema.org/draft/2020-12/schema","type":"object","properties":{"account":{"type":"string","minLength":15,"maxLength":34,"pattern":"^[A-Z]{2}[0-9]{2}[A-Z0-9]{11,30}$"},"quantity":{"type":"integer"},"alternates":{"type":"array","items":{"type":"string","minLength":15,"maxLength":34,"pattern":"^[A-Z]{2}[0-9]{2}[A-Z0-9]{11,30}$"}}},"required":["account","quantity","alternates"]}
Workaround (non-generic RunAsync + new AgentResponse<Order>): format received is the caller's? True; Result.Account = FR7630006000011234567890189
Repro (Program.cs, Repro.csproj): run with dotnet run
// Program.cs
using System.Text.Json;
using System.Text.Json.Nodes;
using System.Text.Json.Serialization;
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
const string Answer = """{"account":"FR7630006000011234567890189","quantity":3,"alternates":[]}""";
JsonSerializerOptions serializerOptions = AgentAbstractionsJsonUtilities.DefaultOptions; // what RunAsync<T> uses by default
// The format the application would like RunAsync<T> to send: Iban described with the rules its converter enforces.
var schemaOptions = new AIJsonSchemaCreateOptions { IncludeSchemaKeyword = true, TransformSchemaNode = DescribeIban };
ChatResponseFormatJson wanted = ChatResponseFormat.ForJsonSchema(
AIJsonUtilities.CreateJsonSchema(typeof(Order), serializerOptions: serializerOptions, inferenceOptions: schemaOptions),
"Order");
// 1. RunAsync<T>: the format the IChatClient receives.
var client = new ScriptedChatClient(Answer);
AIAgent agent = client.AsAIAgent(instructions: "You place orders.");
AgentResponse<Order> typed = await agent.RunAsync<Order>("Place an order.");
Print("agent.RunAsync<Order>(...): format the IChatClient receives", client.LastFormat!.Schema!.Value);
Console.WriteLine($" Result.Account = {typed.Result.Account}");
Console.WriteLine();
// 2. A ResponseFormat the caller sets on AgentRunOptions is replaced by RunAsync<T>.
typed = await agent.RunAsync<Order>("Place an order.", options: new AgentRunOptions { ResponseFormat = wanted });
Print("agent.RunAsync<Order>(..., options: new AgentRunOptions { ResponseFormat = wanted }): format the IChatClient receives",
client.LastFormat!.Schema!.Value);
Console.WriteLine($" is it the caller's format? {ReferenceEquals(client.LastFormat, wanted)}");
Console.WriteLine();
// 3. What the application wants sent.
Print("Wanted: AIJsonUtilities.CreateJsonSchema(typeof(Order), ..., inferenceOptions: schemaOptions)", wanted.Schema!.Value);
// 4. Today's workaround: the non-generic RunAsync with the format, then AgentResponse<T> by hand.
AgentResponse raw = await agent.RunAsync("Place an order.", options: new AgentRunOptions { ResponseFormat = wanted });
var wrapped = new AgentResponse<Order>(raw, serializerOptions);
Console.WriteLine($"Workaround (non-generic RunAsync + new AgentResponse<Order>): format received is the caller's? " +
$"{ReferenceEquals(client.LastFormat, wanted)}; Result.Account = {wrapped.Result.Account}");
static JsonNode DescribeIban(AIJsonSchemaCreateContext context, JsonNode schema)
{
// The schema of a type with a custom converter is `true` (or an object holding only a description).
if (context.TypeInfo.Type == typeof(Iban))
{
return Describe(schema as JsonObject ?? []);
}
// System.Text.Json's exporter does not call the transform for a collection element whose schema is `true`, so the
// collection's own node completes its `items`.
if (context.TypeInfo.ElementType == typeof(Iban) && schema is JsonObject collection)
{
if (collection["items"] is JsonObject items)
{
Describe(items);
}
else
{
collection["items"] = Describe([]);
}
}
return schema;
static JsonObject Describe(JsonObject node)
{
node["type"] = "string";
node["minLength"] = 15;
node["maxLength"] = 34;
node["pattern"] = "^[A-Z]{2}[0-9]{2}[A-Z0-9]{11,30}$";
return node;
}
}
static void Print(string title, JsonElement schema)
{
Console.WriteLine(title);
Console.WriteLine(" " + JsonSerializer.Serialize(schema));
}
/// <summary>A value object: serialized as a JSON string by its own converter, which enforces its rules.</summary>
[JsonConverter(typeof(IbanJsonConverter))]
public readonly struct Iban(string value)
{
public string Value { get; } = value;
public override string ToString() => Value;
}
public sealed class IbanJsonConverter : JsonConverter<Iban>
{
public override Iban Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
if (reader.TokenType != JsonTokenType.String)
{
throw new JsonException("An IBAN is a JSON string.");
}
string text = reader.GetString()!;
if (text.Length is < 15 or > 34)
{
throw new JsonException("An IBAN has 15 to 34 characters.");
}
return new Iban(text);
}
public override void Write(Utf8JsonWriter writer, Iban value, JsonSerializerOptions options)
=> writer.WriteStringValue(value.Value);
}
public sealed record Order(Iban Account, int Quantity, List<Iban> Alternates);
/// <summary>Stands for the model: records the response format it is sent, answers with a fixed text.</summary>
public sealed class ScriptedChatClient(string answer) : IChatClient
{
public ChatResponseFormatJson? LastFormat { get; private set; }
public Task<ChatResponse> GetResponseAsync(
IEnumerable<ChatMessage> messages, ChatOptions? options = null, CancellationToken cancellationToken = default)
{
LastFormat = options?.ResponseFormat as ChatResponseFormatJson;
return Task.FromResult(new ChatResponse(new ChatMessage(ChatRole.Assistant, answer)));
}
public IAsyncEnumerable<ChatResponseUpdate> GetStreamingResponseAsync(
IEnumerable<ChatMessage> messages, ChatOptions? options = null, CancellationToken cancellationToken = default)
=> throw new NotSupportedException();
public object? GetService(Type serviceType, object? serviceKey = null) => null;
public void Dispose()
{
}
}<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Agents.AI" Version="1.24.0" />
</ItemGroup>
</Project>The same as a project with a script: https://github.com/AdCodicem/dotnet-upstream-repros/tree/main/agent-framework-runasync-schema-options
AI assistance: the analysis, the reproduction and this text were prepared with Claude (Anthropic). The reproduction was run on Windows 11 and on Ubuntu 24.04, with the same output, and I reviewed the report, its links and its claims before filing.
Desired Outcome
RunAsync<T> can be given the AIJsonSchemaCreateOptions to create its schema with, so that the same options describe an application's types in its tools and in its agents' structured output, with RunAsync<T> still wrapping a non-object schema and reading the answer as it does today.
Alternatives Considered
- The non-generic
RunAsyncwith a hand-built format, thennew AgentResponse<T>(response, serializerOptions)(the "Workaround" line of the output). It works for aTwhose schema is an object. For any otherTthe application must also wrap the schema the wayRunAsync<T>does, sinceStructuredOutputSchemaUtilitiesis internal (StructuredOutputSchemaUtilities.csL15), and setIsWrappedInObject, which is public here, unlike in Microsoft.Extensions.AI. - Setting
AgentRunOptions.ResponseFormatand callingRunAsync<T>: replaced (second block of the output). - Data annotations on the properties (
[StringLength],[RegularExpression]): Microsoft.Extensions.AI adds the keywords but not"type", and on a list property they describe the list, not its items (shown in [API Proposal]: Accept AIJsonSchemaCreateOptions in ChatResponseFormat.ForJsonSchema<T> and GetResponseAsync<T> dotnet/extensions#7821).
Proposed Direction
Either of these would do; the second keeps the overload count unchanged.
- An overload of
RunAsync<T>that takesAIJsonSchemaCreateOptions? schemaCreateOptions. - A property on
AgentRunOptions, for instanceAIJsonSchemaCreateOptions? ResponseSchemaCreateOptions, whichRunAsync<T>reads when it creates the format.
RunAsync<T> can build the schema today with AIJsonUtilities.CreateJsonSchema(typeof(T), serializerOptions: serializerOptions, inferenceOptions: ...) and ChatResponseFormat.ForJsonSchema(JsonElement, name, description), both public, or with the overload proposed to Microsoft.Extensions.AI in dotnet/extensions#7821, if it is accepted.
Scope and Non-Goals
Only the schema RunAsync<T> sends. Reading and validating the answer is unchanged, and so is the default schema when no options are given.
Language
.NET
Contribution Intent
I am only requesting the feature
Acknowledgements
- I searched existing issues and did not find a duplicate.
- I will wait for explicit maintainer agreement before starting implementation of a non-trivial change.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsNo status
Activity
[-][Feature]: Accept AIJsonSchemaCreateOptions for the structured-output schema of AIAgent.RunAsync<T>[/-][+].NET: [Feature]: Accept AIJsonSchemaCreateOptions for the structured-output schema of AIAgent.RunAsync<T>[/+]