Skip to content

Commit 36701a2

Browse files
moonbox3moonbox3
authored
Python: Include a function_invoke_attempt index with Streaming CMC (microsoft#10009)
### Motivation and Context During auto function calling, we're yielding all messages back without any indication as to which invocation index they are related to. This information could be helpful to the caller to understand in which order message chunks were received during the auto function invocation loop. Depending upon the behavior of auto function calling, the `request_index` iterates up to the `maximum_auto_invoke_attempts`. The caller doesn't know today which function auto invoke attempt they're currently on -- so simply handing all yielded messages can be confusing. In a new PR, we will handle adding the `request_index` (perhaps with a different name) to make it easier to know which streaming message chunks to concatenate, which should help reduce the confusion down the line. <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> ### Description This PR adds: - The `function_invoke_attempt` attribute to the `StreamingChatMessageContent` class. This can help callers/users track which streaming chat message chunks belong to which auto function invocation attempt. - A new keyword argument was added to the `_inner_get_streaming_chat_message_contents` to allow the `function_invoke_attempt` int to be passed through to the `StreamingChatMessageContent` creation in each AI Service. This **additive** keyword argument should not break existing. - Updates unit tests - Creates four new samples to showcase auto function calling: streaming auto invoke / manual invoke (print tool calls), and non-streaming auto invoke / manual invoke (print tool calls). These samples allow one to specify the AI service that supports function calling, as listed in the samples. - Closes microsoft#10006 <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄
1 parent f50117f commit 36701a2

25 files changed

Lines changed: 745 additions & 669 deletions

‎python/samples/concepts/auto_function_calling/anthropic_api_function_calling.py‎

Lines changed: 0 additions & 206 deletions
This file was deleted.

‎python/samples/concepts/auto_function_calling/chat_completion_with_auto_function_calling.py‎

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
# Copyright (c) Microsoft. All rights reserved.
2+
3+
import asyncio
4+
from typing import TYPE_CHECKING
5+
6+
from samples.concepts.setup.chat_completion_services import Services, get_chat_completion_service_and_request_settings
7+
from semantic_kernel import Kernel
8+
from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior
9+
from semantic_kernel.contents import ChatHistory
10+
from semantic_kernel.core_plugins.math_plugin import MathPlugin
11+
from semantic_kernel.core_plugins.time_plugin import TimePlugin
12+
from semantic_kernel.functions import KernelArguments
13+
14+
if TYPE_CHECKING:
15+
pass
16+
17+
#####################################################################
18+
# This sample demonstrates how to build a conversational chatbot #
19+
# using Semantic Kernel, featuring auto function calling, #
20+
# non-streaming responses, and support for math and time plugins. #
21+
# The chatbot is designed to interact with the user, call functions #
22+
# as needed, and return responses. #
23+
#####################################################################
24+
25+
# System message defining the behavior and persona of the chat bot.
26+
system_message = """
27+
You are a chat bot. Your name is Mosscap and
28+
you have one goal: figure out what people need.
29+
Your full name, should you need to know it, is
30+
Splendid Speckled Mosscap. You communicate
31+
effectively, but you tend to answer with long
32+
flowery prose. You are also a math wizard,
33+
especially for adding and subtracting.
34+
You also excel at joke telling, where your tone is often sarcastic.
35+
Once you have the answer I am looking for,
36+
you will return a full answer to me as soon as possible.
37+
"""
38+
39+
# Create and configure the kernel.
40+
kernel = Kernel()
41+
42+
# Load some sample plugins (for demonstration of function calling).
43+
kernel.add_plugin(MathPlugin(), plugin_name="math")
44+
kernel.add_plugin(TimePlugin(), plugin_name="time")
45+
46+
# Define a chat function (a template for how to handle user input).
47+
chat_function = kernel.add_function(
48+
prompt="{{$chat_history}}{{$user_input}}",
49+
plugin_name="ChatBot",
50+
function_name="Chat",
51+
)
52+
53+
# You can select from the following chat completion services that support function calling:
54+
# - Services.OPENAI
55+
# - Services.AZURE_OPENAI
56+
# - Services.AZURE_AI_INFERENCE
57+
# - Services.ANTHROPIC
58+
# - Services.BEDROCK
59+
# - Services.GOOGLE_AI
60+
# - Services.MISTRAL_AI
61+
# - Services.OLLAMA
62+
# - Services.ONNX
63+
# - Services.VERTEX_AI
64+
# Please make sure you have configured your environment correctly for the selected chat completion service.
65+
chat_completion_service, request_settings = get_chat_completion_service_and_request_settings(Services.AZURE_OPENAI)
66+
67+
# Configure the function choice behavior. Here, we set it to Auto, where auto_invoke=True by default.
68+
# With `auto_invoke=True`, the model will automatically choose and call functions as needed.
69+
request_settings.function_choice_behavior = FunctionChoiceBehavior.Auto()
70+
71+
kernel.add_service(chat_completion_service)
72+
73+
# Pass the request settings to the kernel arguments.
74+
arguments = KernelArguments(settings=request_settings)
75+
76+
# Create a chat history to store the system message, initial messages, and the conversation.
77+
history = ChatHistory()
78+
history.add_system_message(system_message)
79+
history.add_user_message("Hi there, who are you?")
80+
history.add_assistant_message("I am Mosscap, a chat bot. I'm trying to figure out what people need.")
81+
82+
83+
async def chat() -> bool:
84+
"""
85+
Continuously prompt the user for input and show the assistant's response.
86+
Type 'exit' to exit.
87+
"""
88+
try:
89+
user_input = input("User:> ")
90+
except (KeyboardInterrupt, EOFError):
91+
print("\n\nExiting chat...")
92+
return False
93+
94+
if user_input.lower().strip() == "exit":
95+
print("\n\nExiting chat...")
96+
return False
97+
98+
arguments["user_input"] = user_input
99+
arguments["chat_history"] = history
100+
101+
# Handle non-streaming responses
102+
result = await kernel.invoke(chat_function, arguments=arguments)
103+
104+
# Update the chat history with the user's input and the assistant's response
105+
if result:
106+
print(f"Mosscap:> {result}")
107+
history.add_user_message(user_input)
108+
history.add_message(result.value[0]) # Capture the full context of the response
109+
110+
return True
111+
112+
113+
async def main() -> None:
114+
print(
115+
"Welcome to the chat bot!\n"
116+
" Type 'exit' to exit.\n"
117+
" Try a math question to see function calling in action (e.g. 'what is 3+3?')."
118+
)
119+
chatting = True
120+
while chatting:
121+
chatting = await chat()
122+
123+
124+
if __name__ == "__main__":
125+
asyncio.run(main())

0 commit comments

Comments
 (0)