Add support for system instructions in the ondevice sdk. - #8663
VinayGuthal wants to merge 2 commits into
Conversation
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. |
📝 PRs merging into main branchOur main branch should always be in a releasable state. If you are working on a larger change, or if you don't want this change to see the light of the day just yet, consider using a feature branch first, and only merge into the main branch when the code complete and ready to be released. |
VinayGuthal
changed the title
|
/gemini review |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces support for systemInstruction in on-device content generation requests across the firebase-ai-ondevice-interop, firebase-ai-ondevice, and firebase-ai modules. It includes API updates, converters, validation logic to ensure only text parts are used in system instructions, and comprehensive unit tests. The feedback suggests optimizing the validation logic in OnDeviceGenerativeModelProvider to avoid filtering the parts list twice in the happy path.
| val unsupportedSystemParts = instruction.parts.filter { it !is TextPart } | ||
| if (unsupportedSystemParts.isNotEmpty()) { | ||
| throw FirebaseAIException.from( | ||
| FirebaseAIOnDeviceInvalidRequestException( | ||
| IllegalArgumentException( | ||
| "On-device model does not support non-text parts in systemInstruction: ${unsupportedSystemParts.map { it::class.java.simpleName }}" | ||
| ) | ||
| ) | ||
| ) | ||
| } | ||
| val systemTextParts = | ||
| instruction.parts.filterIsInstance<TextPart>().also { | ||
| if (it.size > 1) | ||
| Log.w( | ||
| TAG, | ||
| "On-device model does not support multiple text parts in systemInstruction, concatenating them instead" | ||
| ) | ||
| } |
There was a problem hiding this comment.
We can optimize this logic to avoid filtering the parts list twice in the happy path (when all parts are TextPart). By checking if the size of the filtered TextPart list is less than the total parts list, we can determine if there are any unsupported parts and only perform the second filter/error handling when necessary.
val systemTextParts = instruction.parts.filterIsInstance<TextPart>()
if (systemTextParts.size < instruction.parts.size) {
val unsupportedSystemParts = instruction.parts.filter { it !is TextPart }
throw FirebaseAIException.from(
FirebaseAIOnDeviceInvalidRequestException(
IllegalArgumentException(
\"On-device model does not support non-text parts in systemInstruction: ${unsupportedSystemParts.map { it::class.java.simpleName }}\"
)
)
)
}
if (systemTextParts.size > 1) {
Log.w(
TAG,
\"On-device model does not support multiple text parts in systemInstruction, concatenating them instead\"
)
}There was a problem hiding this comment.
can you give a suggested commit
rlazo
left a comment
rlazo
left a comment
There was a problem hiding this comment.
Shouldn't the other api.txt files be updated too?
| @@ -1,5 +1,6 @@ | |||
| # Unreleased | |||
|
|
|||
| - [feature] Added `systemInstruction` property to `GenerateContentRequest`. | |||
| @@ -1,5 +1,6 @@ | |||
| # Unreleased | |||
|
|
|||
| - [feature] Added support for `systemInstruction` in on-device content generation requests. | |||
| @@ -1,5 +1,6 @@ | |||
| # Unreleased | |||
|
|
|||
| - [feature] Added support for `systemInstruction` in on-device and hybrid inference modes. | |||
This branch has not been deployed
2 participants
Adds support for
systemInstructionwhen generating content using on-device and hybrid inference modes.Summary of Changes
firebase-ai-ondevice-interop: Added an optionalsystemInstruction: TextPart?property toGenerateContentRequestand updatedapi.txt.firebase-ai-ondevice: UpdatedGenerateContentRequest.toMlKit()inConverters.ktto convert the interopTextPartinto an ML KitSystemInstruction.firebase-ai:systemInstructionfromGenerativeModel.BuildertoOnDeviceGenerativeModelProvider.OnDeviceGenerativeModelProviderto validate thatsystemInstructioncontains only text parts (throwingFirebaseAIOnDeviceInvalidRequestExceptionfor non-text parts), concatenate multipleTextPartinstances with a newline, and attach it to the on-device request.ConvertersTest,GenerativeModelBuilderTests, andOnDeviceGenerativeModelProviderTests, and updatedCHANGELOG.mdacross all three modules.