Skip to content

Gemma model-family validation is skipped under python -O #7355

Description

@MateehUllah

🔴 Required Information

Describe the Bug:

Gemma.generate_content_async() in src/google/adk/models/gemma_llm.py uses a Python assert to ensure that the requested model is a Gemma model:

model = llm_request.model
assert model is not None and model.startswith('gemma-'), (
    f'Requesting a non-Gemma model ({model}) with the Gemma LLM'
    ' is not supported.'
)

Python removes assert statements when running with optimization (python -O or python -OO). Therefore, the Gemma-only model validation is not enforced in optimized mode, allowing an invalid non-Gemma model to proceed past this validation point.

This also conflicts with the repository's current style guidance under No Asserts in Production Code, which notes that assertions are stripped under python -O and recommends explicit exceptions for runtime validation.

Steps to Reproduce:

  1. Use the same validation logic currently present in Gemma.generate_content_async():
def validate(model):
    assert model is not None and model.startswith("gemma-"), (
        f"Requesting a non-Gemma model ({model}) "
        "with the Gemma LLM is not supported."
    )
    return "continued"


print(validate("gemini-2.5-flash"))
  1. Run it normally:
python repro.py
  1. The invalid model is rejected with an AssertionError.

  2. Run the same code with Python optimization enabled:

python -O repro.py
  1. The assertion is removed and execution continues:
continued

The production assertion in Gemma.generate_content_async() has the same Python optimization semantics.

Expected Behavior:

A non-Gemma model should be rejected consistently by Gemma.generate_content_async() regardless of whether Python is running normally or with -O/-OO.

Runtime validation should use an explicit exception, for example:

if model is None or not model.startswith('gemma-'):
  raise ValueError(
      f'Requesting a non-Gemma model ({model}) with the Gemma LLM '
      'is not supported.'
  )

Observed Behavior:

During normal Python execution, the assert rejects a non-Gemma model.

When Python is executed with -O or -OO, the assertion is removed. The model-family guard therefore disappears and execution can continue to the underlying generate_content_async() implementation.

Environment Details:

  • ADK Library Version (pip show google-adk): N/A — identified in the current repository source
  • Desktop OS: N/A
  • Python Version (python -V): N/A

Model Information:

  • Are you using LiteLLM: No
  • Which model is being used: gemini-2.5-flash as an example invalid non-Gemma model

🟡 Optional Information

Regression:

N/A — I have not verified whether an earlier ADK version used explicit validation here.

Logs:

Normal Python execution:

AssertionError

With assertions disabled using python -O:

continued

Screenshots / Video:

N/A

Additional Context:

The affected code is:

src/google/adk/models/gemma_llm.py

in:

Gemma.generate_content_async()

The repository's current style guide at:

.agents/skills/adk-style/references/typing.md

explicitly states under No Asserts in Production Code that assert is stripped when Python runs with -O, so assertions should not be used as runtime guarantees. It recommends raising ValueError, TypeError, or RuntimeError instead.

I also searched the existing issues for this specific Gemma.generate_content_async() / python -O behavior and did not find a matching report.

Minimal Reproduction Code:

def validate(model):
    assert model is not None and model.startswith("gemma-"), (
        f"Requesting a non-Gemma model ({model}) "
        "with the Gemma LLM is not supported."
    )
    return "continued"


print(validate("gemini-2.5-flash"))

Run normally:

python repro.py

Then compare with:

python -O repro.py

How often has this issue occurred?:

  • Always (100%)
  • Often (50%+)
  • Intermittently (<50%)
  • Once / Rare

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

models[Component] This issue is related to model supportrequest clarification[Status] The maintainer need clarification or more information from the author

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions