Skip to content

[BUG] CriticAgent overwrites ChatAgent's retry_attempts clamp, so retry_attempts=0 answers at random without calling the model #4378

Description

@chrikrah

Required prerequisites

What version of camel are you using?

0.2.91a7 (master fc27907)

System information

Master fc27907, Python 3.12.3, Ubuntu 24.04.5 x86_64. Nothing below reaches the network.

Problem description

ChatAgent.__init__ guards the retry count at chat_agent.py:649 with self.retry_attempts = max(1, retry_attempts). CriticAgent.__init__ calls super().__init__(...) without passing it, then overwrites the guarded value at critic_agent.py:97 with self.retry_attempts = retry_attempts. The clamp applies to a ChatAgent and not to a CriticAgent.

Two loops then assume at least one pass. get_option at :138 is while i < self.retry_attempts, so at 0 it falls to the warnings.warn at :165 and returns random.choice(...): the critic answers at random and the model is never consulted. _get_model_response at chat_agent.py:3666 is for attempt in range(...) with an else: raise, so at 0 it raises Unable to process messages: Unknown error with last_error still None. clone() passes the value on at :251.

Reproducible example code

The Python snippets:

import os
os.environ.setdefault("OPENAI_API_KEY", "sk-dummy")  # nothing below reaches the network

from unittest.mock import MagicMock, patch
from camel.agents import ChatAgent, CriticAgent
from camel.messages import BaseMessage
from camel.types import RoleType

print("ChatAgent(retry_attempts=0)   ->", ChatAgent("sys", retry_attempts=0).retry_attempts)
critic = CriticAgent(system_message="sys", retry_attempts=0)
print("CriticAgent(retry_attempts=0) ->", critic.retry_attempts)
print("CriticAgent(retry_attempts=-5)->", CriticAgent(system_message="sys", retry_attempts=-5).retry_attempts)
print("critic.clone()                ->", critic.clone().retry_attempts)

critic.options_dict = {"1": "proposal A", "2": "proposal B"}
msg = BaseMessage(role_name="user", role_type=RoleType.USER, meta_dict=None, content="pick one")
with patch.object(CriticAgent, "step", autospec=True,
                  side_effect=AssertionError("step() was called")) as step:
    choice = critic.get_option(msg)
print("get_option() returned         =", repr(choice), "| step() calls:", step.call_count)

critic.model_backend.run = MagicMock()
try:
    critic.step("hello")
except Exception as e:
    print(f"critic.step('hello')          -> {type(e).__name__}: {e}")
print("model_backend.run calls       =", critic.model_backend.run.call_count)

Command lines:

$ python -W always repro.py
critic_agent.py:165: UserWarning: Critic failed to get a valid option. After 0 attempts. Returning a random option.
ChatAgent(retry_attempts=0)   -> 1
CriticAgent(retry_attempts=0) -> 0
CriticAgent(retry_attempts=-5)-> -5
critic.clone()                -> 0
get_option() returned         = 'proposal B' | step() calls: 0
critic.step('hello')          -> ModelProcessingError: Unable to process messages: Unknown error
model_backend.run calls       = 0

Expected behavior

retry_attempts=0 becomes 1, one model call happens, and clone() carries the clamped value. Either pass it through to super().__init__ and drop line 97, or repeat the max(1, ...) there. The pass-through keeps one definition of the floor.

Additional context

Not a duplicate. #4360 also concerns CriticAgent.clone() but a different attribute, message_window_size. Of six open pull requests touching critic_agent.py, none changes line 97; #3950 only forwards the value through a clone.

I did not run tests/agents/test_critic_agent.py, four of whose tests need a real credential. If retry_attempts=0 is deliberately the random-choice path, the docstring at :60 is what needs changing instead.

Want the pass-through or the repeated clamp? I can send either with a regression test once this is accepted.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions