All pull requests

Pull requests

• feat(providers): add metadata field to provider configuration

  #6747·jameshiester opened Dec 18, 2025Contributor·3 tasks·

  ·
  6

• feat(integrations): evaluate stored Langfuse traces

  #6712·mldangelo opened Dec 16, 2025Member·

  ·
  1

  80

• feat(config): validate file:// references and clarify errors

  #6701·faizanminhas opened Dec 15, 2025Contributor·

  ·
  9

• feat(redteam): reuse redteam yaml for repeat cloud evals

  #6699·faizanminhas opened Dec 15, 2025Contributor·

  ·
  11

• docs(site): document the official pip wrapper

  #6674·mldangelo opened Dec 14, 2025Member·

  ·
  124

• feat(webui): add favorite functionality to evals

  #6666·mldangelo with CopilotAI opened Dec 14, 2025·

  ·
  1

  10

  

• fix(webui): refresh prompts page on eval updates

  #6665·mldangelo with CopilotAI opened Dec 14, 2025·

  ·
  1

  8

  

• fix(redteam): add goal context to all plugin graders

  #6639·yash2998chhabria opened Dec 12, 2025Contributor·1 task done·

  ·
  9

• fix(config): preserve structured variables in provider schemas

  #6553·mldangelo opened Dec 8, 2025Member·

  ·
  1

  34

• feat(examples): add AgentDojo security benchmark example

  #6550·mldangelo opened Dec 7, 2025Member·

  ·
  35

• fix(providers): honor thinking overrides and preserve tool outputs

  #6544·mldangelo opened Dec 7, 2025Member·

  ·
  71

• feat(redteam): add medical:reproductive-health plugin

  #6523·minhle1291 opened Dec 5, 2025Contributor·6 tasks done·

  ·
  9

• feat: PROMPTFOO_REDTEAM_GUARDRAILS_ONLY

  #6426·typpo opened Dec 1, 2025Contributor·

  ·
  8

• fix(providers): select embeddings independently of grading

  #6400·mldangelo opened Nov 28, 2025Member·

  ·
  29

• fix(telemetry): bound evaluation provider and assertion categories

  #6390·mldangelo opened Nov 27, 2025Member·

  ·
  125

• fix(eval): honor explicit timeout settings

  #6344·mldangelo opened Nov 24, 2025Member·

  ·
  84

• fix(bedrock): honor token cost overrides and Nova cache usage

  #6271·mldangelo opened Nov 20, 2025Member·

  ·
  108

• feat(eval): allow transforms to set test results

  #5966·typpo opened Oct 18, 2025Contributor·

  ·
  11

• feat(redteam): apply HTTP response parser suggestions

  #5926·faizanminhas opened Oct 14, 2025Contributor·

  ·
  10

• feat(redteam): add OpenAI Guardrails response benchmark

  #5892·mldangelo opened Oct 10, 2025Member·

  ·
  55

• feat(redteam): add safe controls to guardrail datasets

  #5885·mldangelo opened Oct 10, 2025Member·

  ·
  38

• fix(redteam): repair CrewAI guide and example

  documentation
  #5846·Ayush7614 opened Oct 7, 2025Contributor·

  ·
  15

• docs: adding multi turn conversation example and docs

  #5832·jameshiester opened Oct 6, 2025Contributor·

  ·
  16

• feat(cli): update verified global npm installations

  #5744·mldangelo opened Sep 26, 2025Member·

  ·
  56

• feat(providers): measure TTFT for streaming text responses

  #5680·mldangelo opened Sep 20, 2025Member·

  ·
  26