An in-the-wild benchmark for AI agents in the production harness.
-
Updated
Sep 18, 2026 - Python
#
agentic-evaluation
Here are 19 public repositories matching this topic...
Gen AI Evaluation Toolkit on AWS is a flexible, cloud-native accelerator built on AWS serverless architecture that enables comprehensive evaluation of generative AI applications.
agent evaluation bedrock evaluation-framework llm agentic-ai agentcore agentic-evaluation agentic-evals
-
Updated
Oct 2, 2026 - TypeScript
DataClawEval: A Benchmark for Engineering Data Agents in Real Industrial Harness
-
Updated
Aug 5, 2026 - Python
A SnitchBench-style benchmark inverted for the dark-forest problem: does a listener AI alert humans about an alien signal when alerting may doom humanity?
-
Updated
Apr 27, 2026 - TypeScript
🚀 AI Evolution Factory - From evaluation tool to continuous AI self-improvement platform. Agentic evaluation, auto-finetuning, global P2P testing, and hardware telemetry for local LLMs
python benchmark arena ai evaluation p2p elo fine-tuning huggingface byzantine-consensus llm vllm local-llm ollama unsloth ai-evolution openai-compatible hardware-telemetry agentic-evaluation crypto-proof
-
Updated
Jun 23, 2026 - Python
Open-source verification for AI systems. Declare expected behavior, check real runs, and keep the evidence behind each result
python rust typescript ai monitoring verification ml assurance agentic-ai ai-infrastructure agentic-evaluation
-
Updated
Oct 3, 2026 - Rust
A small benchmark for agent skills, verification artifacts, and fresh-session resumability.
benchmarking verification developer-tools software-engineering-research ai-engineering coding-agents agent-benchmark agentic-evaluation workflow-evaluation
-
Updated
Jul 10, 2026 - Python
Autonomous Multi-Turn Agentic Metamorphic Fuzzer & Swarm Consensus Stress-Testing Engine. Formulates Martingale context drift estimators, metamorphic execution DAGs, and Sybil groupthink inoculation filters.
metamorphic-testing automated-testing multi-agent-systems ai-safety swarm-intelligence red-teaming byzantine-fault-tolerance llm-benchmarks agentic-evaluation context-drift
-
Updated
Oct 3, 2026 - Python
Proof-bound evaluator stress testing with oracle-witnessed reward-hacking exploits and replayable evidence.
-
Updated
Sep 3, 2026 - Python
Paired rollouts for group-relative RL of LLM agents in stochastic environments
reinforcement-learning statistical-analysis stochastic-processes interpretability agentic-evaluation
-
Updated
Sep 26, 2026 - Python
Two-tier honest-evaluation harness for agentic RTL design (research, WIP)
benchmark eda rtl verilog formal-verification hardware-design llm-agents agentic-evaluation build-with-paper
-
Updated
Jul 2, 2026 - Python
Runnable lab measuring invalidation/staleness in agent memory — paper: Are We Ready For An Agent-Native Memory System? (arXiv:2606.24775)
benchmark invalidation mlx memory-systems bi-temporal rag paper-reproduction llm kuzu llm-agents agent-memory agentic-evaluation build-with-paper
-
Updated
Jun 26, 2026 - Python
Pipeline to investigate structured reasoning and instruction adherence in multimodal LLMs
benchmark robustness grounding out-of-distribution neuro-symbolic robustness-verification instruction-following trustworthy-ai large-language-models faithfulness hallucination-detection agentic-ai llm-alignment agentic-evaluation agentic-reasoning deterministic-eval
-
Updated
May 28, 2026 - Python
The specification behind ESAC, a program of compact benchmarks that measure difficult capabilities without measuring budget. ESAC-GI is released and runnable; ESAC-AG has a harness and one of its seven categories. Items are generators rather than stored questions, and every score carries the version tag of the instrument that produced it.
documentation benchmark constitution evaluation specification mit-license reproducibility benchmark-suite contamination llm esac evals llm-evaluation agentic-ai general-intelligence ai-benchmark llm-benchmark agentic-evaluation ecowestern
-
Updated
Sep 23, 2026 - JavaScript
Scorekeeper is a server that benchmarks AI agents and assistant platforms
-
Updated
Sep 10, 2026 - Python
Deterministic synthetic fixtures and hard-gate scoring for agentic biosafety safeguard routing.
-
Updated
Sep 27, 2026 - Python
A Harbor-format RL environment for pass@k estimation, with a verifier built to resist reward hacking. 17 tests, 10 adversarial solutions rejected, amd64 CI.
docker pytest harbor rl-environments llm-evaluation reward-hacking terminal-bench agentic-evaluation
-
Updated
Sep 8, 2026 - Python
Measure AI agents’ performance with standardized tests across 314 tasks, 33 domains, and 4 difficulty levels for clear, reproducible comparison.
agent benchmark ai discovery openai benchmarks codex claude ai4science llm ai-scientist clawdbot openclaw agentic-evaluation auto-research research-claw
-
Updated
Oct 3, 2026 - Python
A GPU-backed Harbor/Terminal-Bench RL environment for data-poisoning defense: containerized task, oracle solution, and a 25-check verifier that rejects 17 reward-hacking solutions.
docker pytest harbor adversarial-machine-learning rl-environments data-poisoning llm-evaluation reward-hacking terminal-bench agentic-evaluation
-
Updated
Sep 8, 2026 - Python
Add this topic to your repo
To associate your repository with the agentic-evaluation topic, visit your repo's landing page and select "manage topics."