https://android-world.fluiz.ai/

https://abhishekdh.github.io/kirk-engine/

[09/30/2026] 91.4 --> Updated harness and using GPT-6 Astra
[10/6/2025]: 78.4 --> 91.4; details here
[09/19/2025]: Updated score from 63.0 --> 78.4. For this run, we used GPT-5 for reasoning in combination with Gemini-2.5-Pro for acting, while continuing to rely on a hybrid method of the Accessibility (A11y) tree plus corresponding screenshots to provide both structural and visual context to the agent.

[09/2026]: 97.4 --> 100. Trained open source models using our novel training technique on AndroidWorld tasks. Code and model will be open-sourced upon publication.

agent-qa by Vostride AI

https://vostride.com/benchmark/android-world

Note: Open-source self-improving QA agent with memory. Code is available at https://github.com/vostride/agent-qa. agent-qa's memory and self-improvement loop were switched off during the benchmark run.

Gemini 3.7 Flash, Gemini Robotics ER 2

Gemini 3 flash, Gemini 3 flash lite

https://drive.google.com/file/d/1NITEkiYb3itSq46I8BYsmzefEddzYbqm/view?usp=sharing

04/2026: Project is open-source at https://github.com/final-run/finalrun-agent
03/2026: Updated results: https://blogs.finalrun.app/finalrun-achieves-the-new-1-sota-mobile-agent-eclipsing-benchmarks-at-97-4
08/2025: The technical doc/approach can be found here: https://github.com/final-run/finalrun-android-world-benchmark

askui AndroidVisionAgent

askui AndroidVisionAgent, Claude 4.5 Sonnet + Claude 4.0 Sonnet

gemini 3 pro + sonnet 4.5

https://autodevice.io/benchmark/

pass@2 95.69, pass@3 97.41

Trajectory submissions: https://midscenejs.com/android-world-benchmark-report

Llama 4-scout, Gemini 2.5 pro, GPT-5 nano

https://minitap.ai/benchmark

[12/16/2025] 84.5% -> 91.4%
[10/1/2025]: 77.6% -> 84.5%
[09/12/2025]: Updated score from 74.1% --> 77.6%
[08/19/2025]: Initial trajectory labeling issue (MarkorCreateNoteFromClipboard) has been corrected by the authors. They provide Discord support for setup issues.
[08/18/2025]: Repo has been reported broken by some users (not independently verified). Some trajectories (e.g., MarkorCreateNoteFromClipboard, BrowserMaze) are labeled as successful by the authors but contain incorrect actions

https://work.aliyun.com/alimail/openLinks/downloadMimeMetaDiskBigAttach?id=netdiskid%3Av001%3Afile%3ADzzzzzzNqZy%3BG8EhE7UV7U9nk2t2O6opysa%2BgRf5Zm6eTqXqtVApm1nDD8W%2BxnLAgdDsJsOSlxfXTx44NU7qRM%2BkJMuIA0smltxGWspXGX2Ox228yqp07rNX%2FIm2QehcGJBgHwUs0f81cz2sX76WNxvxnhpZnqmxiYxHNHRd2hybrPLi2DRQ8bM5ORhI%2BtGG%2B7n%2BcxdY1Ji0

https://hcompai.github.io/android-world-traces/

Agent is based on https://github.com/hcompai/surfer-h-cli - same architecture, but with modifications to prompt and action space

Sonnet 4.5 + Sonnet 4

https://github.com/babelcloud/android_world_benchmark/tree/main/GBOX/trajectory

GBOX uses Claude code as the agent with GBOX mcp. Link to report -> https://github.com/babelcloud/android_world_benchmark/blob/main/GBOX/report.md

[9/26/2025]: Updated score from 75.8 --> 80.2 This is an updated version of AutoGLM-Mobile. In this version, we added more pretraining data and trained the VLM at full image resolution.

Gemini-2.5-Pro+UI-TARS-1.5

Qwen2.5-VL-72B + Qwen2.5-VL-7B

https://github.com/k2-agent/k2-agent/blob/main/androidworld-trajectories.zip

K²-Agent, a hierarchical approach that self-evolves a Qwen2.5-VL-72B for high-level planning and post-trains a Qwen2.5-VL-7B for low-level execution.
Our GitHub repository, which contains a detailed description of our approach, can be found here: https://github.com/k2-agent/k2-agent.

Make further post-training based on the GUI-Owl-32b model. Optimize the memory and knowledge module of the MobileUse framework.

Gemini 2.5 Computer Use

Gemini 2.5 Computer Use

https://github.com/inclusionAI/UI-Venus/blob/main/vis_androidworld/UI-Venus-androidworld.zip

https://github.com/inclusionAI/UI-Venus/blob/main/vis_androidworld/UI-Venus-androidworld.zip

GPT-4o, OS-Atlas-Pro 4B, Qwen2-VL-2B-Instruct

Qwen2-VL-2B (fine-tuned)

A relatively simple prompt involving one LLM /VLM call

A multi-agent architecture involving several LLM calls and a protocol to coordinate the various agents or an LLM wrapped into an advanced agent with memory, subgoal planning, etc.