19 GUI agents evaluated on 557 tasks across 17 simulated apps.
Accuracy across overall performance, task length, and numerical-reasoning subtype. ZH/EN are task-weighted accuracies over the Chinese and English app groups; #S is the human-annotated step count. Bold = best, underline = second best per column.
| Agent Model | Overall Benchmark | Task Length | Numerical Reasoning Subtype | |||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Overall | ZH | EN | #S≤5 | #S 6–10 | #S≥11 | Cnt. | Arith. | Comp. | Thres. | |
| Claude-Opus-4.7 | 50.27% | 60.07% | 40.15% | 58.87% | 54.29% | 28.15% | 52.31% | 55.56% | 52.00% | 42.86% |
| Gemini-3-Pro | 49.91% | 60.07% | 39.42% | 56.38% | 52.86% | 33.33% | 46.15% | 42.22% | 49.33% | 23.81% |
| GPT-5.5 | 49.73% | 58.66% | 40.51% | 59.22% | 52.14% | 27.41% | 44.62% | 57.78% | 52.00% | 33.33% |
| Gemini-3.1-Pro | 49.55% | 59.36% | 39.42% | 56.74% | 52.86% | 31.11% | 43.08% | 42.22% | 50.67% | 23.81% |
| Doubao-Seed-1.6 | 39.14% | 50.88% | 27.01% | 45.74% | 43.57% | 20.74% | 23.08% | 22.22% | 36.00% | 9.52% |
| GPT-5 | 33.39% | 37.81% | 28.83% | 43.62% | 31.43% | 14.07% | 27.69% | 35.56% | 40.00% | 4.76% |
| Doubao-Seed-1.8 | 32.14% | 34.63% | 29.56% | 43.26% | 27.86% | 13.33% | 23.08% | 31.11% | 28.00% | 4.76% |
| Qwen-3.6-Plus | 31.42% | 39.58% | 22.99% | 39.01% | 31.43% | 15.56% | 23.08% | 13.33% | 28.00% | 9.52% |
| Doubao-Seed-2.0 | 29.26% | 36.40% | 21.90% | 37.94% | 28.57% | 11.85% | 27.69% | 28.89% | 20.00% | 14.29% |
| GPT-5.4 | 29.08% | 34.63% | 23.36% | 38.30% | 25.71% | 13.33% | 24.62% | 24.44% | 30.67% | 9.52% |
| Qwen-3.6-Flash | 28.19% | 38.87% | 17.15% | 38.30% | 25.71% | 9.63% | 20.00% | 24.44% | 26.67% | 4.76% |
| Claude-Sonnet-4.6 | 27.29% | 31.10% | 23.36% | 36.17% | 27.86% | 8.15% | 29.23% | 33.33% | 26.67% | 14.29% |
| Claude-Haiku-4.5 | 22.80% | 22.61% | 22.99% | 34.04% | 15.71% | 6.67% | 20.00% | 22.22% | 20.00% | 9.52% |
| Claude-Sonnet-4.5 | 21.01% | 24.03% | 17.88% | 31.91% | 14.29% | 5.19% | 23.08% | 26.67% | 21.33% | 14.29% |
| Gemini-2.5-Pro | 17.24% | 21.20% | 13.14% | 26.24% | 14.29% | 1.48% | 23.08% | 17.78% | 17.33% | 9.52% |
| MobileAgent-v3.5 | 31.42% | 35.34% | 27.37% | 41.49% | 26.43% ↓15.06 | 15.56% ↓10.87 | 18.46% | 24.44% | 32.00% | 4.76% |
| UI-TARS-1.5-7B | 22.62% | 31.10% | 13.87% | 30.14% | 22.86% ↓7.28 | 6.67% ↓16.19 | 20.00% | 8.89% | 16.00% | 4.76% |
| AgentCPM-GUI | 10.05% | 9.19% | 10.95% | 17.02% | 5.71% ↓11.31 | 0.00% ↓5.71 | 3.08% | 2.22% | 5.33% | 4.76% |
| V-Droid | 9.87% | 13.43% | 6.20% | 15.25% | 7.86% ↓7.39 | 0.74% ↓7.12 | 15.38% | 11.11% | 9.33% | 0.00% |
Cnt./Arith./Comp./Thres. = counting, arithmetic, numeric comparison, threshold filtering. Subtypes are not mutually exclusive.
AvgAct = average valid actions per task. ActOH = action overhead on successful tasks (valid actions / human reference steps). Budg. Exh. = trajectories exhausting the 50-step budget. Budg. ACC = accuracy among those. Rep.≥3 / Rep.≥5 = budget-exhausted runs with ≥3 / ≥5 consecutive repeated actions.
| Agent Model | ACC ↑ | AvgAct ↓ | ActOH ↓ | Budg. Exh. ↓ | Budg. ACC ↑ | Rep.≥3 ↓ | Rep.≥5 ↓ |
|---|---|---|---|---|---|---|---|
| Claude-Opus-4.7 | 50.27% | 14.01 | 1.98 | 33 | 0.00% | 27 (81.82%) | 20 (60.61%) |
| Gemini-3-Pro | 49.91% | 10.79 | 1.69 | 14 | 0.00% | 5 (35.71%) | 3 (21.43%) |
| GPT-5.5 | 49.73% | 15.04 | 2.09 | 45 | 0.00% | 27 (60.00%) | 15 (33.33%) |
| Gemini-3.1-Pro | 49.55% | 10.69 | 1.80 | 19 | 10.53% | 7 (36.84%) | 3 (15.79%) |
| Doubao-Seed-1.6 | 39.14% | 15.59 | 1.73 | 85 | 1.18% | 74 (87.06%) | 52 (61.18%) |
| GPT-5 | 33.39% | 23.15 | 2.94 | 136 | 0.00% | 105 (77.21%) | 61 (44.85%) |
| Doubao-Seed-1.8 | 32.14% | 9.71 | 1.60 | 44 | 0.00% | 36 (81.82%) | 27 (61.36%) |
| Qwen-3.6-Plus | 31.42% | 12.66 | 1.62 | 23 | 0.00% | 11 (47.83%) | 9 (39.13%) |
| Doubao-Seed-2.0 | 29.26% | 10.76 | 1.81 | 46 | 0.00% | 33 (71.74%) | 28 (60.87%) |
| GPT-5.4 | 29.08% | 10.88 | 1.56 | 16 | 0.00% | 12 (75.00%) | 7 (43.75%) |
| Qwen-3.6-Flash | 28.19% | 13.30 | 2.10 | 38 | 0.00% | 27 (71.05%) | 23 (60.53%) |
| Claude-Sonnet-4.6 | 27.29% | 29.80 | 3.11 | 253 | 0.40% | 222 (87.75%) | 171 (67.59%) |
| Claude-Haiku-4.5 | 22.80% | 31.63 | 4.07 | 221 | 0.00% | 208 (94.12%) | 162 (73.30%) |
| Claude-Sonnet-4.5 | 21.01% | 23.31 | 3.30 | 93 | 3.23% | 81 (87.10%) | 55 (59.14%) |
| Gemini-2.5-Pro | 17.24% | 22.10 | 2.38 | 106 | 0.00% | 74 (69.81%) | 40 (37.74%) |
| MobileAgent-v3.5 | 31.42% | 16.79 | 2.02 | 83 | 1.20% | 69 (83.13%) | 40 (48.19%) |
| UI-TARS-1.5-7B | 22.62% | 24.76 | 1.67 | 226 | 0.00% | 156 (69.03%) | 138 (61.06%) |
| AgentCPM-GUI | 10.05% | 24.82 | 1.71 | 242 | 0.00% | 203 (83.88%) | 198 (81.82%) |
| V-Droid | 9.87% | 33.30 | 4.60 | 313 | 0.32% | 313 (100.00%) | 295 (94.25%) |
| Summary | 30.76% | 18.58 | 2.15 | 2036 | 0.44% | 1690 (83.01%) | 1347 (66.16%) |
Each cell shows one model's accuracy on one app. Deeper green = higher accuracy, deeper red = lower accuracy. Click any cell to see the per-task correctness grid for that (model, app) pair.
Each cell is one (model, task) pair. correct wrong. Rows are models (ordered by overall accuracy), grouped by app. Hover a cell for the task instruction.