Asapi
The AI Agent Trials / The AIs
The AIs

The AIs we tested

We tested 28 AIs from eight makers. They include OpenAI's GPT models, the family behind ChatGPT, Anthropic's Claude models, such as Opus, Sonnet, Haiku and Fable, and one of Google's, Gemini 4 Argon. The other five makers offer AIs anyone can download. Most of OpenAI's ran in Codex, most of Anthropic's in Claude Code and Google's in Antigravity, the makers' own apps for running an AI as an agent on a computer. We haven't tested the ChatGPT or Claude chat apps, or Microsoft's AIs, and we didn't test every AI on every job.

The strongest

The three strongest were close

On the 62 jobs we gave all five of the most‑tested AIs, GPT‑6 Astra, Opus 5.5 and GPT‑6.1 Sol came out close, ahead of Sonnet 5.5 and GPT‑6 Luna. Even they got everything right on only 5 to 7 of those jobs.

AISizeEverything rightAverage score
GPT‑6 AstraOpenAILarge5 of 6274
Opus 5.5AnthropicLarge7 of 6272
GPT‑6.1 SolOpenAIMid‑sized5 of 6270
Sonnet 5.5AnthropicMid‑sized3 of 6265
GPT‑6 LunaOpenAISmall0 of 6227

Everything right: the jobs where the AI met every check, every time we counted. Average score: its average out of 100 across the jobs these five shared.

By benchmark

Who did best in each benchmark

Your AI personal assistant
GPT‑6 Astra had the best result, or shared it, on four of the ten jobs, and with GPT‑6.1 Sol got everything right on next term's classes.
Your AI office assistant
GPT‑6 Astra did best on the office jobs, with the top score on five of the eleven, one of them shared with Fable 5.1.
Your AI app builder
Opus 5.5 had the best result on seven of the 13 apps, and was the only AI to get everything right on one, the train‑ticket catcher.
Your AI customer service agent
GPT‑6 Astra, Opus 5.5 and GPT‑6.1 Sol each got all 80 repair customers right, Opus 5.5 had the best result, or tied for it, on three of the seven jobs, and Sonnet 5.5 got every rental search right.
Your AI operations manager
Opus 5.5 got everything right on four of the nine jobs, and GPT‑6.1 Sol on three.
Your AI hospital assistant
Opus 5.5 got everything right on all three jobs, and was the only AI to do so on the ward.
Your AI facilities engineer
Sonnet 5.5 did best on the cooling failure and Fable 5.1 on the server hall, and GPT‑6 Astra and Opus 5.5 shared the top score on moving the air handler.
Your AI engineering assistant
Sonnet 5.5 got everything right reading the plant's scans, and Fable 5.1 bringing its drawings up to date; GPT‑6 Astra did best on the watchdog and shared the top score on the sketches.
Your AI DIY assistant
Opus 5.5 had the best average and the top score on three of the six jobs, one shared with Fable 5.2; GPT‑6 Astra did best on the furnished room and the wet floor, and shared the top score on the lighting plan with Fable 5.2.
Your AI robot operator
GPT‑6.1 Sol, the only AI tested on the ten builds, completed none under their strict check; at batting it hit two pitches, more than any other AI.
Your AI race team
GPT‑6.1 Sol, GPT‑6.1 Terra and GPT‑6 Astra finished both races; designing a race car, GPT‑6 Astra came closest to the fastest achievable car, 2.0% slower.
Size

How much size matters

We sort most of the AIs into large, mid‑sized and small, as a rough guide. Bigger usually does better, but not always. On the 69 jobs where we tested AIs of all three sizes, a large AI got everything right on 14, a mid‑sized one on 11 and a small one on 2. GPT‑6.1 Sol, a mid‑sized AI, kept up with the two strongest large ones.

Open AIs

AIs anyone can download

Five makers offer AIs anyone can download and run. DeepSeek V4.1 Flash came closest to the strongest, and had the top score, or shared it, on 2 of the 13 jobs we tested it on: it got everything right on the radiation‑dose records program, and made the best insurance claim for a broken freezer. On the hospital ward it scored 96. Qwen3.8‑27B also got everything right on the radiation‑dose records program.

All of them

Every AI we tested

AISizeJobs testedEverything right
Opus 5.5AnthropicLarge679
GPT‑6.1 SolOpenAIMid‑sized788
GPT‑6 AstraOpenAILarge697
Sonnet 5.5AnthropicMid‑sized653
Fable 5.1AnthropicLarge503
GPT‑6.1 TerraOpenAI–52
GPT‑6 LunaOpenAISmall691
Opus 5AnthropicLarge351
DeepSeek V4.1 Flash openDeepSeekSmall131
Qwen3.8‑27B openAlibabaSmall101
GPT‑5.6 SolOpenAILarge31
Sonnet 5AnthropicMid‑sized350
GPT‑6 SolOpenAIMid‑sized190
GLM‑5.3‑Flash openZhipuSmall110
Kimi K3 openMoonshotMid‑sized90
Fable 5.2Anthropic–80
GPT‑5.6 LunaOpenAISmall70
Haiku 4.5AnthropicMid‑sized20
Haiku 5.5Anthropic–20
Opus 4.8AnthropicLarge20
GPT‑5.6 TerraOpenAI–20

Everything right: the jobs where the AI met every check, every time we counted. We tested seven more AIs on one job each: DeepSeek V3.2 (DeepSeek), GLM‑5 (Zhipu), GLM‑5.3 (Zhipu), Gemini 4 Argon (Google), Kimi K2.5 (Moonshot), Nemotron Super 3 (NVIDIA), Qwen3‑235B (Alibaba).