Which AI makes the fewest mistakes on everyday work?
As of 2026-10-17, Gemini 3.8 Flash made the fewest mistakes on our 10 everyday tasks, passing 35 of 35 checks. Qwen 3.8 27B passed 30 of 30 and gave no answer on 1 task. On the 5 trap questions, models slipped in 2 of 20 answers. The most common mistake was “invented” (2 times).
Which AI has the lowest error rate on everyday tasks?
Sorted by checks passed. A model that gave no answer on a task ranks below models that answered everything, since unanswered tasks aren't scored. Ties go to the cheaper model. The four mistake columns count failed checks by type.
| Model | Checks passed | Invented | Changed | Dropped | Broke a rule | No answer | Cost per 100 runs (USD) | Avg response |
|---|---|---|---|---|---|---|---|---|
| Gemini 3.8 Flash | 35 / 35 | 0 | 0 | 0 | 0 | 0 | $4.70 | 40 s |
| OpenAI gpt-oss-120b | 34 / 35 | 1 | 0 | 0 | 0 | 0 | $0.50 | 6 s |
| Gemini 3.5 Flash-Lite | 34 / 35 | 1 | 0 | 0 | 0 | 0 | $0.50 | 2 s |
| Qwen 3.8 27B | 30 / 30 | 0 | 0 | 0 | 0 | 1 | $2.18 | 23 s |
Which AI made things up on the trap questions?
Each trap asks for something the source does not contain, or is built on a false premise. A correct answer says so.
| Model | Fact not in the source | False premise | Quote that doesn't exist | Promise not in the policy | Total from a table |
|---|---|---|---|---|---|
| Gemini 3.8 Flash | ✓ | ✓ | ✓ | ✓ | ✓ |
| OpenAI gpt-oss-120b | ✓ | ✓ | ✓ | ✗ No made-up warranty length | ✓ |
| Gemini 3.5 Flash-Lite | ✓ | ✓ | ✓ | ✗ No made-up warranty length | ✓ |
| Qwen 3.8 27B | ✓ | ✓ | ✓ | ✓ | ✓ |
Which AI dropped or changed details in normal work?
Meeting notes, a customer reply, an Excel formula, a report summary and a slide outline.
| Model | Meeting notes | Customer reply | Excel formula | Report summary | Slide outline |
|---|---|---|---|---|---|
| Gemini 3.8 Flash | ✓ | ✓ | ✓ | ✓ | ✓ |
| OpenAI gpt-oss-120b | ✓ | ✓ | ✓ | ✓ | ✓ |
| Gemini 3.5 Flash-Lite | ✓ | ✓ | ✓ | ✓ | ✓ |
| Qwen 3.8 27B | No answer | ✓ | ✓ | ✓ | ✓ |
Is ChatGPT accurate for work tasks?
This test does not cover the ChatGPT app or paid ChatGPT models yet. The OpenAI model here is gpt-oss-120b, OpenAI's open-weight model, tested through Cloudflare's API. Its results tell you how an OpenAI model handles these tasks, not how the ChatGPT app will. We plan to add paid flagship models (ChatGPT, Claude, Gemini Pro) once the test has a stable audience.
How do I check AI output for wrong numbers?
Most mistakes in this test are numbers that were dropped or changed. Paste the original text and the AI's answer into Number Check: it marks every number in the AI text that isn't in the original. It runs in your browser and uploads nothing.
How the test works, and its limits
Every Tuesday the same 10 English tasks are sent to each model once: 5 everyday work tasks and 5 trap questions. A program checks each answer against fixed rules, such as whether a date from the source appears, whether a number was changed, or whether the answer invents a fact the source doesn't contain. No AI grades another AI, so the result can be reproduced. The scenarios, companies and people are invented for the test.
Mistakes are sorted into four types: invented (a fact or quote not in the source), changed (a wrong number or wrong logic), dropped (a required detail left out) and broke a rule (ignored an instruction such as a word limit).
Limits: one run per model per task, so a single result can be luck. Rule-based checks can miss a correct answer phrased in an unusual way. Models are tested through their APIs, which can behave differently from the consumer apps.
- Gemini 3.8 Flash is a Google Gemini model.
- OpenAI gpt-oss-120b is OpenAI's open-weight model (not the model inside the ChatGPT app).
- Gemini 3.5 Flash-Lite is a Google Gemini model.
- Qwen 3.8 27B is Alibaba's open model.
No answer means the model returned no text, for example because its reasoning used up the 4,000-token output limit. Those tasks are left out of the score and shown separately.
Cost per 100 runs is the price of running all 10 tasks 100 times, using each provider's published price for the tokens actually used:
- gemini-3.8-flash: US$0.75 per million input tokens, US$3.75 per million output tokens (ai.google.dev/gemini-api/docs/pricing (2027-01-01起調為1.50/7.50))
- gemini-3.5-flash-lite: US$0.3 per million input tokens, US$2.5 per million output tokens (ai.google.dev/gemini-api/docs/pricing)
- gpt-oss-120b: US$0.35 per million input tokens, US$0.75 per million output tokens (developers.cloudflare.com/workers-ai/platform/pricing)
- qwen3.8-27b: US$0.45 per million input tokens, US$3.2 per million output tokens (developers.cloudflare.com/workers-ai/platform/pricing)
Download the raw results
Every check from the 2026-10-17 run, one row per check: bench-2026-10-17.csv. Free to use with a link back to this page.