Stage 4: tested with real people.
The rule that placed itA randomized trial or field study has tested it with real people doing the task.
The evidenceMETR's 2026 randomized follow-up points to AI now speeding developers up, but METR calls the result an unreliable signal. METR, Feb 24, 2026.
A stage says how far the evidence has got. It is not a rating of the AI and not advice.
- 1ClaimedNo evidence yet
- 2LaunchedNo evidence yet
- 3Independently testedNo evidence yet
- 4Tested with real peoplePlaced here
- 5Regulator-clearedNo approval process exists
- 6Widely used with a verified recordNo evidence yet
Everything we found.
In ladder order, newest first within each kind. Makers' own claims are labeled as theirs.
Tested with real people
- Randomized trial, 57 developers, 800+ tasks
METR's 2026 randomized follow-up points to AI now speeding developers up, but METR calls the result an unreliable signal.
“an unreliable signal of the current productivity effect of AI tools.”
METR · Feb 24, 2026
- Randomized trial, 4,867 developers at Microsoft, Accenture and a Fortune 100 firm · run with the maker
In three randomized field experiments run inside companies, developers given GitHub Copilot completed about 26% more tasks; two authors work for Microsoft, which owns GitHub.
“our analysis reveals a 26.08% increase (SE: 10.3%) in completed tasks among developers using the AI tool.”
Cui, Demirer, Jaffe, Musolff, Peng and Salz (SSRN; Management Science) · Aug 21, 2025
- Randomized trial, 16 developers, 246 tasks
In METR's 2025 randomized trial, experienced open-source developers took 19% longer with AI tools, though they believed they were faster.
“Surprisingly, we find that when developers use AI tools, they take 19% longer than without—AI makes them slower.”
METR · Jul 10, 2025
Not found in our sources yet: what the makers claim; what has launched; what independent tests found; how many people use it. No regulator approves AI for this task, so the approval step doesn't apply.
Track record and tools.
Failures and court cases are shown beside the stage, never folded into it. Tools are listed because the evidence names them, in no order of merit.
- GitHub Copilot
