Intelligence Verdict

The answer

Stage 3: independently tested.

The rule that placed itSomeone other than the maker has tested it: an independent benchmark, a peer-reviewed paper, or a hands-on test by a trusted outlet.

The evidenceStanford researchers found leading legal AI tools gave incorrect information more than 17% of the time, and one more than 34%. Stanford HAI and RegLab, May 23, 2024.

A stage says how far the evidence has got. It is not a rating of the AI and not advice.

1
ClaimedNo evidence yet
2
LaunchedNo evidence yet
3
Independently testedPlaced here
4
Tested with real peopleNo evidence yet
5
Regulator-clearedNo approval process exists
6
Widely used with a verified recordNo evidence yet

The evidence

Everything we found.

In ladder order, newest first within each kind. Makers' own claims are labeled as theirs.

Tested

What independent tests found

Stanford RegLab and HAI
Stanford researchers found leading legal AI tools gave incorrect information more than 17% of the time, and one more than 34%.
“the Lexis+ AI and Ask Practical Law AI systems produced incorrect information more than 17% of the time,”
Stanford HAI and RegLab · May 23, 2024

Not found in our sources yet: what the makers claim; what has launched; tested with real people; how many people use it. No regulator approves AI for this task, so the approval step doesn't apply.

Beside the stage