TaxProf Blog

Editor: Paul L. Caron, Dean
Pepperdine University School of Law

Monday, February 26, 2018

Artificial Intelligence Software Is More Accurate, Faster Than Expert Lawyers In Reviewing Contracts

AILMashable, An AI Just Beat Top Lawyers at Their Own Game:

The nation's top lawyers recently battled artificial intelligence in a competition to interpret contracts — and they lost.

A new study, conducted by legal AI platform LawGeex in consultation with law professors from Stanford University, Duke University School of Law, and University of Southern California, pitted twenty experienced lawyers against an AI trained to evaluate legal contracts. Competitors were given four hours to review five non-disclosure agreements (NDAs) and identify 30 legal issues, including arbitration, confidentiality of relationship, and indemnification. They were scored by how accurately they identified each issue.

Unfortunately for humanity, we lost the competition — badly.

The human lawyers achieved, on average, an 85 percent accuracy rate, while the AI achieved 95 percent accuracy. The AI also completed the task in 26 minutes, while the human lawyers took 92 minutes on average. The AI also achieved 100 percent accuracy in one contract, on which the highest-scoring human lawyer scored only 97 percent. In short, the human lawyers were trounced.

Intellectual property attorney Grant Gulovsen, one of the lawyers who competed against the AI in the study, said the task was very similar to what many lawyers do every day. "The majority of documents, whether it's wills, operating agreements for corporations, or things like NDAs...they're very similar," Gulovsen told Mashable in a phone interview.

So does this spell the end of humanity? Not at all. On the contrary, the use of AI can actually help lawyers expedite their work, and free them up to focus on tasks that still require a human brain.

https://taxprof.typepad.com/taxprof_blog/2018/02/arificial-intelligance-software-is-more-accurate-faster-than-expert-lawyers-in-reviewing-contracts.html

Legal Education | Permalink

Comments

The linked article says 26 seconds, not 26 minutes for the AI, which sounds more correct actually.

Posted by: notsure | Feb 26, 2018 12:46:33 PM

"So does this spell the end of humanity? Not at all. On the contrary, the use of AI can actually help lawyers expedite their work"

Translation: even less need for new associates.

Posted by: Unemployed Northeastern | Feb 27, 2018 8:47:08 AM