Does ChatGPT Produce Fishy Briefs?

ABA Journal, Does ChatGPT Produce Fishy Briefs?:

Lawyers are abuzz about the possible uses of ChatGPT. Could the artificial intelligence-powered chatbot write a persuasive legal brief worthy of judicial consideration? Given its limitations, we believe that’s unlikely. ChatGPT, a large language model developed by the San Francisco company OpenAI that launched in November, can draw only on sources available on the web; it cannot crawl appellate records or access subscription-based services such as Westlaw. Still, the ABA Journal decided to put the technology to the test just for kicks. ...

We knew the language generated by the technology would likely not measure up to noted legal lexicographer and longtime ABA Journal columnist Bryan A. Garner’s standards, but we gave it a try, asking ChatGPT for arguments for and against the petition to review the case. Are these briefs fishy? Is one better than the other? You be the judge.

Prior TaxProf Blog coverage:

• A Human Being Wrote This Law Review Article: GPT-3 And The Practice Of Law (May 11, 2022)
• The Implications Of OpenAI’s Assistant For Legal Services And Society (Dec. 7, 2022)
• ChatGPT And Law School Exams (Dec. 29, 2022)
• GPT Will Soon Be Able To Pass The Multistate Bar Exam (Jan. 5, 2023)
• Using ChatGPT To Write Law School Exams, Bar Exams, And Strategic Plans (Jan. 11, 2023)
• ChatGPT Gets B|B- Grade On Wharton MBA Exam (Jan. 24, 2023)
• ChatGPT Gets C+ Grade On Four Minnesota Law School Exams (C- In Tax) (Jan. 24, 2023)
• The Rise Of The Robotic Tax Analyst (Jan. 27, 2023)
• Ryznar: Exams In The Time Of ChatGPT (Feb. 1, 2023)
• Bishop Posts Two Papers On ChatGPT (Feb. 8, 2023)
• ChatGPT Almost Passed The Bar, But Competent Lawyers Do Much More (Feb. 23, 2023)
• It’s Not Just Our Students: ChatGPT Is Coming For Faculty Scholarship (Feb. 25, 2023)
• New AI Detector Is 97% Effective In Catching Students Cheating With ChatGPT (Feb. 28, 2023)
• It’s Not Just Our Students: ChatGPT Is Coming For Faculty Scholarship (Feb. 25, 2023)
• ChatGPT's Tax Advice Was Wrong 100% Of The Time (Mar. 7, 2023)
• Does ChatGPT Produce Fishy Briefs? (Mar. 8, 2023)
• Colleges (And Law Schools) Are Rushing To Respond To ChatGPT (Mar. 9, 2023)
• Was The Sermon You Heard At Church Today Written By ChatGPT? (Mar. 12, 2023)
• GPT-4 Beats 90% Of Aspiring Lawyers On The Bar Exam (Mar. 17, 2023)
• ChatGPT Thinks I Am Way More Interesting Than I Am (Mar. 22, 2023)
• Should ChatGPT Be In Law School? (Mar. 30, 2023)
• Merritt: GPT-4 On Legal Education And Lawyer Licensing (Apr. 4, 2023)
• Turnitin Plagiarism Detector Will Catch Students Who Cheat With ChatGPT With 98% Accuracy (Apr. 5, 2023)
• ChatGPT Gets 148 (37th Percentile) And 157 (70th Percentile) On The LSAT (Apr. 6, 2023)
• AI Tools for Lawyers: A Practical Guide (Apr. 6, 2023)