Benjamin Alarie (Toronto; Google Scholar; CEO, Blue J Legal), The Rise of the Robotic Tax Analyst, 178 Tax Notes Fed. 57 (Jan. 2, 2023):
As a bold taxwriting experiment, this installment of Blue J Predicts has been generated with the help of an AI assistant, OpenAI’s “Generative Pre-Trained Transformer 3” (GPT-3). GPT-3 is a large language model developed by OpenAI and backed by Microsoft. It is an inexhaustible generator of text and can write with accuracy in English about almost any topic. It has performed its duty, with my human companionship, indefatigably.
This isn’t the first time that a legal academic has invoked a robotic coauthor, and I expect that these kinds of tools will become increasingly commonplace. I expect that eventually they will be about as remarkable as using a spelling or grammar checker. At this moment, however, before the rise of the robotic tax analyst, using GPT-3 to help write this article is likely to raise some eyebrows.
In October 2021 I produced a peer-reviewed law review article with my academic colleague, the late (and great) tax law professor Arthur Cockfield of Queen’s University. The article was notable for being the first peer-reviewed law review article to extensively leverage GPT-3 in its production. In that article — after a short introduction penned by us, Benjamin Alarie and Cockfield — we gave GPT-3 control of the metaphorical keyboard and allowed it to produce its textual analysis uninterrupted and unedited.
The results were mixed and intriguing, and certainly pointed in the direction of future possibility. In our view, GPT-3 had potential. In the article, we speculated on the future of AI in legal scholarship and asked provocatively in the title, “Will Machines Replace Us?” Cockfield and I concluded that “although GPT-3 is not up to the task of replacing law review authors currently, we are far less confident that GPT-5 or GPT-100 might not be up to the task in future.”
Although these words were published just over a year ago, technological developments in 2022 suggest that we were probably somewhat conservative in our gauging of the likely rate of improvement of AI in the form of large language models. The rapid progress in the power of these models has been thoroughly impressive, with more progress to come. To help showcase the power of GPT-3 to readers of Tax Notes, I believe that I am the first author in these pages to rely on GPT-3’s help with drafting a contribution.
As 2023 begins, developments with large language models, most prominently ChatGPT, a recent offshoot of GPT-3, are generating buzz in legal technology circles. They are also triggering wariness among some in legal academia and practice. On November 30, 2022, OpenAI made ChatGPT available for public use through an open beta. In less than one week, the system was deployed by more than 1 million registered users. The extremely rapid adoption was driven by claims — and plenty of evidence — of impressive algorithmic feats performed by ChatGPT. The capabilities of ChatGPT reportedly include achieving passing scores on practice bar and medical board examinations.
Within days of the release of ChatGPT, Andrew Perlman, the dean of law and a professor at Suffolk University, prompted the program to write an essay on its own likely influence in the future of law [The Implications Of OpenAI’s Assistant For Legal Services And Society]. The abstract to the essay, which Perlman assures us he wrote himself, concludes with the observation that “the disruptions from AI’s rapid development are no longer in the distant future. They have arrived, and this document offers a small taste of what lies ahead.”
It’s not only GPT-3 and its progeny that have been making waves. Other algorithms garnered media attention in 2022 for exhibiting surprisingly strong image-generating capabilities. These latent text-to-image diffusion models, which include DALL-E 2, Stable Diffusion, and Midjourney, have been widely used, criticized, deployed, and celebrated.
The New York Times reported in September 2022 that an image generated by a text-to-image diffusion model won a digital art competition at the Colorado State Fair, causing a furor among artists, fellow competitors, and social media users. It is fair to say that machine learning and AI are beginning to make their influence felt in some unexpected places. There is little reason to expect these and related developments to abate at the doorstep of tax analysis.
To wrap up Blue J Predicts for 2022, we will do some Dickensian stocktaking, visiting Blue J predictions past, present, and future. The initial order of business is to retrospectively analyze how the Blue J Predicts analyses have fared in 2022. Have our predictions mapped on to reality? Have the cases gone the way the machine-learning models and the algorithms have anticipated? Broadly, the answer is yes, although with the important caveat that we do not yet have the final word from the courts in many of the cases.
We then consider the state of AI and machine learning in tax research and analysis. We outline some new ways in which machine-learning technology is poised to affect tax research and analysis this year, at Blue J and beyond. Finally, we look ahead to anticipate how AI could affect tax analysis and research, and explain why it is important to begin turning our attention to the possibilities.
Prior TaxProf Blog coverage of Chat GPT:
- A Human Being Wrote This Law Review Article: GPT-3 And The Practice Of Law (May 11, 2022)
- The Implications Of OpenAI’s Assistant For Legal Services And Society (Dec. 7, 2022)
- ChatGPT And Law School Exams (Dec. 29, 2022)
- GPT Will Soon Be Able To Pass The Multistate Bar Exam (Jan. 5, 2022)
- Using ChatGPT To Write Law School Exams, Bar Exams, And Strategic Plans (Jan. 11, 2023)
- ChatGPT Gets B|B- Grade On Wharton MBA Exam (Jan. 24, 2023)
- ChatGPT Gets C+ Grade On Four Minnesota Law School Exams (C- In Tax) (Jan. 24, 2023)
- The Rise Of The Robotic Tax Analyst (Jan. 27, 2023)
- Ryznar: Exams In The Time Of ChatGPT (Feb. 1, 2023)
- Bishop Posts Two Papers On ChatGPT (Feb. 8, 2023)
- ChatGPT Almost Passed The Bar, But Competent Lawyers Do Much More (Feb. 23, 2023)
- It’s Not Just Our Students: ChatGPT Is Coming For Faculty Scholarship (Feb. 25, 2023)
- New AI Detector Is 97% Effective In Catching Students Cheating With ChatGPT (Feb. 28, 2023)
- It’s Not Just Our Students: ChatGPT Is Coming For Faculty Scholarship (Feb. 25, 2023)
- ChatGPT's Tax Advice Was Wrong 100% Of The Time (Mar. 7, 2023)
- Does ChatGPT Produce Fishy Briefs? (Mar. 8, 2023)
- Colleges (And Law Schools) Are Rushing To Respond To ChatGPT (Mar. 9, 2023)
- Was The Sermon You Heard At Church Today Written By ChatGPT? (Mar. 12, 2023)
- GPT-4 Beats 90% Of Aspiring Lawyers On The Bar Exam (Mar. 17, 2023)
- ChatGPT Thinks I Am Way More Interesting Than I Am (Mar. 22, 2023)
- Should ChatGPT Be In Law School? (Mar. 30, 2023)
- Merritt: GPT-4 On Legal Education And Lawyer Licensing (Apr. 4, 2023)
- Turnitin Plagiarism Detector Will Catch Students Who Cheat With ChatGPT With 98% Accuracy (Apr. 5, 2023)
- ChatGPT Gets 148 (37th Percentile) And 157 (70th Percentile) On The LSAT (Apr. 6, 2023)
- AI Tools for Lawyers: A Practical Guide (Apr. 6, 2023)
Prior TaxProf Blog coverage Blue J Predicts:
- Blue J Predicts: An Unprofitable Pretax Venture Can Still Be A Partnership (June 29, 2021)
- Blue J Predicts With 68% Confidence That IRS's Economic Substance Challenge Will Fail In District Court Perrigo Case (Aug. 6, 2021)
- Blue J Predicts With 77% Confidence That Reserve’s § 501(c)(15) Appeal Will Be Dismissed By The Tenth Circuit (Sept. 10, 2021)
- Blue J Predicted With 95% Confidence That 7th Circuit Would Affirm Tax Court In Innocent Spouse Cases (Oct. 5, 2021)
- Blue J Predicts With 74% Confidence That 8th Circuit Will Find Customary/Usual Management Fees Are Deductible Under § 162 (Nov. 3, 2021)
- Blue J Predicts: Taxpayers Face Uphill Battle In Assignment Of Income Cases (Dec. 8, 2021)
- Blue J Predicts: Worker Classification In The Gig Economy (Dec. 20, 2021)
- Blue J: Using Machine Learning To Crack The Tax Code (Feb. 8, 2022)
- Blue J Predicts With 86%-95% Confidence That 10th Circuit Will Find That Taxpayer In Olsen Was Not Engaged In A Trade Or Business (Feb. 28, 2022)
- Blue J Predicts With 65%-94% Confidence Federal Circuit's Step Transaction Decision In GSS Holdings (Apr. 5, 2022)
- Blue J Predicts With 86% Confidence Debt-Equity Decision In Tribune Media (May 4, 2022)
- Blue J's AI Prediction Of Result Of Tax Court Appeal Faces First Test In Federal Circuit Court (June 14, 2022)
- Blue J: Reserve Mechanical Microcaptive Insurance Arrangement Denied On Appeal (July 7, 2022)
- Blue J: Accurate Financial Recordkeeping And Business Deductions (Aug. 12, 2022)
- Blue J: Deducting Legal Expenses In Mylan (Sept. 7, 2022)
- Blue J Predicts: Tax Credits That Bond A Partnership (Oct. 20, 2022)
- Blue J Predicts: The Economic Substance Doctrine And Excise Tax Credits Related To Alcohol-Fuel Mixtures (Dec. 2, 2022)
- Blue J Predicts: Conflicting Duties And The Trust Fund Recovery Penalty (Jan. 17, 2023)