Blue J: The Rise Of Generative AI For Tax Research

Benjamin Alarie (Osler Chair in Business Law, University of Toronto; CEO, Blue J Legal) et al., The Rise of Generative AI for Tax Research, 179 Tax Notes 1609 (May 29, 2023):

ChatGPT has exploded into the popular consciousness. Generative artificial intelligence, in the form of natural language chatbots, is newly poised to significantly shift how tax professionals conduct research. These emerging advanced AI systems can analyze vast amounts of text and generate coherent, contextually relevant responses.

As various industries continue to adopt AI-powered tools, lawyers need to understand and adapt to these new technologies. As Jason Chen has written in Tax Notes, “It is only by incorporating these types of technological advances into daily business practices that modern tax professionals can gain valuable advantages over others in the increasingly competitive world of taxation.”

In this article, we will consider the benefits of using generative AI, examine the concerns, and discuss ways that it can be improved.

To illustrate these considerations, we submitted a series of tax law questions to three different generative AI models: ChatGPT (based on GPT-3.5), ChatGPT Plus (based on GPT-4), and Ask Blue J. The questions were selected to represent a variety of tax law areas and to present the types of research questions that a typical tax law practitioner might encounter in their day-to-day practice.

For this experiment we posed identical questions to ChatGPT and Ask Blue J and recorded their responses. We tested the questions against the GPT-3.5 and GPT-4 versions of ChatGPT. We primed each conversation with ChatGPT using the following instruction: “I want you to answer all questions as they apply in the context of U.S. federal income tax law. Please provide IRS resources along with your answers.” This additional instruction was provided to ChatGPT only because Ask Blue J is already fine-tuned to focus on tax and to provide sources with its responses. Providing the priming instruction to the GPT models ensures a fair comparison. Other than this introductory prompt, our questions to each chatbot were the same. The results highlight the importance of developers and users engaging with this technology thoughtfully when harnessing it for tax research purposes.

Blue J Tax Notes Federal articles:

• An Unprofitable Pretax Venture Can Still Be a Partnership, 171 Tax Notes Fed. 1951 (June 17, 2021)
• Economic Substance Doctrine: Still Giving Perrigo Heartburn?, 172 Tax Notes Fed. 599 (July 26, 2021)
• Captive Insurance Appeal in Reserve Mechanical Will Likely Fail, 172 Tax Notes Fed. 1431 (Aug. 30, 2021)
• Seventh Circuit Affirms Spouse Is Not So Innocent on Appeal, 172 Tax Notes Fed. 2149 (Sept. 27, 2021)
• Would Management Fees by Any Other Name Still Be Deductible?, 173 Tax Notes Fed. 499 (Oct. 25, 2021)
• Battling Uphill Against the Assignment of Income Doctrine: Ryder, 173 Tax Notes Fed. 1253 (Nov. 29, 2021)
• Predicting Worker Classification in the Gig Economy, 173 Tax Notes Fed. 1733 (Dec. 20, 2021)
• Using Machine Learning To Crack The Tax Code, 174 Tax Notes Fed. 661 (Jan. 31, 2022)
• Using Machine Learning to Evaluate the Existence of a Trade or Business: Olsen, 174 Tax Notes Fed. 1231 (Feb. 28, 2021)
• Timing Is Everything: The Step Transaction Doctrine in GSS Holdings, 174 Tax Notes Fed. 1849 (Mar. 28, 2021)
• The Debt-Equity Distinction and Tribune Media, 175 Tax Notes Fed. 593 (Apr. 25, 2022)
• Disguised Distributions and Management Fees: Aspro Revisited, 175 Tax Notes Fed. 1401 (May 30, 2022)
• Reserve Mechanical Microcaptive Insurance Arrangement Denied On Appeal, 175 Fed Tax Notes 2037 (June 27, 2022)
• Situational Awareness: Accurate Financial Recordkeeping and Business Deductions, 176 Tax Notes Fed. 713 (Aug. 1, 2022)
• Deducting Legal Expenses: Unpacking the IRS's Appeal In Mylan, 176 Tax Notes Fed. 1419 (Aug. 29, 2022)
• Tax Credits That Bond a Partnership: Revisiting Cross Refined Coal, 176 Tax Notes Fed. 2069 (Sept. 26, 2022)
• Chemoil: Economic Substance, Tax Credits, and Unprofitable Ventures, 177 Tax Notes Fed. 719 (Oct. 31, 2022)
• Cashaw: Conflicting Duties And The Trust Fund Recovery Penalty, 177 Tax Notes Fed. 1257 (Nov. 28, 2022)
• The Rise of the Robotic Tax Analyst, 178 Tax Notes Fed. 57 (Jan. 2, 2023) 
• The Intersection Between Tax Credits And Trade Or Business, 178 Tax Notes Fed. 689 (Jan. 30, 2023)
• Relief of Innocent Spouses — Not So Podlucky, 178 Tax Notes Fed. 1339 (Feb. 27, 2023)
• Overcoming Accuracy-Related Penalties With Reasonable Cause, 178 Tax Notes Fed. 2145 (Mar. 27, 2023)
• Unbridled Losses: Harnessing Machine Learning for Tax Analysis, 179 Tax Notes Fed. 637 (Apr. 24, 2023)