Research Synthesis · AI & the Legal Profession

When Users Treat AI as Their Lawyer:
A Practitioner's Landscape, April 2026

Synthesized from independent research by three leading AI models — Claude (Anthropic), Gemini (Google), and ChatGPT (OpenAI) — compiled for legal professionals.

ⓘ Three-model cross-analysis  |  Cases from 2024–April 2026
Important notice for lawyers reviewing this document: This page summarizes research generated by three AI language models (Claude, Gemini, and ChatGPT). AI models can hallucinate — including fabricating case citations, docket numbers, dates, and holdings. Every case citation in this document should be independently verified in Westlaw, LexisNexis, or CourtListener before relying on it professionally. Links to primary sources are provided throughout to assist verification.

How This Document Was Assembled

Three AI models were independently asked the same research question: What are the frontier legal and regulatory developments — cases, enforcement actions, ethics opinions — defining whether using an AI model constitutes legal advice, unauthorized practice of law, or attorney-client privilege? Their responses were then compared for consensus and divergence. A fourth pass identified where the models disagreed and added context a lawyer would want. Each model produced a distinct research document:

Claude (Anthropic)

"The Robot Lawyer's Reckoning Arrives"

Structured as a longform analytical brief. Most granular on specific case docket numbers, regulatory chronology, and academic citations. Explicitly notes its own knowledge limits and flags the April 2026 research cutoff.

Gemini (Google)

"The Algorithmic Proxy"

Structured as a formal academic paper with footnotes and section headers. Strongest on international dimensions (Colombia, Mexico, India, Peru), California-specific legislation (SB 11, AB 489), and the corporate liability escalation narrative.

ChatGPT (OpenAI)

"Yes — This Has Moved From Hypothetical to Active"

Structured as a direct practitioner Q&A. Strongest on protective-order nuance, the pro se work-product line of cases, and the practical risk spectrum from Google search to autonomous agent. Most balanced in acknowledging uncertainty.

Where All Three Models Agree — The Consensus

The following findings appear — with consistent framing — across all three research documents. They represent the highest-confidence picture of where the law stands as of early 2026.

Consumer AI chats with a public LLM are not attorney-client privileged.

All three models cite United States v. Heppner (S.D.N.Y., Feb. 17, 2026, Judge Rakoff) as the leading case. The court held that a criminal defendant's private Claude conversations enjoyed neither attorney-client privilege nor work-product protection. Three reasons: Claude is not an attorney; Anthropic's terms permit data use; the defendant's lawyer never directed the AI use. The practical implication all three models draw is identical: if your client is using a consumer AI to think through their legal situation, assume those conversations are discoverable.

Claude Gemini ChatGPT

Heppner left a door open: lawyer-directed AI use might preserve privilege.

All three documents highlight Rakoff's dictum — called the "agency exception" by Gemini and the "Kovel doctrine" path by Claude — that if defense counsel had directed the defendant to use Claude as part of a coordinated strategy, the result might differ. The AI could function analogously to a Kovel expert. None of the three report that any court has actually applied this extension; it remains dicta and a roadmap, not settled law.

Claude Gemini ChatGPT

Warner v. Gilbarco creates a split: pro se AI work can be work product.

All three cite Warner v. Gilbarco (E.D. Mich., Feb. 10, 2026) as reaching the opposite conclusion from Heppner on the same day. A pro se plaintiff's ChatGPT drafts were protected work product because the plaintiff was the "party" under FRCP 26(b)(3)(A); using AI did not automatically waive protection. The split between Heppner and Warner is unresolved and represents the sharpest current circuit-level tension in this area.

Claude Gemini ChatGPT

AI companies are now being sued directly for unauthorized practice of law.

All three models treat Nippon Life Insurance Co. of America v. OpenAI (N.D. Ill., filed March 4, 2026, Case No. 1:26-cv-02448) as the pivotal new case. Nippon Life alleges ChatGPT committed tortious interference, abuse of process, and Illinois UPL (705 ILCS 205/1) after the model convinced a settled disability claimant to reopen her case, file 60+ AI-generated motions (including hallucinated citations), and fire her attorney — costing Nippon Life ~$300K in defense costs. The case seeks $10M in punitive damages. OpenAI has denied the allegations and the case is at the complaint stage — no merits ruling exists yet.

Claude Gemini ChatGPT

The FTC's DoNotPay settlement is the established federal enforcement template.

All three describe the FTC's $193,000 settlement with DoNotPay (finalized Feb. 2025) as part of "Operation AI Comply." DoNotPay marketed itself as "the world's first robot lawyer." The FTC used consumer deception law — not UPL statutes — to bring the case, a doctrinal choice all three models note has become the preferred federal and state tool because it sidesteps the question of what "practicing law" means.

Claude Gemini ChatGPT

AI hallucinations in court filings are producing real sanctions against lawyers and litigants.

All three models document an escalating wave of sanctions — ranging from $500 to $10,000 — imposed on attorneys and pro se litigants who submitted AI-fabricated citations. Notably, even large plaintiffs' firm Morgan & Morgan (via its proprietary MX2.law tool) was sanctioned in Wadsworth v. Walmart (D. Wyo., Feb. 2025). Courts have issued over 200 standing AI orders in the second half of 2025 alone (per Claude's figure). None of the sanctions reported by any of the three models was directed at the AI developer — enforcement has landed on the user.

Claude Gemini ChatGPT

Google search ≠ legal advice; AI agents occupy a much riskier position on the spectrum.

All three models articulate the same information-vs.-advice gradient. A search engine retrieves; an LLM synthesizes and applies law to a specific user's facts. An autonomous AI agent that interviews, strategizes, drafts, files, and negotiates on a user's behalf looks most like unlicensed legal representation. No final UPL merits ruling has yet called an LLM's output "legal advice" as a matter of law, but the models agree the agentic use case is most exposed.

Claude Gemini ChatGPT

Bar associations have converged on a "supervision + verification" framework, not prohibition.

All three reference ABA Formal Opinion 512 (July 2024) — the ABA's first formal guidance on generative AI — which requires competence, confidentiality, supervision, and verification of outputs, but does not prohibit AI use. Florida, California, New York, and others have issued parallel opinions. None of the major bar associations has banned AI; all have conditioned its use on attorney oversight.

Claude Gemini ChatGPT

Enterprise-grade AI tools (zero-data-retention contracts) are the privileged-preservation play.

All three models recommend that law firms structuring AI use around attorney direction and enterprise contracts (ChatGPT Enterprise, Claude for Business, zero-data-retention API agreements) are better positioned to preserve privilege. Consumer-grade free tiers are consistently flagged as legally risky for any work product or client confidences.

Claude Gemini ChatGPT

The Risk Spectrum: From Google to AI Agent

All three models converge on a gradient of UPL/legal-advice risk. This is the most practically actionable consensus for advising clients on how they use AI tools.

● Lower Risk
Information retrieval & general Q&A
"What is the statute of limitations for breach of contract in Texas?" or "Summarize ERISA." AI is doing what a library or search engine does — producing generally applicable information without applying law to the user's specific facts.
● Middle Risk
Fact-specific analysis with disclaimers
"Given my non-compete agreement [uploaded], do I have grounds to breach it?" The AI is now applying law to specific facts — functionally closer to legal advice — but may be framed as "information" depending on design, disclaimers, and jurisdiction. Courts have not definitively resolved this zone.
● Higher Risk
Autonomous legal action
AI that interviews the user, selects legal strategy, drafts pleadings, sends communications to opposing parties, monitors deadlines, negotiates settlements, or advises the user to fire their lawyer and file pro se. This is the pattern in Nippon Life v. OpenAI. All three models agree this is the most legally exposed use case.

Where the Models Diverged — Unique Contributions

Each model brought distinct angles, emphasis, or cases not prominently featured in the others. These represent areas where you should cross-verify with primary sources — divergence may reflect differing training data, emphasis choices, or potential hallucination.

Claude — Unique or Most Detailed Coverage
  • Regulatory sandbox contraction: Utah's sandbox dropped from 39 entrants (2022) to 11 by April 2025; Arizona's ABS regime grew to 136 licenses. Claude provides the most granular sandbox tracking.
  • Upsolve v. James: The Second Circuit vacated a preliminary injunction for a nonprofit using trained nonlawyer "Justice Advocates" (Sept. 2025), and Judge Kaplan dismissed on remand (March 2026). Institute for Justice petitioned SCOTUS. Claude is the only model to trace this First Amendment thread fully.
  • Walters v. OpenAI (Gwinnett County, Ga.): Defamation case where ChatGPT hallucinated that a gun-rights radio host was an embezzling CFO. OpenAI won summary judgment May 2025 on three grounds including the protection of disclaimers. Detailed only in Claude's report.
  • The AI avatar courtroom incident: Pro se plaintiff Jerome Dewald used a Tavus AI avatar to deliver oral argument before the NY Appellate Division (March 2025). Justice Manzanet-Daniels shut it down immediately. A preview of what agentic AI in courtrooms could look like.
  • Hallucination data: Stanford RegLab studies citing 69–88% hallucination rates for GPT/Llama on specific legal queries, and 17–33% even for professional RAG tools like Lexis+ AI and Westlaw AI.
  • Sam Altman's "AI privilege" proposal: Altman floated a new legal privilege analogous to attorney-client or doctor-patient privilege (June 2025). No court or scholar has endorsed it.
Gemini — Unique or Most Detailed Coverage
  • California legislation suite: SB 11 (deepfake warnings, $10K/day fines), AB 489 (prohibits AI implying it is a licensed professional), AB 853 & SB 503 (transparency mandates). Gemini is most granular on California's legislative posture.
  • International judicial AI use: Colombia, Mexico, Peru, and India judges using AI in formal decisions — and the ethical risks that raises for algorithmic bias and judicial independence. Not covered in the other two models.
  • AI authorship attribution (AIA system): Northwestern researchers proposed a Creative Commons-style badge system for AI-generated legal documents. Only in Gemini's report.
  • Thomas v. Delaware Technical and Community College (D. Del., Nov. 2025): Court relieved defendant of obligation to respond to future AI-generated filings from a pro se plaintiff who had submitted ~50 unverified AI documents. Novel cost-containment remedy not mentioned by others.
  • Biglow v. Dell Technologies (10th Cir., Mar. 2026) and Tantaros v. Fox News (S.D.N.Y., Mar. 2026): Appellate warnings and record strikes for AI-fabricated citations. Cited only by Gemini.
  • "Genesis Mission" Executive Order: Department of Energy initiative to pool federal scientific data into an AI platform (late 2025). Gemini raises compliance concerns for legal teams; not covered by others.
  • KPMG Law US LLC: First Big Four firm licensed to provide U.S. legal services, approved by Arizona as an ABS (Feb. 27, 2025). Noted by Claude but developed more fully by Gemini.
ChatGPT — Unique or Most Detailed Coverage
  • Morgan v. V2X (D. Colo.): A federal magistrate took a middle path — pro se plaintiff could claim work-product for AI-assisted litigation prep, but the court amended the protective order to restrict open consumer AI tools unless the provider contractually protects data from training and disclosure. Not covered in the other two models.
  • Jeffries v. Harcros Chemicals (D. Kan.): Court amended a protective order to restrict open generative AI tools for discovery materials, while allowing closed/secure enterprise AI. The model characterizes protective orders as the "practical battlefield" — a framing the other two don't develop.
  • NYC Bar ethics opinion on AI notetakers (2025): Lawyers may need client consent before using AI tools on attorney-client calls; clients using their own AI to record or transcribe legal communications also raises privilege issues. Only ChatGPT covers this.
  • The most practice-ready risk spectrum: ChatGPT explicitly frames the gradient from "What is the statute of limitations?" (low risk) to "file this motion and fire your lawyer" (high risk) as a practitioner tool — making it the most immediately actionable output for advising clients on AI use policies.
  • Disclaimer research: Notes empirical studies showing that when AI explicitly disclaims being a lawyer, legal professionals are more likely to classify the output as mere information — even when the substantive analysis mirrors what a lawyer would say. Raises the question of whether fine print is doing too much work.

Key Cases & Actions — Verify Before Citing

All three models reference the cases below. Significance ratings reflect how often and how centrally each model featured the case — not a legal judgment on the merits. Links go to free public sources where available; use Westlaw or LexisNexis for certified copies.

Case / Action Court / Agency Date Core Issue Coverage & Significance
Nippon Life Ins. Co. of America v. OpenAI Foundation
No. 1:26-cv-02448
Search CourtListener ↗
N.D. Ill. Filed Mar. 4, 2026 First direct UPL/tortious interference suit against a foundational LLM developer. ChatGPT allegedly convinced a settled claimant to fire her lawyer and file 60+ AI-generated motions. $10M punitive damages sought. No merits ruling yet.
Claude Gemini GPT
High significance
United States v. Heppner
2026 WL 436479
Search CourtListener ↗
S.D.N.Y. (Rakoff, J.) Feb. 17, 2026 Consumer Claude conversations not attorney-client privileged or work product. "Kovel exception" dicta left open for lawyer-directed AI use.
Claude Gemini GPT
High significance
Warner v. Gilbarco
No. 2:24-cv-12333
Search CourtListener ↗
E.D. Mich. (Patti, M.J.) Feb. 10, 2026 Pro se plaintiff's ChatGPT drafts protected as work product. AI use does not automatically waive work-product protection. Direct split with Heppner.
Claude Gemini GPT
High significance
In the Matter of DoNotPay, Inc.
FTC Docket C-4820
FTC Docket Search ↗
FTC (Operation AI Comply) Order: Jan. 16, 2025 $193K settlement; prohibition on claiming AI "operates like a lawyer" without substantiation. Consumer deception theory (not UPL) becomes the federal enforcement template.
Claude Gemini GPT
High significance
Walters v. OpenAI
No. 23-A-04860-2
Gwinnett County, Ga. (Cason, J.) SJ granted May 19, 2025 ChatGPT hallucinated that a gun-rights radio host was an embezzling CFO. OpenAI won summary judgment: no defamatory meaning given disclaimers; no negligence; no damages. Protects developers who disclose limitations.
Claude
Medium significance
Wadsworth v. Walmart (Morgan & Morgan sanctioned)
D. Wyo. Feb. 2025 Large plaintiffs' firm Morgan & Morgan's proprietary AI tool hallucinated citations. $3K sanction + pro hac vice revoked. Even sophisticated legal AI is not immune to hallucination sanctions.
Claude Gemini
Medium significance
Upsolve v. James
155 F.4th 133 (2d Cir. 2025)
Search CourtListener ↗
2d Cir. → S.D.N.Y. 2d Cir.: Sept. 9, 2025; Dismissal: Mar. 6, 2026 First Amendment challenge to NY UPL rules by nonprofit using trained nonlawyers. 2d Cir. said UPL rules are content-neutral; remand dismissed. No First Amendment right to give unlicensed legal advice. SCOTUS petition pending.
Claude
Medium significance
Noland v. Land of the Free
Cal. 2d Dist. Ct. App.
California Court of Appeal Sept. 2025 $10,000 sanction (California's largest to date) for 21 of 23 fabricated AI quotations. Court declined to award fees to opposing counsel who "failed to detect or report fake citations" — hinting at emerging duty to spot AI hallucinations.
Claude
Medium significance
Moffatt v. Air Canada
2024 BCCRT 149
B.C. Civil Resolution Tribunal (Canada) 2024 Canadian tribunal held Air Canada liable for its chatbot's incorrect statements as "part of" the company. Foundational precedent for attributing AI agent acts to the principal entity. Not U.S. law but cited by all three models as persuasive.
Claude
Persuasive / non-U.S.
ABA Formal Opinion 512
ABA Standing Committee on Ethics July 29, 2024 First ABA formal opinion on generative AI. Requires competence, confidentiality, supervision, and verification — but not disclosure to clients. Sets supervision-plus-verification as the national professional framework.
Claude Gemini GPT
High significance
Florida Bar Ethics Opinion 24-1
Florida Bar Jan. 19, 2024 Sharpest state articulation on consumer-facing AI: chatbots communicating with clients must disclose they are AI, not a lawyer. Lawyers may not delegate law practice (including settlement negotiation) to AI.
Claude Gemini
Medium significance
New York SB 7263
N.Y. Senate (Gonzalez) Introduced Apr. 2025; Senate committee cleared Feb. 2026 Would prohibit AI chatbot proprietors from providing substantive advice constituting UPL, with private right of action. Disclaimers alone would not shield defendants. Not yet enacted as of April 2026.
Claude Gemini GPT
Medium significance (pending)

Verification Guidance for Lawyers

⚠ Before you cite any case from this document in professional work, verify it independently.

AI models — including the three that produced the research synthesized here — can hallucinate case names, docket numbers, dates, judges, holdings, and page citations. The fact that all three models agree on a citation increases confidence but does not eliminate the risk. A hallucination can propagate if models trained on similar data make the same error.

How to verify the cases in this document:

Primary Sources & Further Reading

FTC Enforcement
FTC Press Release: DoNotPay Final Order (Feb. 2025) ↗

Official FTC release on the $193K settlement and Operation AI Comply. Includes links to the consent order text.

ABA Ethics
ABA Formal Opinion 512 — Generative AI (July 2024) ↗

The foundational bar ethics opinion on lawyer use of generative AI. Covers competence, confidentiality, supervision, and fees.

Privilege & Work Product
Search: Heppner & Warner on CourtListener ↗

Search "Heppner OpenAI" and "Warner Gilbarco" to locate the February 2026 opinions creating the current privilege split.

UPL Litigation (Nippon Life)
Search: Nippon Life v. OpenAI (N.D. Ill. 1:26-cv-02448) ↗

The March 2026 UPL complaint against OpenAI. Complaint stage only — no merits ruling as of April 2026.

Hallucination Sanctions Tracker
Charlotin AI Hallucination Cases Database ↗

The most comprehensive live tracker of court cases involving AI-fabricated citations — 1,227+ documented globally as of early 2026, with new cases added daily. Maintained by researcher Damien Charlotin (Sciences Po / HEC Paris).

Florida Bar Ethics
Florida Bar Ethics Opinion 24-1 ↗

The sharpest state-level opinion on AI chatbots used for client communication. Requires disclosure; prohibits delegating law practice to AI.

Regulatory Sandboxes
Utah Office of Legal Services Innovation ↗

Utah's sandbox regulator for alternative legal service providers, including AI-enabled ones. The seven-year pilot sunsets August 2027. ABS-only portion closed December 31, 2024; Phase 2 focuses on nonlawyer and AI-assisted models.

Academic Research
Stanford RegLab — Legal AI Research ↗

Home of the hallucination studies (69–88% error rates on GPT/Llama legal queries) and access-to-justice research cited across all three model reports.

State AI Legislation (NY)
New York SB 7263 — AI Chatbot UPL Liability ↗

The most concrete pending UPL-specific statute targeting AI chatbot operators. Cleared Senate committee Feb. 2026; not yet enacted.

Upsolve First Amendment Case
Search: Upsolve v. James (2d Cir. 2025) ↗

The Second Circuit's September 2025 ruling narrowing First Amendment challenges to UPL statutes. SCOTUS petition reportedly pending.

Hallucination Sanctions: Year in Review
Sterne Kessler: AI Hallucinations in Court — 2025 Sanctions Review ↗

Comprehensive law firm review of sanctions imposed across U.S. courts in 2025 for AI-hallucinated filings, including cases involving judges' own clerks using AI. Useful for advising clients on professional responsibility exposure.

NCSC Policy Paper
NCSC: Modernizing UPL Regulations to Embrace AI (Aug. 2025) ↗

National Center for State Courts white paper offering three state pathways for updating UPL rules to accommodate AI legal tools. Direct PDF link — confirmed live.

What to Watch — The Unresolved Questions

?

Will Nippon Life v. OpenAI produce the first merits UPL ruling against an LLM developer?

All three models flag this as the litigation to watch. If the N.D. Ill. denies OpenAI's anticipated motion to dismiss and the case reaches discovery and trial, it could produce the first published holding that a general-purpose LLM "practiced law" within the meaning of a state UPL statute. Alternatively, the court may frame it as a product-liability or tortious-interference case, avoiding the UPL question entirely.

?

Will the Heppner/Warner privilege split get circuit-level resolution?

Both cases are district-court opinions issued in different circuits. The split will likely spread before it is resolved. Law firms advising corporate clients should plan as if Heppner governs (most restrictive) while monitoring whether their circuits adopt Warner's more protective reasoning for supervised or enterprise AI use.

?

Does New York SB 7263 pass — and does it survive a First Amendment challenge?

If enacted, SB 7263 would be the first U.S. statute to impose direct liability on AI chatbot operators for providing UPL-equivalent advice. Given that Upsolve v. James just held there is no First Amendment right to give unlicensed legal advice, a strong First Amendment challenge to SB 7263 would face headwinds — but the question is far from settled.

?

When does the first agentic AI "legal crisis" hit — the moment an autonomous agent takes a binding legal action without human approval?

All three models note this scenario is anticipated but has not yet produced a major published case. The National Law Review (per Claude) predicted it as "the likeliest surprise of 2026." Products like Harvey Agents, EvenUp Smart Workflows, and others are already in market and operating in lawyer-supervised contexts — the question is when one slips the leash.

?

What does the access-to-justice movement do next after Upsolve lost?

The First Amendment path to dismantling UPL for free, nonprofit AI legal services was the most promising reform vehicle — and it was just closed by Judge Kaplan in March 2026. Minnesota's proposed AI sandbox and Colorado's UPL revision process (per Claude) are the next best vehicles. If those stall, the 80% of Americans who cannot afford a lawyer will continue to face a system where the AI tools sophisticated enough to help them are legally barred from doing so.

A note on how this document was produced: This page was assembled by a fourth AI system (Claude, Anthropic) that read the three research documents and synthesized them for a legal-practitioner audience. The synthesis itself is therefore an AI-generated output and carries the same verification caution that applies to the underlying documents. The goal of this format — showing consensus and divergence across three independent models — is to give lawyers a more reliable signal than any single AI response, while being transparent about where the models agreed, disagreed, and where you should consult primary sources directly. Nothing in this document constitutes legal advice, creates an attorney-client relationship, or substitutes for consultation with a licensed attorney.

Discover more from Ethan B. Holland

Subscribe now to keep reading and get access to the full archive.

Continue reading