The AI ​​industry wants models to help in legal battles, but will they help?

So far this year, three of the four major AI laboratories have either launched or pitched products to law enforcement. Google previewed Gemini Enterprise for Legal in late August with four of the world’s largest firms, three months after Anthropic released what is now known as Claude Legal Solutions. Elon Musk is also interested, with his SpaceXAI lab setting up a dedicated legal solutions page on his website.

Together, it marks a push into legal services that will be strengthened even as lawyers continue to be fined and fined for filing AI filings replete with fictitious case law. In June, the Ontario Law Society Tribunal ordered lawyer Shahryar Mazaheri to pay tens of thousands of dollars for using applications drafted with Grok that contained fake case citations. Researcher Damien Charlotin’s database has cataloged almost 2,000 such cases worldwide, involving more than 800 lawyers and 1,100 self-represented litigants.

The Ontario Law Society found last December that Mazaheri used an earlier version of Grok to draft a factum (a written legal argument) for a client’s appeal. This, court documents say, produced fabricated quotes and invented legal principles. A costs decision issued in June ordered Mazaheri to pay $31,150, with the ruling noting that an LLM “does not appreciate nuance or exercise judgment or use a moral compass,” and is “strongly predisposed to give an answer — any answer — rather than admit ignorance.” Judges described his AI-generated materials as “gibberish”.

Sullivan & Cromwell, the white-shoe law firm that is one of the highest-grossing in the United States and pays its brand new, first-year lawyers a $225,000 base salary, is among the culprits. It filed an emergency letter with the Southern District of New York in April after a bankruptcy motion it filed with Chief Judge Martin Glenn found 42 AI-generated fabrications. In Canadian courts alone, reported cases involving fabricated citations increased from just seven in 2024 to 86 in 2025, with 39 more in the first quarter of 2026.

Before any major AI company released a specific legal tool, DoNotPay dropped plans in early 2023 for an AI “robot lawyer” to argue a case in court after state bar associations threatened criminal prosecution.

Google and Anthropic both imply that their legal products are built for verification. Google’s product routes queries via connectors to external legal databases such as Everlaw and NetDocuments, rather than relying on the model’s training data in isolation. Google Cloud CEO Thomas Kurian said at the launch that “ensuring every aspect of these agent workflows is accurate, factual and grounded and legal authority is of critical importance.” Gottlieb, Freshfields, Weil and Williams & Connolly, four of the largest law firms in the world, are involved in the preview program.

Anthropic took a similar approach with its Claude Legal Solutions product. It includes a reported 20 plugins on legal platforms in addition to 12 pre-built plugins. The lab claimed a 90.9 percent score on the BigLaw Bench legal reasoning benchmark for Opus 4.7 and counts Freshfields and Quinn Emanuel among its clients. The key difference between these products and the Mazaheri case is that instead of asking the model to generate quotes from memory, it pulls them from external, verified sources.

SpaceXAI’s Grok-powered legal product, meanwhile, has thinner credentials. A Cursor blog post said the Grok 4.5 is suitable for “finance, legal work, or anything else you do on a computer,” but SpaceXAI’s subsequent announcement for the model focused on “coding, agent tasks, and knowledge work” and made no mention of legal applications. The page also does not disclose an architecture or mechanism for tracking citations to primary sources. SpaceX officially acquired Cursor on August 15 for $60 billion, bringing Code Editor’s legal marketing under the Grok umbrella.

In a 2024 study, Stanford’s RegLab found hallucination rates of 17 percent and 33 percent after testing purpose-built legal AI platforms from LexisNexis and Westlaw. There has not yet been a comparable independent audit on Google or Anthropic legal tools.

All attorneys are subject to bar association rules that place a duty to verify citations on the attorney submitting them, regardless of whether AI was used or not. Mazaheri’s $31,150 fine was not imposed because he used AI, but because his output was not thoroughly checked. That obligation won’t change, no matter how carefully designed a legal AI tool is, or how many vendor safeguards are packaged inside.