AI risks in law firms: confidentiality and hallucinations

The risks of using AI at a law firm are real, documented, and have names and case files. They’re also manageable. This framework separates the three main risks, assesses their scale using verifiable data, and assigns a specific mitigation to each one.

# What are the real risks of AI at a law firm?

Three, in order of severity for your professional responsibility: breach of confidentiality from uploading client information to improperly configured tools; hallucinations—that is, fabricated citations, precedents, or facts that appear plausible; and diluted accountability, when no one at the firm is responsible for what the machine produced.

Notice what’s missing from the list: whether AI works. The risk with these systems isn’t that they always fail, but that they’re right often enough to make you let your guard down. An associate who has reviewed fifty correct Claude drafts will tend to trust number fifty-one. The framework exists precisely for that moment.

# What about your clients’ confidentiality?

The risk depends on the plan you use, not the brand. With commercial products such as Claude for Work and the API, Anthropic does not use your inputs or outputs to train its models by default. OpenAI offers equivalent commitments on its enterprise plans. The typical problem is something else: lawyers using free personal accounts, outside the law firm’s control.

• Mitigation 1: subscribe to work plans (Team/Enterprise); never use personal accounts for client matters.
• Mitigation 2: disable feedback features that may share conversations with the provider.
• Mitigation 3: define in writing which categories of information must never be uploaded, with or without an enterprise plan.

# How serious is the hallucination problem?

Serious and growing. The Damien Charlotin database documents more than 1,800 court rulings across 59 jurisdictions where a court detected AI-hallucinated content, most of them in the United States. The landmark case is Mata v. Avianca: in June 2023, a federal judge in New York imposed sanctions of 5,000 dollars on two lawyers and their firm for submitting precedents fabricated by ChatGPT, and ordered them to notify their client and the judges who had been falsely cited.

And it’s not just the free chatbot. A Stanford study found that specialized legal tools hallucinate too: 33% of responses in Westlaw AI-Assisted Research and 17% in Lexis+ AI. The operational takeaway: no tool, specialized or general-purpose, removes the need to verify.

The good news is that mitigation is mechanical, not heroic: every citation, legal argument, or precedent produced by AI is checked against the primary source before being included in a filing. In Mexico, that means the Federal Judicial Weekly and official portals—not the model’s memory. It’s a step that takes minutes and avoids becoming case number 1,848 in Charlotin’s database.

# How can each risk be mitigated in practice?

With a specific control for each risk and a named person responsible. This is the framework we implement with law firms:

The rule that sums it all up: whatever a lawyer signs has been verified by a lawyer. AI can draft; responsibility to the client and the court cannot be delegated. Mexico does not yet have published sanctions equivalent to those in the United States, but professional confidentiality and civil liability already cover these situations. Don’t wait for the first local precedent.

# When should you not use AI yet?

Don’t use it yet if you don’t have an enterprise plan or a written policy: you’d be institutionalizing risk, not productivity. Nor should you use it for legal research without tools that cite verifiable sources, or in matters where the client’s confidentiality agreement prohibits processing data with third parties.

Saying “not yet” in those scenarios isn’t technophobia. It’s exactly the same judgment you’d use not to let a junior associate file a complaint without review. The difference is that the associate learns and faces consequences; the model does not. That’s why the right order is policy first, tool second.

# Frequently asked questions

# What is an AI hallucination in a legal context?

It’s fabricated content that appears plausible: nonexistent precedents, false citations, or made-up facts. More than 1,800 court cases have already documented instances where a court detected this content, according to Charlotin’s database.

# Can I be sanctioned in Mexico for false AI-generated citations?

There are not yet published sanctions equivalent to those in the United States, but submitting false citations may constitute a breach of professional duties and give rise to civil liability to the client under the existing framework. The reputational risk is immediate.

# Does Claude train on my firm’s data?

Not by default in commercial products: Anthropic states that it does not use inputs or outputs from Claude for Work or the API to train models unless you explicitly authorize it, for example by submitting feedback.

# Do specialized legal tools eliminate hallucinations?

No. Stanford’s study measured 33% hallucination rate in Westlaw AI-Assisted Research and 17% in Lexis+ AI. These tools reduce the risk compared with a general-purpose chatbot, but human verification is still required.

# Should I ban AI at my firm in the meantime?

A total ban usually leads to hidden use through personal accounts, which is the worst-case scenario for confidentiality. It’s safer to enable approved tools with a policy and training than to pretend no one uses them.

# Conclusion

• Three manageable risks: confidentiality, hallucinations, and unclear accountability.
• Mata v. Avianca (5,000 USD in sanctions) and 1,800+ documented cases show that the risk of false citations is real.
• Even specialized legal tools hallucinate: 17-33% according to Stanford.
• Mitigation: an enterprise plan, verification against primary sources, and a usage policy with designated owners.
• A secret ban mitigates nothing; enabling use with rules does.

A firm that manages these risks in writing can capture AI’s savings without putting its professional license on the line. See how we structure this at Claude Legal for Law Firms, see the implementation guide for lawyers or contact us to design your usage policy.