How to Run a Claude Pilot at a Law Firm

Legal practice has conditions that make generative AI both appealing and risky: extensive document work, specialized language, deadlines, confidential information, and consequences for subtle errors. A convincing demonstration, therefore, is not the same as an acceptable implementation.

This method outlines a pilot for learning under controlled conditions. It does not replace an analysis of professional obligations, the contract with the provider, or review by a lawyer responsible for the relevant jurisdiction and area of practice.

# What should a legal pilot aim to demonstrate?

The pilot should answer an operational question: whether Claude can assist with a defined task at an acceptable level of quality, review, and risk. It should not try to prove that “AI is useful for the law firm” in the abstract.

A useful hypothesis takes this form: “For this type of document and this stage of the workflow, a trained user can produce a reviewable first result in less time, without increasing critical errors or omitting controls.”

The hypothesis includes the task, user, result, comparison, and limit. If any of these elements is missing, it will be difficult to interpret the result.

# Which use case should you choose first?

It is best to choose a frequent, well-defined, and reversible task. The output should be an input for review, not a final decision automatically sent to a client, an authority, or a counterparty.

Possible candidates:

• extract clauses and data from contracts using a defined template;
• compare two versions and prepare a list of changes;
• summarize an internal case file with references to the source material;
• turn authorized notes into a first draft;
• classify documents according to previously agreed criteria;
• formulate review questions about missing information.

Avoid starting with final legal advice, automatic filing of legal documents, outcome prediction, or research without source verification. The ease of generating text can conceal plausible errors.

The American Bar Association, in its Formal Opinion 512, identifies obligations related to competence, confidentiality, communication, supervision, duties to courts, and fees. Although that opinion does not replace the rules applicable in Mexico, it illustrates why a pilot should be designed as a supervised professional process.

# Which team should participate?

A pilot should not be left solely in the hands of the tool enthusiast. It needs distinct roles:

In small firms, one person may take on more than one role, but responsibilities should remain clear. The person configuring the pilot should not be the only one evaluating its success.

# Which documents can be used in the first test?

The first sample should minimize exposure and allow for an objective review. Public documents, in-house templates, synthetic materials, anonymized versions, or matters with documented authorization are preferable.

Simply removing the client’s name is not enough. Facts, dates, amounts, counterparties, and combinations of data can make a matter identifiable. Anonymization requires an assessment of the context, not just text substitutions.

Before uploading content, document:

• information classification;
• purpose of processing;
• fields to be removed or replaced;
• plan, role, and authorized workspace;
• people with access;
• expected retention period;
• how to delete or withdraw materials;
• person responsible for approving exceptions.

Anthropic states that it does not use chats from its commercial offerings to train models unless users explicitly participate or opt in, but it also explains that products that store conversations may retain them to provide the experience. The firm should compare the public policy with its current configuration and agreement.

# How do you prepare an evaluation set?

The sample should represent both routine work and exceptions. A small but intentional set is better than uploading hundreds of documents without expected answers.

For each case, record:

1. authorized input;
2. requested task;
3. reference result or review criteria;
4. facts that must not be invented;
5. items that must be cited;
6. condition that requires escalation;
7. impact of an error.

Include incomplete and contradictory cases. A useful legal assistant must detect when it lacks a sufficient basis; responding confidently despite insufficient information is a failure, even if the text sounds professional.

# How should Claude be configured for the pilot?

The project should contain only the instructions and sources needed for the task. Anthropic describes Projects as spaces with instructions and knowledge that can be shared across work plans. That structure is useful, but it does not replace curation.

The instructions should establish:

• limited assistance role;
• jurisdiction and practice area, where relevant;
• authorized sources and their order of priority;
• prohibition on inventing facts, citations, or authorities;
• output format;
• requirement to flag uncertainty and missing information;
• references to source pages, clauses, or excerpts;
• warning that the result requires review by a lawyer.

Avoid asking it to “act as the best lawyer.” A grandiose instruction does not create competence or accountability. It is more useful to describe the task, criteria, and limits.

# What workflow should each case follow?

The pilot should be repeatable. A reasonable sequence is:

1. The user confirms that the document is authorized.
2. Select the approved task template.
3. Claude produces a structured draft and identifies sources or missing information.
4. The lawyer reviews it against the original document and checklist.
5. Corrections, time, and error type are recorded.
6. Only the lawyer decides whether the result can be incorporated into the work.
7. The findings inform a weekly review of the instructions and scope.

During the pilot, avoid quietly editing the configuration after each case. Track versions so you can tell whether an improvement came from the model, the instructions, or a different sample.

# What metrics should be measured?

Time matters, but it should not dominate the evaluation. A fast draft that requires more difficult review may increase the total cost.

Use at least four dimensions:

Classify errors by severity. An incorrect comma and an invented authority cannot be averaged as if they were equivalent. Define in advance which failure requires stopping the test.

# How long should the pilot last?

It should last long enough to observe repetition and variability, not just to accumulate activity. A cycle of several weeks usually allows time to prepare the sample, train users, run cases, make adjustments, and repeat; the exact timeframe depends on how often the process occurs and reviewer availability.

Progress is measured not by weeks elapsed, but by cases evaluated with evidence. If the firm handles only two matters of that type per month, promising a quick conclusion would be irresponsible.

# What criteria can help determine whether to scale?

Before you begin, define three possible decisions: continue, adjust, or stop. Scaling requires the workflow to meet quality, data, operational, and economic criteria simultaneously.

You can continue when:

• critical errors remain within the approved threshold;
• the review remains clear and traceable;
• total time improves or frees up useful capacity;
• users follow the information rules;
• there is an owner for the process;
• the expected cost is sustainable.

Adjust if quality depends too heavily on one person, sources are disorganized, or the task needs to be divided. Stop when the risk cannot be controlled, review eliminates the benefit, or the tool does not provide the required control.

# How do you move from the pilot to operations?

Moving to operations requires consolidating instructions, permissions, training, support, and periodic review. You must also define which changes require repeating the evaluation: a new model, a different document type, a different practice area, a connector, or an automation.

Don’t move the pilot into production just because participants are satisfied. The operational version needs documentation, an owner, an incident channel, and suspension criteria.

For a broader overview, see Claude for Lawyers: Implementation Guide.

# Frequently Asked Questions About Claude Pilots at Law Firms

# Can real case files be used?

Only after a documented review of confidentiality, privacy, contract terms, configuration, and applicable authorization. For the initial test, public, synthetic, or properly anonymized materials are usually preferable.

# Can Claude provide final legal advice to a client?

Claude can assist with tasks, but the firm retains professional responsibility. All legal output must be reviewed and approved by a qualified attorney before use.

# How many lawyers should participate?

There is no universal number. At a minimum, you need representation from the process owner, an expert evaluator, and actual users. The sample should allow you to compare results, not demonstrate popularity.

# Should we connect the document management system from the outset?

Generally, no. First, validate the task with a controlled sample. A connection expands access and complexity; add one only when the use case and permissions are clear.

# How do you detect hallucinations?

By requiring references to the source material, comparing each relevant claim, and using cases where information is missing. The review should look for fabricated facts, nonexistent authorities, and conclusions that go beyond the source.

# Does a successful pilot guarantee results when scaled?

No. Performance can change with users, documents, volume, models, and configurations. Scaling requires ongoing monitoring and reevaluation.

# Conclusion: A legal pilot should produce evidence, not enthusiasm

A Claude pilot at a law firm works when it reduces uncertainty about a specific task. Its results should show what improves, what errors arise, how much review costs, and what controls the operation needs.

The success criterion is not producing an impressive document. It is building a workflow where a lawyer can verify, correct, and decide with clarity. Without that traceability, there is not yet a defensible legal implementation.

# Sources consulted

• American Bar Association: Formal Opinion 512 on generative AI
• Anthropic: Claude for the legal sector
• Anthropic: What Projects are
• Anthropic: Data use in commercial offerings
• Anthropic: Data retention for an organization
• Chamber of Deputies: LFPDPPP
• NIST: AI Risk Management Framework