A 30-Day Legal AI Pilot: Accuracy, Review Time and Real Operating Cost

The decision is whether to expand the workflow, not whether the demo looks good.

A law firm can generate a document faster and still create more work for its attorneys. Missing source references, inconsistent facts and a failed handoff can move the bottleneck downstream. The managing partner needs to know whether an AI workflow produces usable work at an acceptable cost, and whether the team can keep it reliable after launch.

Gadient Consulting designs and manages that evaluation. We select one recurring task, measure the current process, configure the permitted workflow, train the participating team and maintain a scorecard through a controlled pilot. The deliverable is an operating decision: expand, revise and retest, or stop. A software subscription is only one input to that decision.

Most firms engage us at $10 million to $25 million in annual revenue. That describes a common starting point, not a minimum, cap or growth ceiling. At that stage, protecting attorney attention and coordinating existing systems can matter as much as adding another tool. This guide gives owners and COOs a practical way to evaluate the next AI investment without losing sight of those constraints.

The 30-day structure below starts after access, materials, responsible reviewers and the permitted use are approved. It is a working evaluation calendar, not a guaranteed implementation deadline or a promise that every workflow can be validated in a month. Low task volume, unresolved access or a changing process can require a longer test.

We define one unit of work and a finish line the firm can inspect.

We start with a queue the team already owns: extracting a defined set of facts from an approved record packet, preparing a source-linked first draft for attorney review, answering internal procedure questions from approved guidance, or organizing marketing reports for a manager. We do not combine unrelated tasks into a single AI productivity score.

For a record-review workflow, the unit might be one packet processed into the firm's required review table. Acceptance means the designated reviewer has checked required facts and references, resolved exceptions, approved the output and confirmed it reached the correct restricted location. A generated draft is an attempted output. It becomes completed work only after that acceptance and handoff.

We write a scope card with the allowed inputs, required output fields, authoritative sources, prohibited actions, reviewer, destination and exception owner. The firm determines the legal standard and what material is authorized. Gadient maps the workflow and configures the approved operating steps. Attorney advice, case acceptance and legal judgment remain with the firm's responsible people.

• Choose a task with enough recurring volume to observe ordinary work and difficult conditions.
• Name the system of record and confirm what access, export and integration permissions actually exist.
• Define what happens when the source is missing, contradictory, unreadable or outside scope.
• Set a spending ceiling, reviewer capacity and rollback owner before activating the pilot.
• Exclude tasks whose quality cannot be checked by an available qualified reviewer.

Prepare the required inputs using our PI firm AI readiness checklist.

The 30-day operating plan has four approval gates.

We assign evidence and an owner to each phase. The calendar advances when its gate is satisfied. If the test reveals a material access failure or the team cannot review the outputs, the next phase waits. Expanding access to stay on schedule would defeat the purpose of the pilot.

The firm's sponsor approves scope and spending. Its designated attorney or domain reviewer owns output acceptance. The operations lead owns the receiving queue and staff availability. Gadient owns the agreed configuration work, evaluation register, issue coordination and management report. The system administrator approves permissions and supports technical changes within the firm's access rules.

Swipe or scroll the table sideways to view every column.

Gadient implementation framework. Phase lengths adjust to access, volume, risk and the firm's approval capacity.

See the full service lifecycle for AI implementation and ongoing management for law firms.

We measure the existing work before asking whether AI improves it.

We observe the same task through preparation, execution, checking, correction and delivery. The baseline register records task type, input size, complexity, staff role, active work minutes, waiting time, outcome and reasons for rejection. We include messy work as well as clean packets. Otherwise the AI receives a curated test while the human comparison carries the real operating burden.

Selection is explicit: the date window, eligible queue, number selected, number excluded and reason for each exclusion. We report the sample size in tasks and reviewers, not just documents or prompts. A convenience sample supports a local operating decision; it does not establish a national accuracy rate or performance on every matter type.

For paired tests, we use the same authorized source materials and acceptance rules. Where feasible, independent reviewers assess outputs without knowing the method. If the same reviewer sees both versions, we record the order and account for familiarity. A second pass can look faster simply because the reviewer already knows the packet.

We also keep the comparison fair after launch. A week of short, clean packets cannot prove an improvement over a baseline of long, conflicting records. We separate results by the meaningful task categories and show missing time records rather than silently treating missing values as zero.

We separate configuration practice from the final evaluation.

The configuration set helps us improve instructions, output structure and routing. The held-out evaluation set checks how the frozen workflow performs on material not used to tune those choices. Repeatedly fixing the same five packets and calling their final scores validation would overstate what the system has learned to handle.

For record extraction, we include conflicting dates, missing pages, duplicate documents, poor scans, multiple people with similar names and facts that the source does not establish. The expected response to missing evidence is an explicit unresolved field and assigned review, not a plausible completion. For an internal knowledge assistant, we test outdated procedures, questions with no approved answer and requests outside a user's permissions.

We separately test operational failures: unavailable reviewer, rejected credentials, destination outage, duplicate retry and incomplete write. A failed delivery must remain visible and recoverable without silently creating a second record. A tool that answers well but loses work between systems has not passed the workflow test.

NIST's AI Risk Management Framework Playbook recommends documenting evaluation methods and test sets, testing in conditions relevant to deployment, and involving appropriate assessors. We use those principles to support this task-specific evaluation design. They do not prescribe a universal sample size or pass mark for law firms.

Source: NIST AI RMF Playbook: Measure, checked October 19, 2026

The scorecard separates accuracy from correction and final acceptance.

We agree the rubric before scoring. The reviewer distinguishes critical errors, material corrections and minor formatting issues. Incorrect identity, unauthorized disclosure or an unsupported statement with material consequences cannot be averaged away by correctly formatted rows. The firm sets the severity rules for its task and decides what warrants suspension.

First-pass quality and final acceptance answer different questions. If an attorney repairs an incorrect output, that may produce accepted work, but it does not turn the original generation into an accurate result. We retain both scores and the repair time. A high final acceptance rate paired with heavy correction can expose a workflow that only appears efficient.

The ABA's Formal Opinion 512 emphasizes an appropriate degree of independent verification or review based on the tool and task, with lawyers retaining responsibility for their work. The firm's legal leadership determines its obligations and review standard; Gadient makes the approved review path operational and measurable.

Source: ABA Formal Opinion 512, July 29, 2024; reviewed October 19, 2026

Swipe or scroll the table sideways to view every column.

Original evaluation worksheet, not measured performance. Define applicable facts, severity, cohorts and cost allocation before using these formulas.

We price accepted work, including the attempts that failed.

The recurring cost numerator includes attributable platform fees and minimum commitments, usage, storage or connectors, preparation, human review, correction, administration and monitoring. Failed attempts still consume those resources. The denominator contains only accepted, successfully delivered tasks. If none are accepted, cost per accepted task is undefined and the pilot has not demonstrated usable throughput.

We show one-time setup separately: workflow design, integration, security review, test preparation, training and migration. The operating view helps a firm understand steady-state use. The investment view adds setup over an explicitly stated planning horizon. Neither view should hide unused purchased capacity or charge the entire shared subscription to several workflows at once.

Consider this hypothetical planning worksheet for one 30-day cohort, created for this article on October 19, 2026. All inputs below are assumptions, not client results, vendor prices or industry averages. Both methods attempt 100 comparable tasks and deliver 90 accepted outputs. Labor minutes include work on all 100 attempts; missing records are assumed absent only for the calculation. Replace every input with measured firm data before making a buying decision.

Swipe or scroll the table sideways to view every column.

Hypothetical model: 100 task attempts per method, 90 accepted; not an observed sample. Rounded to cents. No revenue, tax, financing or downstream case-outcome effect is modeled.

Less total labor can still mean a more expensive constraint.

In that model, total labor falls from 3,500 to 2,000 minutes, but attorney review doubles. Recurring cost increases $500 and cost per accepted task increases $5.56. The apparent staff-time improvement consumes more of the firm's more expensive review capacity. A staff-only time-saved headline would miss the decision that matters.

Holding the model's other assumptions constant, AI attorney review must be no more than $2,500, or 833.33 total minutes, to match the baseline's $4,500 recurring cost. Across 100 attempts, that is approximately 8.33 review minutes per attempt. This is a break-even calculation for these inputs, not a recommended attorney-review limit. The firm's required quality review takes priority over hitting a cost target.

We use the finding to select an action. Better source references may reduce searching; a narrower task may remove difficult inputs; a clearer template may reduce rework. We retest that change before expanding licenses. If the reviewed work remains more expensive, leadership can still justify it for a separately measured quality or service benefit, but the decision memo must name that benefit and its evidence.

Freed salaried time is capacity, not automatically cash savings. We identify what work will use it, whether overtime or outsourced cost actually changes, and whether the bottleneck moved. A 30-day pilot also cannot establish improvements in settlement value, collected fees or profit for matters with much longer life cycles.

We manage the pilot through evidence, approvals and versioned changes.

Every run receives a task ID, source version, configuration version, operator, reviewer, outcome and time/cost record. Sensitive work stays in approved restricted systems. The management scorecard uses only the information needed to make decisions, with access appropriate to each role. Vendor access, retention, logging, subprocessors and export terms must be reviewed for the actual configuration; a marketing claim is not an access-control test.

Gadient reviews the exception register with the operating owner daily during controlled use. We group failures by cause: source quality, instruction ambiguity, unsupported output, permissions, routing, integration or human review capacity. The weekly meeting assigns a correction, owner, retest and next decision. We freeze a configuration for evaluation so that mixed versions do not masquerade as one result.

For voice workflows, our ongoing service includes additional AI review bots that analyze transcripts daily for missed intake questions, repetitive behavior and sentiment signals. They produce flagged exchanges and recommendations. A human evaluates and approves changes before we update applicable agent instructions, knowledge or workflow settings, run scenario tests and monitor subsequent calls. Sentiment informs review; it is not a definitive judgment about a caller.

That process refines the configured agent. It does not imply retraining a vendor's underlying model. We also check the review bots against human-reviewed material, including calls they did not flag, because an automated reviewer can miss an error. Review access and transcript handling follow the firm's approved confidentiality controls.

Apply the voice-specific controls in our Bland AI versus ElevenLabs intake guide.

We make the final decision with gates, not a blended score.

Before testing, we document the quality floor, critical stop conditions, acceptable human workload, delivery requirements and spending limit. These are agreed for the firm's task and risk, not borrowed from a generic AI benchmark. Zero observed critical failures in a small test does not prove zero future risk. If the sample is too narrow, the correct decision is to continue a restricted evaluation.

The strongest objection to a formal pilot is that it can cost more than simply letting a small team try a low-cost tool. That is reasonable for a limited, reversible task using non-sensitive material with easy human checking. We scale the evaluation effort to the consequences. We do not apply an enterprise rollout process to every brainstorming prompt, or a casual trial to unattended client-facing work.

Swipe or scroll the table sideways to view every column.

Gadient decision framework. Quality and access gates precede any benefit calculation.

What the firm receives, and who owns it after day 30.

The engagement produces a workflow and responsibility map, permitted-use scope card, baseline register, approved test set and rubric, configured pilot, integration and recovery test record, task-level scorecard, cost worksheet, training guidance and final decision memo. The memo names the recommended investment, the evidence behind it, unresolved limitations and the next review date.

If the workflow continues, we agree the ongoing management scope: quality sampling, exception review, approved configuration changes, access maintenance, usage-cost review and team coaching. The firm retains a named business owner and appropriate attorney oversight. A new vendor release, changed source format, expanded task or material incident can trigger reevaluation rather than waiting for the next routine meeting.

Gadient Consulting provides this work through scoped AI implementation and management engagements and, where appropriate, the broader fractional CMO relationship. That connection matters when AI competes with marketing staff, vendor spending or other growth priorities. We bring those choices into one operating plan rather than letting a software purchase become an unowned side project.

Bring one recurring work queue, its current owner and a description of where it stalls to a consultation. We establish what a successful pilot must demonstrate, what the firm must supply, and whether AI is the right next use of management attention.

Connect the decision to fractional CMO leadership and budget priorities.

Bring your workflow to an AI implementation consultation.