Legal AI is usually framed as a model problem. Better models. Larger models. More capable models. The assumption is that if the technology is powerful enough, usefulness will follow.
The empirical evidence suggests a different conclusion. Legal AI does not fail because models are insufficiently advanced. It fails because the dominant metaphor is wrong.
The most effective legal AI behaves less like an automated system and more like a mentor.
This insight emerged during a series of empirical classroom pilots run through Product Law Hub using an AI-based legal coach called Frankie. The pilots were designed to observe how users develop judgment-based legal skills when working alongside AI. The findings draw on quantitative engagement data and qualitative interviews conducted throughout the course.
What consistently produced better learning outcomes was not authority, speed, or completeness. It was collaboration.
Automation Is The Wrong Aspiration
Much of legal AI development is oriented around automation. Reduce effort. Eliminate steps. Deliver answers faster. That framing works for clerical or repetitive tasks. It breaks down when the task is judgment.
Judgment cannot be automated without being diminished. It requires context, prioritization, and explanation. When AI systems attempt to replace those processes with outputs, they strip away the very work that produces expertise.
In the classroom pilot, authority-driven interactions exposed this limitation quickly. When the AI behaved like a tool that delivered conclusions, engagement dropped. Users deferred rather than reasoned. Learning slowed.
The model was capable. The interaction was wrong.
Mentorship Is How Lawyers Actually Learn
Lawyers do not develop judgment by being handed answers. They develop it through guided struggle. A senior lawyer asks questions, challenges assumptions, and explains why something matters. They do not solve the problem for you unless it is necessary.
The most effective AI interactions in the pilot mirrored that dynamic. When the system asked clarifying questions, surfaced tradeoffs, and prompted users to articulate reasoning before responding, engagement increased. Quantitative data showed longer sessions and more iterative exchanges. Interviews revealed greater confidence and stronger retention.
The AI did not become smarter. It became more mentor-like.
Authority Shuts Learning Down
One of the clearest contrasts in the data was between collaborative and authoritative modes. When the AI asserted answers early or framed guidance as definitive, users disengaged. They moved faster but learned less.
This is not surprising. Authority short-circuits curiosity. Once an answer is presented as final, there is little incentive to explore alternatives or test assumptions.
In contrast, when the AI withheld judgment and instead invited reasoning, users stayed cognitively involved. They treated the interaction as a conversation rather than a transaction.
Legal AI that defaults to authority undermines its own value.
Collaboration Scales Better Than Control
There is a temptation to believe that authoritative AI is safer. Clear answers feel controllable. Collaborative systems feel messy.
The pilot suggests the opposite. Collaborative AI produced more durable learning and more trust. Users were better able to explain their reasoning and adapt it across scenarios.
Control may reduce short-term risk. It increases long-term dependence. Mentorship builds capability.
This distinction matters as AI becomes embedded in training and workflows. Systems that act as authorities create passive users. Systems that act as mentors create better lawyers.
Why Models Keep Getting The Metaphor Wrong
Part of the problem is language. We talk about models, not relationships. We optimize for outputs, not interactions. We evaluate correctness, not growth.
Mentorship does not fit neatly into benchmark metrics. It is harder to demo. It takes longer to show value. But it aligns far more closely with how legal expertise actually develops.
The Product Law Hub pilot made this visible by stripping away performance theater. Students did not care how fast the AI responded. They cared whether it engaged with their thinking.
Mentors Adapt. Models Repeat.
Another insight from the pilot was how quickly trust eroded when the AI repeated itself or applied the same framework regardless of context. Repetition signaled inattention. Users disengaged.
Mentors do not repeat scripts. They adapt. They notice what the learner already understands and adjust accordingly.
When the AI adapted its approach based on prior exchanges, users attributed greater intelligence to it, even when its substantive guidance was constrained. Trust followed attentiveness, not sophistication.
The Cost Of Choosing The Wrong Metaphor
Choosing automation as the dominant metaphor for legal AI carries a cost. It encourages tools that optimize for speed over understanding and authority over engagement. Those tools may look impressive but fail quietly in practice.
Choosing mentorship as the metaphor changes design priorities. It emphasizes questioning over answering, adaptation over uniformity, and explanation over assertion.
The classroom data suggests that this shift is not philosophical. It is practical.
What This Means For Builders And Buyers
For builders, the takeaway is clear. Stop asking how much the model can do. Start asking how it behaves when a user is uncertain, wrong, or exploring.
For buyers, the question is not how many tasks a system can automate. It is whether the system helps lawyers think better over time.
Legal AI will be judged not by its outputs, but by its influence on judgment.
The Future Of Legal AI Is Relational
The most important lesson from the empirical classroom work is that legal AI succeeds when it respects how lawyers learn. That learning is relational. It is iterative. It depends on challenge and explanation.
Models will continue to improve. That is inevitable. What is not inevitable is how we choose to deploy them.
If legal AI continues to chase automation, it will keep disappointing. If it embraces mentorship, it has a chance to become something far more valuable.
Legal AI does not need to replace lawyers. It needs to teach them how to think.
Olga V. Mack is the CEO of TermScout, where she builds legal systems that make contracts faster to understand, easier to operate, and more trustworthy in real business conditions. Her work focuses on how legal rules allocate power, manage risk, and shape decisions under uncertainty. A serial CEO and former General Counsel, Olga previously led a legal technology company through acquisition by LexisNexis. She teaches at Berkeley Law and is a Fellow at CodeX, the Stanford Center for Legal Informatics. She has authored several books on legal innovation and technology, delivered six TEDx talks, and her insights regularly appear in Forbes, Bloomberg Law, VentureBeat, TechCrunch, and Above the Law. Her work treats law as essential infrastructure, designed for how organizations actually operate.
The post Why Legal AI Needs Mentors, Not Models appeared first on Above the Law.