Peer to Peer: ILTA's Quarterly Magazine
Issue link: https://epubs.iltanet.org/i/1544492
P E E R T O P E E R M A G A Z I N E ยท S P R I N G 2 0 2 6 65 I t started with a demo. Someone from Microsoft, Harvey, or Thomson Reuters walked into your conference room, showed your managing partner what generative AI could do with a contract or a research memo, and within weeks, your firm had a deployment timeline. Maybe it was Copilot across the Microsoft 365 stack. Maybe it was CoCounsel embedded in Westlaw. Maybe it was Harvey plugged into your document management system. Whatever the tool, the pitch was the same: deploy fast, gain efficiency, win clients. That was 18 months ago. And now your help desk is fielding a different kind of call. THE SYMPTOMS ARE ALREADY HERE Your AI tools are getting dumber. Not because the models degraded, but because the repositories they draw from have. Every firm that pointed a generative AI tool at its document management system without first cleaning that system handed the AI an undifferentiated mass of drafts, duplicates, outdated templates, and abandoned work product. The AI does not know that the 2019 lease template was superseded. It does not know that the third draft of a brief was the one that went to court. It treats everything as equally authoritative, and the results reflect it. Attorneys who were impressed six months ago are now complaining that the AI "isn't as good as it used to be." It is exactly as good as it was. Your content just made it functionally worse. Sensitive content is surfacing where it should not. A litigation AI assistant that can search across matters is also an AI assistant that can surface privileged communications, work product from adverse parties in prior representations, and documents subject to ethical walls. Most firms implemented AI access controls that mirror their existing DMS permissions -- which were never designed to account for a tool that could search, synthesize, and surface content across thousands of matters in seconds. The permission model that was "good enough" for human-speed document retrieval is dangerously inadequate for AI-speed content synthesis. Several firms have already had close calls. Some have had worse than close calls and are not talking about it publicly.