P2P

16 WHEN AGENTS GO WRONG: LEARNING FROM EARLY FAILURES As things move quickly, and while this article presents a rosy view of what is possible, it's essential to balance this against the risks and mitigate them appropriately. High-Profile Missteps: Early AI agents have made costly errors, revealing the technology's current limitations and need for oversight. Legal AI systems have cited non-existent cases that have gone unchecked. In consulting, AI agents have misinterpreted client requirements, producing work that was technically competent but strategically misaligned. Microsoft researchers have identified several novel failure modes specific to agentic systems, including agent compromise, agent injection, and agent impersonation. Accountability Gaps: When an AI agent makes an autonomous decision that goes wrong, determining responsibility becomes complex. Unlike traditional software failures, agentic AI operates with sufficient independence that simple "user error" explanations often do not apply. It'll be necessary to grapple with questions such as: Who is liable when an AI agent negotiates unfavorable contract terms? How do you maintain professional responsibility standards when an AI system is making substantive decisions? The Control Paradox: The more autonomous an AI agent becomes, the harder it is for humans to maintain meaningful oversight without defeating the purpose of delegation. Many organizations find themselves caught between micromanaging their AI agents (eliminating efficiency gains) or giving them too much autonomy (increasing risk). This tension requires careful calibration that most organizations are still learning to navigate. Cascading Failures: Perhaps most concerning, safety risks arise when AI malfunctions in critical systems can trigger cascading failures throughout interconnected networks. The lack of human supervision can exacerbate these risks, particularly in deployments within enterprise environments. Mitigation Strategies: Developing failure protocols including comprehensive audit trails, clear escalation triggers, regular competency testing for AI agents, and "circuit breakers" that automatically revert to human control when certain risk thresholds are met. SUMMARY In the past few months, AI has rapidly matured from a helpful tool to an autonomous agent capable of orchestrating parts of knowledge work and even delegating tasks back to humans. This emerging paradigm, where AI serves as a colleague, promises significant leaps in productivity and new modes of innovation, provided organizations adapt accordingly. Success will require reimagining job roles and work, building trust in human–AI collaboration, and instituting strong human-in-the-loop oversight to ensure these powerful agents remain aligned with human goals and organizational values. The companies and leaders that get this right stand to unlock tremendous value, as AI moves from performing tasks for us to partnering with us in the pursuit of knowledge work excellence. ABHIJAT SARASWAT helps lawyers spend less time managing work and more time doing the work. He is the Chief Revenue Officer at Lupl - the task and work management tool for lawyers. Ab is also the Founder of Fringe Legal, though which, for the last six years, he has created cutting-edge content for legal innovators focused on putting ideas into practice. He is a Barrister (non- practising) and was called to the Bar of England and Wales in 2015. Abhijat has worked for several large multi-national corporations across a range of sectors and holds a Bachelor's Degree in Forensic Science and Neuroscience from the University of Keele, UK.

• Cover