publication of the International Legal Technology Association
Issue link: https://epubs.iltanet.org/i/1193169
I L T A S U R V E Y R E S U L T S | A R T I F I C I A L I N T E L L I G E N C E & M A C H I N E L E A R N I N G 9 2 0 1 9 A R T I F I C I A L I N T E L L I G E N C E & M A C H I N E L E A R N I N G S U R V E Y R E S U L T S What steps are you taking to manage and/ or eliminate bias in your organization's AI- powered software? There were really interesting responses about steps being taken to manage and/or eliminate bias in AI-powered soware. The most popular response was around aggregating data and doing data cleanup and review/ normalization. Some said it's most important to pay close attention to training data, actively reviewing for precision, recall and Xref results. A few interesting perspectives: • Using traditional tools to compare and vet the results • Defining AI principles to be governed against • Widening the data set and increasing the number of people involved in training and getting multiple inputs from different groups. Many people are not there yet, still just looking for appropriate training data. Conclusion Using some form of legal AI-powered software is an inevitability for the majority of legal organizations in the next few years. Looking at the business problems you are trying to solve and identifying where an AI tool might be the right answer, is key. And once you decide on a direction, following the discipline that makes technolo and software projects successful, and recognizing the special challenges with AI tools, will serve you well. Some thoughts on questions we need to be asking next: • What are the real-world results being achieved by AI tools, how broadly are AI tools being adopted, is there a full embrace of this technolo? • Size of firms adopting AI tools, what small to mid-size firms have successfully implemented within budget? • What business problems would law firms and law departments like the legal AI vendors to tackle, and what solutions would they like vendors to develop? • Who is really doing the training? SME? PMs? Outside consultants? • Do you plan to offer a "self-service" model to clients powered by AI on the back end? • If AI will reduce the need for lawyers and courts will you still support it? • Who bears liability for the results of AI decision-making? • What policies/procedures are being put in place to ensure that the use is legal and not misused? • How do law firms handle language issues? Non-English documents extract fewer value from out-of- the-box tools. ILTA