Early Adopters Of Gen AI In Law

My current feeling about generative AI is that it will be useful and important for lots of organizations, but that it’s an existential issue if your company creates or distributes content for its living. That’s true of several industries—my own of education being one of them—including the legal profession. Creating contracts, briefs, opinions, filings, even arguments in court are all activities that could be dramatically transformed by gen AI. It’s also possible that the technology could misinform attorneys with its hallucinations in a dramatic way, as it has already done in a few cases. So getting it right is really critical for law firms, general counsels, legal software providers, and other players in the industry. Although AI might transform the future of the profession, things in general are moving pretty slowly in law firm AI, as they did for previous new technologies. I daresay most lawyers haven’t experienced any changes at all. However, I found in my book All In on AI that some companies are adopting AI much faster than others, and I suspected that might be true for law firms as well. They are indeed. For this piece I spoke with two early and aggressive adopters—one historically based in the UK, one in the US. Allen & Overy (now A&O Shearman after its recent merger with Shearman & Sterling), is historically a UK-based global firm that has been pursuing AI for several years. Shearman & Sterling was also active in AI before the merger, but perhaps less so than A&O. I also spoke with Wilson Sonsini, a Silicon Valley-based firm that has offices across the US and in several different countries. They’re known for their legal work with tech firms, so it is not surprising that they are also early adopters of AI. Generative AI at A&O Shearman I spoke with David Wakeling at (then) Allen & Overy, who was clearly the right person to chat with. He began as a conventional attorney at the firm, and over time moved into a technology innovation leadership role. He heads a group of developers and data scientists, is Chair of the AI Steering Committee, and Head of the Markets Innovation Group at the firm, which, he says, is funded with an investment fund and “empowered to disrupt the legal market.” That group is effectively a tech startup in a big law firm. Market leaders, I have found, get an early start with generative AI, and A&O did so. They started working with the technology (using GPT-4 in beta) a month or so ahead of the ChatGPT announcement in November 2022. They also got early access to Harvey, a law-specific custom version of LLMs, shortly after it was announced. By Christmas Eve 2022 the firm rolled out an LLM for 2000 lawyers to begin experimenting with. Wakeling believes that generative AI will be (and in some cases, already is) a great aid to increased productivity for lawyers, but he doesn’t think the current technology is of existential importance for big law firms like his—although it may be with future versions of the technology. Today, after the firm has used it for more than a year, his comments are decidedly mixed: This will make lawyers more productive and efficient. But the fundamental nature of generative AI is to make errors. We’ve found hallucinations in every system we’ve tried. You need an expert in the loop or you will get bad law. When we rolled it out we described it as a very articulate and elegant 13-year-old who doesn’t know its own limitations. It will clearly mean that lawyers will have to rethink how they train, and applying critical thinking to the output will be key to their success. The firm’s lawyers are achieving productivity gains across all levels. It’s particularly helpful in legal research using retrieval-augmented generation (RAG) prompting approaches, although lawyers and researchers still need to check sources. They can pass outputs on to a midrange lawyer, who can assess how the output compares to previous opinions. A partner at the firm might use it in a different way, e.g.: “I know my market practice in area X—ask Generative AI to tell me similar provisions to an existing set in a database, and express them in a way I might use it in a client.” Wakeling estimates that the productivity gains from using a Harvey-type system out of the box are at least 2 hours per week. They have developed “Contract Matrix,” a contract negotiation workflow driven by generative AI, which delivers bigger gains—5 to 10 hours per week. In terms of fine-tuning LLM model weights for legal applications, Wakeling says it is difficult, in large part because there isn’t enough data for training. They don’t want to employ their client documents to train general models for privacy reasons. Fine-tuning is also an expensive process. However, the firm has plenty of data for RAG-based prompt tuning, and the data is sufficiently well-curated to get good results. He expects that there will eventually be thousands of RAG-based LLMs that are used within large firms, but only 6 or 7 fine-tuned models based on specific industry sectors. Over the long run, Wakeling and his colleagues believe, generative AI will yield “very serious competitive advantage—it’s both an existential opportunity and an existential threat.” He doesn’t believe that any other firm is doing as much custom development as (then) Allen & Overy. Wakeling says he’s begun to see the technology as a strategic priority for most big firms in the last quarter, with senior managing partners mentioning AI in talks and articles. However, he’s not sure whether generative AI can become a competitive advantage in a conventional practice without the innovation focus his own firm has. “Is their culture aligned for innovation?” he said—suggesting that few are. Generative AI at Wilson Sonsini David Wang, the Chief Innovation Officer at Wilson Sonsini, assured me that his firm’s culture was also aligned for innovation and that it’s doing plenty of custom development. Like Wakeling, Wang started his law career as a conventional associate, but he’d led a small tech consulting firm before going to law school, and as a lawyer he began to practice “citizen development” for applications that would make him more productive and benefit his clients. The leaders of the firm noticed and eventually put him in charge of tech-driven innovation. Wilson Sonsini got an early start with AI in general and generative AI in particular. Wang said that AI work at the firm started about five years ago with an investment in Lexion, an AI-based contract development platform. He was on the company’s board for a few years. Lexion was recently sold to Docusign for $165 million—certainly a successful exit for Wilson Sonsini’s investment. At about the same time the firm embarked upon a long-term project to digitize, automate, and employ AI in multiple places across the business. Attorneys at the firm were, for example, discouraged from using paper notepads. More and more steps became automated; Wang says that 30 to 40% is very feasible. As at A7o Shearman, work on generative AI started before GPT-4 and ChatGPT, and the firm now has several production systems in use. And like Wakeling, he believes that the initial round of LLMs in law is interesting, but not yet disruptive to the practices of “big law” firms like his. Two key developments with the technology are taking place. First, there is a general enterprise level shift to having generative AI in every application, starting with email. Wang doesn’t see a lot of strategic benefit from this, but it has productivity benefits. Second, there is a separate, more strategic initiative with verticalized solutions that incorporate generative AI. This initiative is still very nascent, but Wilson Sonsini is very active in the market for such solutions. Wang expects that there will eventually be a “Cambrian Explosion” of generative AI models for use by lawyers and their clients. One new verticalized solution was announced by Wilson Sonsini last month. The firm had previously offered a digitization and automation platform for startup legal processes called Neuron. It recently added a new capability to Neuron that is described as “agentic,” i.e., taking actions on its own. It is a fixed-fee product—though it includes review by human attorneys—to review contracts for cloud services. The AI agent was created by a legal startup named Dioptra in a partnership with Wilson Sonsini. Wang believes that the solution’s accuracy in complex contract review, and the level of autonomy it offers, makes it a substantial breakthrough in legal AI. David Wang doesn’t minimize the challenge of getting law firms to view advanced technology as an integrated part of their business. A very small fraction of law firms have achieved that, he says. Wang doesn’t feel that Wilson Sonsini is quite there yet, but it’s making good progress. He certainly feels that it’s well ahead of most big law firms. What A&O Shearman and Wilson Sonsini Have in Common These two firms have some differences, but they have more in common with respect to AI. They both have leaders of innovation that are working to infuse AI into the business model. Both leaders are impressed by aspects of generative AI—particularly the productivity it can offer—and were early adopters of it, but feel that the technology needs to advance significantly before it is truly transformational. Both firms are working not only on internally-oriented, productivity-enhancing use cases, but on tools to transform legal services more broadly. Both are working with external startups in legal AI. Both have created contracts-focused products that employ AI. Both firms believe that careful review by human attorneys is an important step in generative AI usage. Legal services—and professional services more broadly—are generally slow to adopt new technologies. They charge by the hour, haven’t changed their work processes much over time, and investments typically come out of partners’ pockets. We probably shouldn’t look to firms in that industry for pioneering developments with generative AI or other technologies. Despite the obstacles, however, both A&O Shearman and Wilson Sonsini are working with great energy to change how legal work is done.