Stanford Study Finds Lexis+ AI Provides Accurate Responses at “More Than Three Times” the Rate of Thomson Reuters Product

A new research paper published by Stanford University's Center for Human-Centered Artificial Intelligence (HAI), which is based on results from an independent empirical evaluation of AI-driven legal research tools, concludes that Lexis+ AI significantly outperforms Thomson Reuters’s Ask Practical Law AI in a number of important measures.

The study was conducted by researchers from Stanford’s RegLab and Stanford’s Center for Human-Centered Artificial Intelligence (HAI). The team documented that “Ask Practical Law AI’s answers are incomplete more than 60% of the time, leading Lexis+ AI to provide accurate responses at more than three times the rate.”

Among other head-to-head comparisons reported in the study are the following metrics:

• Lexis+ AI provides accurate (i.e., correct and grounded) responses on 65% of queries, while Ask Practical Law AI “refuses to answer queries 62% of the time” and “responds accurately just 18% of the time”;
• Lexis+ AI and GPT-4 were “quite capable at navigating our false premise queries and often corrected the false premise without hallucination. Lexis+ AI steers the user towards the correct answer”; and
• When looking solely at responsive answers, Ask Practical Law AI hallucinates at a similar rate to GPT-4 and “more than twice as often as Lexis+ AI.”

The researchers were somewhat limited in their ability do a true head-to-head comparison of products as LexisNexis and Thomson Reuters have to date charted different courses with their Gen AI product strategies.

Lexis+ AI offers users a single ecosystem drawing on the vast LexisNexis content set for AI-powered legal research, while Thomson Reuters offers two fragmented products — Ask Practical Law AI and Westlaw Precision AI — to customers. In fact, Westlaw Precision AI has not even been made available to law schools at this time, while Lexis+ AI was rolled out to law schools for testing in the Fall of 2023 and was made available to all faculty and law students in February 2024.

Continuous Improvement of Technology

The research paper is generally skeptical of the product claims by the leading legal research providers, finding a “more nuanced reality” to be the case with respect to the quality of the answers surfaced by Legal AI tools at this point in time. But the authors acknowledge that the products are improving, writing that “even over the course of our study, we noticed the responses of these systems — particularly Lexis+ AI — evolve over time.”

We are, in fact, pouring investment into our technology solutions and continuously refining safeguards that aim to deliver the highest quality answers. We are particularly committed to constantly improving the performance of our AI-powered solutions, as illustrated by the recent rollout of a second-generation legal AI assistant on our breakthrough Lexis+ AI platform.

For example, this new release of the platform featured an enhanced Retrieval-Augmented Generation (RAG) Model that addresses many of the service issues noted in the study. This model parses user queries and extracts key semantic concepts, then goes out and captures the most relevant answers to those queries by consulting the comprehensive LexisNexis collection of primary law, case law, regulations, analysis and more. RAG technology enables Lexis+ AI to minimize hallucinated or unsubstantiated answers; it is becoming more precise at an astonishing rate and users will continue to see performance improvements on a week-by-week basis.

There is another important element to understanding a key difference in the way that Lexis+ AI delivers its answers to legal professionals.

Authority-Backed Linked Legal Citations

When we announced the launch of Lexis+ AI, our new generative artificial intelligence (Gen AI) solution designed to transform legal work, we described how our proprietary technology enabled us to deliver a Legal AI product “authority-backed linked legal citations” that are grounded in the world’s largest repository of accurate and exclusive legal content from LexisNexis.

The Stanford study raises questions about the risk of hallucinations with any AI-driven legal research tool, but it’s important to understand that our promise to you is not perfection, but that all linked legal citations are hallucination-free.

No Gen AI tool today can deliver 100% accuracy, regardless of who the provider is. Unlike other vendors, however, Lexis+ AI delivers 100% authority-backed linked legal citations connected to source documents, grounding those responses in authoritative resources that can be relied upon with confidence.

In the rare instance where a citation provided by Lexis+ AI does not include a link, users should always review the citation for accuracy. Remember, Lexis+ AI is meant to enhance the work of an attorney, not replace it. No technology application or software product can ever substitute for the judgment and reasoning of a lawyer. This is the same fundamental principle behind every tool in our tech stack.

When we built Lexis+ AI, our goal was to deliver the most authoritative content from LexisNexis back to inform the answer. Our technical architecture, combined with the world’s largest repository of accurate and exclusive legal content, allows us to make that happen. That is how we are able to deliver on the promise of delivering to you  hallucination-free linked legal citations.

Lexis+ AI Leading the Way

The new study from researchers at Stanford University provides empirical data that Lexis+ AI is delivering superior results than our competitors’ AI-generated legal research tools in a number of key areas. To learn more or to request a free trial of Lexis+ AI, please go to www.lexisnexis.com/ai.