Last year, a lawyer in New York found out the hard way that ChatGPT has a tendency to make up stuff, a phenomenon known as hallucination. 

The case was Mata vs Avianca, Inc (2023). The claimant, Robert Mata, said he was injured by a metal serving cart during a flight to Kennedy International Airport in New York, USA. 

When the Colombian airline Avianca tried to brush off the case, Mata’s lawyers strongly opposed, shooting back with a ten-page brief with references to court decisions like Shaboon v. EgyptAir, Martinez v. Delta Air Lines, Zicherman v. Korean Air Lines, and Varghese v. China Southern Airlines.

The twist: none of these cases actually exist.

This fiasco is a perfect example of the pitfalls of jumping on the AI bandwagon without a safety net. While the new technology is making waves in medicine, research, and automating workflows, the legal sector has remained relatively untouched.

It is perhaps due to the demand for accuracy and a sharp eye for nuance, something AI hasn’t quite mastered—yet.

But can AI assist legal professionals and agencies in enhancing their systems—without compromising on accuracy and ethical standards? Or will it lead to more issues like the one encountered in the Avianca case?

Ask AI, but verify

According to industry insiders, while AI is helping lawyers tackle repetitive tasks, concerns over accuracy remain.

Mumbai-based AUCL, a corporate and legal advisory firm, is a case in point. The firm utilises AI-driven tools for legal research and document review, processing vast amounts of legal data—especially in complex cases. The firm has largely been benefited by AI to streamline certain areas of legal work, particularly when managing large volumes of documents during the discovery phase or when researching case law. 

However, the firm uses multiple layers of human oversight to ensure the accuracy of these solutions. 

“At AUCL, we see AI as a powerful tool to support legal professionals, streamlining repetitive tasks but not replacing human expertise. As AI evolves, it’s crucial to maintain a balance where technology complements rather than competes with professional judgment,” says Akshat Khetan, Founder of AU Corporate Advisory and Legal Services.

While concerns remain about potential pitfalls—such as the missteps highlighted in Mata vs Avianca, Inc—there’s a growing consensus that AI, when used responsibly and under proper oversight, can be a valuable asset to legal professionals.

“It’s always dangerous to use AI for any kind of research for that matter. It doesn’t always give an accurate answer and it’s confusing, especially in legal work where it generalises or makes up case laws. Hence, it’s always a problem to use AI,” says Harshita Tungesh, an advocate practising at the Supreme court. 

“On the basis of ability to think and analyse, AI can not replace a human brain. Your analysis is of your own, AI gives you a generalised perspective which is only fed to it. Careful cross-check, comparison and use of the human brain is very important,” she adds. 

The Supreme Court of India has acknowledged the role of AI in enhancing legal processes. Recently, the apex court confirmed that AI is being used for tasks such as translating judicial documents, enhancing legal research, and automating related activities. 

Minister of Law and Justice Arjun Ram Meghwal, in a written response to the Lok Sabha in August, mentioned that emerging technology is being adopted to transcribe oral arguments into regional languages in Constitution Bench matters.

“The Supreme Court is taking the lead on this by actively translating Supreme Court judgments into multiple regional languages. The initiative is yet to gain significant traction at the High Court levels,” Vikas Mahendra, Partner Keystone & Co-Founder, CORD & TERES tells YourStory. 

When asked about the role of AI in legal research and drafting, Mahendra explained that the key to managing the risks lies in treating AI like the product of a very smart and hard working, but inexperienced junior lawyer.

“AI is here to stay. Lawyers and judges are recognising the incredible power of AI and won’t go back to the old ways. Clients are beginning to expect cost savings achieved through use of AI being passed on to them. Younger lawyers, who are trained to do tasks across various facets of life using AI, will find it difficult to do without,” he explains. 

Furthermore, a committee has been formed by a Supreme Court judge to oversee the translation of important judgments made by the Supreme Court and High Courts. So far, the Supreme Court has translated over 36,271 judgments into Hindi and around 17,142 judgments into 16 regional languages, all available on the e-SCR portal.

Eight High Courts in the country have already launched their e-HCR (electronic high court record) portals, with several others in development.

Booming startups in the space

In 2021, global investments in legal tech startups soared past $1 billion—the highest annual total the sector has ever seen. In India, significant funding activity in legal tech began primarily around 2013, according to a report by IIMA Ventures.  The report titled “Beyond the Bench” reveals that between 2013 and June 2022 (the latest year for which such data is available), 32 Indian startups in this space raised over $57 million across 70 funding rounds.

However, pioneering efforts in the legal AI space aren’t new to India. In 2017, Mumbai-based Cyril Amarchand Mangaldas became the country’s first law firm to utilise AI for its legal services by signing an agreement with Canada-based Kira Systems, a machine learning provider.

Despite a Deloitte report highlighting the unstructured nature of legal data and the heavy reliance on professionals as barriers to GenAI disruption in the legal space, India is witnessing a surge of companies aiming to bridge this gap—and capture a significant share of the market opportunity of legal technology. 

One such company is Visakhapatnam-based LawyerDesk, which offers multiple AI-based platforms for lawyers, students, and enterprises. Among its offerings is Advocase.ai, featuring tools like automated legal drafting that reduce the time to produce complex legal documents from hours to minutes.

“While our AI tools are designed with precision and care, it’s essential for outputs to be reviewed by a legal professional before use. This includes verifying the accuracy of references and ensuring that all legal nuances are correctly addressed,” Anushita S P Karunakaram, Co-Founder and CEO, LawyerDesk tells YourStory.

The company collaborates closely with a team of over 50 professionals, including developers, designers, and legal experts. Notably, more than 2,000 lawyers joined Advocase within just the first 30 days of its launch to help both legal professionals and students, along with addressing emerging challenges in the field. 

Similarly, OpenNyAI is an initiative focused on leveraging AI to enhance justice in India, with founding collaborators including EkStep, Thoughtworks, and Rohini Nilekani Philanthropies.

It is an open and collaborative mission to develop open-source software and datasets to help create AI-powered justice solutions in India.

Mumbai-based Lexlegis is built specifically for Indian legal practitioners. Using proprietary algorithms and advanced Natural Language Processing (NLP) techniques on Indian legal data—including historical judgments, statutes, and case law—Lexlegis claims to understand the intricacies of India’s legal language and interpretations.

Earlier this year, Sarvam AIalso introduced Sarvam AI Legal, a generative AI platform designed to enhance the capabilities of legal professionals.

Another online legal research platform is Jharkand-based mLeAP, which supports judgments from the Supreme Court of India. The platform allows users to input queries in plain English to analyse and retrieve relevant case judgments. This feature was specifically designed with lawyers in Tier II and III cities in mind, who may be more comfortable with local languages. 

Bengaluru-based jhana.ai, a legaltech startup founded in 2021 at Harvard University, recently raised $1.6 million in seed funding, led by Freshworks CEO Girish Mathrubootham. The company offers a generative AI-led paralegal or legal assistant. 

“At this early stage in Indian legal AI, much of the work to do is necessarily centred around education, helping clients to understand what to make of this technology and how to safely take advantage of its potential. Over the next couple of years, we’ll start to see early adopters of legal AI gaining a competitive advantage in the space. Clients of advocates who adopt these tools can expect to see work products and counsel delivered faster, with a lower rate of errors, just as we’ve already seen in software development and management consulting,” says Benjamin Hoffner-Brodsky, Co-Founder of jhana.ai.

While industry insiders believe GenAI will help lawyers, they are treading carefully to avoid trouble. 

The case of Mata vs Avianca, Inc (2023) is a stark reminder. The lawyers and their firm were imposed a whopping $5000 penalty for citing fabricated case citations, showcasing the importance of due diligence and risks while relying on AI for legal proceedings.