SC: AI-generated precedents catastrophic for justice system

SUpreme margaret margaret court Addresses AI-Generated PrecedentsNew new delhi: The sublime Court on th said fake, non-existent and AI-generated precedents pose the same threat to the justice system that “methyl isocyanate” poses to human life— “invisible, insidious, and catastrophic by the time anyone notices.”The court was referring to a National Company Law Tribunal judgement that cited several hallucinated precedents.Such material, the apex court added, contaminates the judicial process and strips judicial determination of “its very lifeblood.”Warning against the use of artificial intelligence in court proceedings, a bench of Justices PS Narasimha and Alok Aradhe set aside the judgments of NCLT  and the National Company Law Appellate Tribunal (NCLAT) in an insolvency dispute after finding that the NCLT had relied on non-existent, fake and hallucinated precedents generated by AI tools and that the NCLAT had failed to detect the error while upholding NCLT’s order.Proceedings in the court strongly suggested that NCLT may have relied on AI tools to write its judgement, although the bench chose to not dwell on this.The bench’s reference to methyl isocyanate harks back to the Bhopal gas tragedy in 1984 that killed thousands in the world’s deadliest industrial accident.The bench further said that all courts must adopt a “zero tolerance” policy for citing or relying on “ fake, non-existent, and hallucinated material,” generated through AI.It directed the Bar Council of India (BCI) to constitute a committee to examine the issue of lawyers placing fake or hallucinated AI-generated material before courts as though it were genuine precedent. It asked the BCI to frame guiding principles and prescribe disciplinary consequences for violations.“For us, i.e., for those in the province of adjudication and determination of disputes, this by-product of AI, i.e., the production of fake, non-existent, and hallucinated material and its utilisation as precedents in law, is like the release of methyl isocyanate in the province of law and justice: invisible, insidious, and catastrophic by the time anyone notices,” the Supreme Court said.The Supreme Court also said that any advocate who cites hallucinated judgments, commits professional misconduct, and that judges, who rely on such judgements or precedents, commit a serious lapse in doing so.Any decision tainted by even “an iota” of such material, the court said, is “no decision in the eyes of law” and must fall, regardless of whether or not the fake precedent influenced the final outcome.The Court was hearing an appeal filed by the suspended director of a private company, challenging the orders of NCLT admitting the firm to the corporate insolvency resolution process on a Section 7 application filed under the Insolvency and Bankruptcy Code by the Jammu and Kashmir Bank.NCLT Mumbai had, on August 28, 2024, admitted the insolvency plea and on September, 2025, NCLAT had upheld the order.Senior advocate Madhavi Divan, who appeared for the appellant, however, told the Supreme Court that several citations or judgements relied upon by NCLT either did not exist or contained paragraphs that no law report recorded.The court then conducted its own examination and found multiple errors. Some citations referred to judgments that never existed. Others cited genuine cases but attributed fabricated passages to them. One citation even carried the name of an entirely different Supreme Court decision.The bank told the court that its counsel had not cited those and that the adjudicating authority had perhaps cited them through its own research.The court noted that the fake precedents escaped scrutiny even before NCLAT. It also said that courts cannot realistically verify the authenticity of every judgment cited by advocates and that the justice system depends on lawyers placing genuine authorities before the Bench.It said that while technology had always aided the dispensation of justice, AI was different because it can “replace human thinking, reasoning and decision-making.”The ability to distinguish truth from falsehood, right from wrong, justice from injustice and “dharma and adharma” develops through discipline and lived experience and abandoning the exercise of independent thought would erode the very foundation of human judgment, the court said.It added that courts alone cannot resolve the issue and that “absolute and total control” over AI will require public policy and enforceable rules. The real safeguard, however, would lie in the willingness of judges and lawyers to use the technology with “restraint and verification,” it said.