‘Catastrophic, invisible’: Supreme Court flags use of AI in legal process as ‘misconduct’

THe deliberate on the veracity of unreal intelligence agency (AI) has reached the sublime margaret court of India after a National Company Law Tribunal (NCLT) reportedly relied on fake precedents and cases generated using AI.The top court on Thursday set aside an NCLT judgment on Essel Infraprojects insolvency as it found that the tribunal had relied on 'non-existent, fake and hallucinated judgments/precedents generated using AI, news agency PTI reported.The bench comprising Justices PS Narasimha and Alok Aradhe said, "The production of fake, non-existent, and hallucinated material and its utilisation as precedents in law, is like the release of methyl Isocyanate in the province of law and justice: invisible, insidious, and catastrophic by the time anyone notices. It not only contaminates but takes away the very lifeblood of judicial determination."Also read: Air India Express restarts flights to West Asia, Salalah, Kuwait back in serviceThe court held that it amounts to misconduct on the part of the advocate."It is necessary for Courts to adopt a zero-tolerance mode for producing, citing or using AI-generated precedents without verification. It is a misconduct on the part of an advocate to cite such judgments without verification," the Bench was quoted as saying.The bench said it was equally a "serious lapse" if a judge relies on such a fake or hallucinated AI-generated material as precedents in support of the determination."We have no hesitation in declaring that such a decision is no decision in the eyes of the law, irrespective of whether such material had a direct or indirect bearing on the decision-making. Such decisions are to be set aside even if an iota of fake or hallucinated material enters the decision-making process, as it would violate the sanctity of adjudication," it said.Also read: AI, defence, healthcare: What was discussed in Modi's meet with Japan PM Sanae TakaichiThe bench said it is absolutely necessary to maintain integrity in decision-making.The Court also noted that the case underscored wider concerns about the use of AI in legal adjudication and stressed that although AI can assist the process, adjudication must remain under the total and absolute control of human decision-makers.“What is significant for our decision-making is our resolve to adopt artificial intelligence technology in the aid of adjudication, while at the same time asserting, and declaring total and absolute control over adjudications, to the human in the loop, at every stage,” the judgment stated.Also read: What's in a name: Why FSSAI has called out Red Bull, Sting makers for ‘energy drink’ brandingThe bench said mere declaration of prohibitory action is not sufficient and there must be a consequential action following accountability."So far as the responsibility of the bar is concerned, we direct the Bar Council of India (BCI), being the apex statutory body, to constitute a committee and deliberate on this issue of members of the bar submitting such fake and hallucinated material before the Court as if they are precedents of law," it said.The verdict said the apex bar body must take up this issue with utmost seriousness, deliberate earnestly, and prescribe a guiding principle to prevent such occurrences, along with the disciplinary action that will follow a violation of the norms.(With PTI inputs)