Lawyer’s AI reliance results in fictitious court citations in Supreme Court petition
The petition, based on fabricated legal precedents, was dismissed after being flagged by the court.
In a petition filed with the SuIn a petition filed with Israel’s Supreme Court, a lawyer who relied on artificial intelligence tools referred the court to rulings "that did not exist." The Supreme Court's ruling indicates that the legal argument presented in the petition was based primarily on non-existent rulings and quotes from rulings that did not exist. This resulted from the lawyer’s reliance on information provided by "a website recommended by colleagues," information that she was misled into thinking was accurate without verifying its truthfulness.
Supreme Court Justice Gila Canfy-Steinitz noted in her ruling that "although the petitioner's attorney did not specify the unknown 'website' she relied on blindly, the references to non-existent judgments, inconsistencies in the internal logic, and citations that did not exist indicate with a high degree of probability that the petitioner's attorney used a website based on artificial intelligence."
The judge dismissed the petition after considering its merits, in order not to disadvantage the woman, and ruled that "it seems that the hallucination output she received (referring to the lawyer) was perceived by her as so credible that she did not bother to check the veracity of its content. It is worth noting that this is not the first case in which court documents containing false legal references related to artificial intelligence have been submitted to the courts."
The Supreme Court’s ruling dealt with a petition filed by a woman challenging the decision of the Sharia Court of Appeals. The woman and her ex-husband are members of the Muslim community. After their relationship faltered, they consulted a lawyer together and, following his advice, drafted a short divorce agreement, which they eventually signed. The focus of the proceedings was the woman's request to cancel a clause in the divorce agreement that was approved by the Sharia Court. The woman claimed she had learned that she had signed away all her civil law rights and that the waiver clause was inserted into the agreement after the couple's terms had been established, without her knowledge or consent. As stated, the judge considered the petition to avoid disadvantaging the woman despite her lawyer's conduct, and ruled that "the petition should be dismissed outright, in the absence of grounds for interfering with the rulings of the Sharia Courts."
The judge also addressed artificial intelligence in the ruling, stating, "At this time, it should be emphasized that lawyers' use of artificial intelligence tools does not absolve them from their responsibility or professional judgment. Lawyers must use this technology responsibly, carefully, and critically, understanding its capabilities and limitations, and staying informed about its strengths and weaknesses. In particular, they are not permitted to submit documents to the court without thoroughly examining their accuracy, and they must avoid submitting erroneous or misleading documents while blindly relying on these tools."
The judge noted that while the potential benefits of artificial intelligence tools for human development are speculative, "like any technology, it also carries drawbacks and risks, some of which present new challenges to the legal world. One of these challenges is the fear of information leakage into an artificial intelligence system... Another challenge, which has already surfaced in trial courts, concerns the admissibility of evidence generated by an artificial intelligence system, given the difficulty in establishing its reliability and the inability to subject its creators to cross-examination... Another challenge, relevant to this case, is the phenomenon of artificial intelligence hallucinations. This term, borrowed from psychology, describes a situation where an artificial intelligence system 'hallucinates'—creating content with false or misleading information that is presented as fact."
The judge also ruled that "a lawyer who submits a court document containing fictitious content, while relying on non-existent references, is in breach of their duties to the party they represent, to the court, to the opposing party, and to the legal profession."
The judge concluded that the judicial toolbox contains appropriate legal tools for dealing with misleading court documents. However, in this case, she ruled, "Since this is the first case of its kind brought before this court, and well beyond the scope of existing law, I have not found it necessary to use any of these tools. However, it is appropriate for courts to exercise their powers in similar future cases, under the right conditions, to maintain the integrity and soundness of the legal process. This is not a right, but an obligation."
In conclusion, the judge stated, "It should be clarified—the seriousness of the act before us, and the concerns it raises, do not overshadow the appropriate uses of artificial intelligence tools. We should not throw the baby out with the bathwater. Artificial intelligence tools hold great potential for improving legal work, increasing access to the courts, making the law more accessible, deepening legal research, improving pleadings, streamlining proceedings, shortening the duration of litigation, achieving compromises, and reducing the burden on the judicial system. It would be unwise to miss these benefits due to understandable concerns."
Attorney Ariel Dubinsky, an expert in intellectual property law and artificial intelligence who did not represent anyone in the case, explained that "the ruling marks an important milestone in the legal system's handling of the challenges posed by artificial intelligence. The Supreme Court emphasizes that the use of advanced technological tools does not exempt lawyers from their professional responsibility and the increased duty of care—both toward the court and their clients. While artificial intelligence can assist in legal work, it is no substitute for professional judgment and the thorough examination of every claim submitted to the court. Lawyers are advised to adopt a cautious and controlled approach in using these tools, ensuring verification and quality control of the products they obtain."
