In the 21st century, where technology advances with accelerating momentum, there is no professional field left untouched by digital transformation. The legal sector, which preserved its traditional structure for many years, has not remained outside of this transformation either; integration of artificial intelligence has gained momentum, particularly in data mining, precedent search, and petition drafting processes. However, this rapid adaptation process has brought along previously unencountered legal and ethical discussions. While the inclusion of artificial intelligence tools in legal practice provides time and labor efficiency on the one hand, the risk of presenting erroneous or fictional data produced by these tools before courts has re-opened the lawyer’s professional duty of care to discussion. In this article, the “hallucination” risk arising from the technical operating principles of Large Language Models will be taken under examination; the legal, criminal, and disciplinary liability of the lawyer will be analyzed in light of concrete events in international and national judicial practice.
1-) WHAT IS A LARGE LANGUAGE MODEL (LLM)?
A Large Language Model (LLM) is an advanced artificial intelligence architecture trained on vast amounts of written text data, possessing the ability to understand human language, generate context-appropriate text, summarize, and answer complex questions within a logical sequence using natural language processing techniques.
These Large Language Models form the very foundation of artificial intelligence systems we use today, such as ChatGPT, Gemini, and Claude. However, for these systems to be positioned correctly in legal practice, their technical operating principles must be completely understood:
- Nature of Statistical Prediction Engine: Contrary to the general opinion in society, Large Language Models are not relational databases where verified data is stored, legislation banks, or precedent search engines. Large Language Models convert language patterns and word relationships in the massive data sets presented to them into mathematics, operating with a sequence mechanism that predicts the statistically most probable next word following a word.
- Contextual Fluency in Text Generation: Thanks to its natural language processing capability, the system constructs the logical flow and grammar of the sentence it generates flawlessly. However, this fluency is not a guarantee that the presented information aligns with material reality. The system aims not to “present accurate information”, but to “create the semantically most persuasive and fluent text”. Ignoring this distinction in legal practice constitutes the fundamental reason why lawyers encounter hard-to-compensate errors in judicial processes.
2-) THE “HALLUCINATION” PROBLEM IN ARTIFICIAL INTELLIGENCE SYSTEMS AND ITS LEGAL DIMENSION
The most dangerous phenomenon emerging as a natural and inevitable consequence of the probability-based operating principle of Large Language Models, which have begun to find a place in law offices alongside advancing technology, is the “hallucination” problem.
Hallucination in legal technologies is the situation where the artificial intelligence model constructs and presents court files, precedents, statutory articles, and even academic citations that are not legally valid or existent in the concrete event as if they were real, fabricating their citations and content. The system fills in the blanks with data that appears statistically logical but is entirely a figment of imagination in order to satisfy the user’s request.
The hallucination in question cannot be evaluated merely as a software glitch or a technical error. If these fictional data are served before judicial authorities (courts, prosecutors’ offices, enforcement offices) without being subjected to audit, the matter changes dimension and brings along extremely severe legal, criminal, and disciplinary liabilities.
2.1-) Precedent Example in International Law: Mata v. Avianca Case
The first and most resounding precedent example in the world of the damage created by artificial intelligence hallucination in judicial practice occurred in the “Mata v. Avianca” case heard at the U.S. District Court for the Southern District of New York.
Steven Schwartz, a lawyer with 30 years of professional experience, utilized the artificial intelligence application named ChatGPT while conducting precedent research in a compensation lawsuit filed by his client against an airline company. The system presented precedent decisions to the lawyer that overlapped one-to-one with the subject of the lawsuit. The lawyer asked the artificial intelligence, “Are these decisions real?”, and upon the system’s response, “Yes, they are available in Westlaw and LexisNexis databases”, added the decisions to his petition without verifying them and submitted them to the court.
Court’s Determination and Sanction: In the research conducted by Judge P. Kevin Castel and opposing counsel; it was determined that decisions cited in the petition, such as Varghese v. China Southern Airlines, docket/decision numbers, judge names, and precedent texts were completely fabricated (were hallucinations) by ChatGPT. Attorney Schwartz threw himself on the mercy of the court, stating that he had never used the technology before and had no bad intent.
Indeed, the American Bar Association decided within the framework of the Model Rules of Professional Conduct that the lawyer violated the obligation of competent representation and monitoring technological tools. The lawyer was fined $5,000, the situation was reported to the bar association, and all judges affected by the fabricated decisions were notified of the situation with an official letter.
2.2-) Concrete Reflection in Turkish Judicial Practice: The Ankara Kızılcahamam Incident
The freshest and most striking concrete example of the hallucination problem in Turkish law occurred in a case file at a civil court of first instance in the Kızılcahamam district of Ankara.
In the case file, a senior lawyer resorted to an artificial intelligence model to support his defense petition with precedent decisions. The artificial intelligence application presented texts of Court of Cassation decisions fitting his legal arguments to the lawyer. However, the system generated the chamber numbers, docket/decision years, and citations of these decisions, which appeared legally appropriate in terms of content, in an entirely fictional (fabricated) manner.
Judicial Determination and Initiated Investigations: Upon the opposing counsel’s objection to the precedent citations in the petition and the detailed examination conducted by the court judge over UYAP and the Court of Cassation Knowledge Bank; it was determined that the decisions included in the petition had no systemic or legal counterpart.
Investigation Process: Faced with the attempt to mislead judicial authorities with fabricated precedents, the Kızılcahamam Court reported the relevant lawyer on the same day both to the Ankara Chief Public Prosecutor’s Office (in terms of abuse of duty / attempting to influence a fair trial) and to the Presidency of the Ankara Bar Association (with a request to open a disciplinary investigation).
This case proved that presenting artificial intelligence outputs in petitions without subjecting them to audit explicitly violates the lawyer’s duty of care in terms of Turkish law as well and leads to severe sanctions.
3-) THE LAWYER’S DUTY OF CARE AND LEGAL BOUNDARIES OF PROFESSIONAL LIABILITY
Pursuant to Turkish Legal Legislation, the lawyer’s liability toward their client and the justice mechanism relies on a dual legal foundation: Attorneys’ Law No. 1136 and Turkish Code of Obligations No. 6098.
3.1-) Evaluation in Terms of Article 34 of the Attorneys’ Law
Article 34 of the Attorneys’ Law is mandatory: “Lawyers are obligated to fulfill the duties they undertake with care, truthfulness, and honor fitting the sanctity of this duty, to act in accordance with the respect and trust required by the title of attorney, and to comply with the professional rules set by the Union of Turkish Bar Associations.”
The use of artificial intelligence technologies does not suspend or eliminate this duty of “care and truthfulness” regulated in the law. A lawyer who puts a decision fabricated by artificial intelligence into a petition is deemed to have directly violated the obligations of “not presenting misleading information” to the court and “conducting their work with care”.
ATTORNEYS’ LAW ARTICLE 34:
Lawyers are obligated to fulfill the duties they undertake with care, truthfulness, and honor fitting the sanctity of this duty, to act in accordance with the respect and trust required by the title of attorney, and to comply with the professional rules set by the Union of Turkish Bar Associations.
The responsibility for the data generated by artificial intelligence ultimately belongs to the lawyer who signs under that petition, wears their robe, and represents their client before court. Jurists who combine technology with care will be the ones making a difference in the legal practice of the future.
3.2-) Evaluation in Terms of Article 506 of the Turkish Code of Obligations
The relationship between a lawyer and a client is legally an attorney/agency contract (TBK Art. 502), and pursuant to paragraph 3 of Article 506 of the Turkish Code of Obligations No. 6098, the attorney is obligated to execute the work and services undertaken with care and loyalty by observing the justified interests of the principal. Since the objective standard of care is taken as the basis in the attorney’s liability pursuant to paragraph 2 of the same article, the lawyer’s duty of care is determined according to the standards of professional attention that an attorney expert in a similar field must display. Within this legal framework; in case of presenting fabricated or fictional precedents to judicial authorities, the lawyer seeking refuge in the justification of “the artificial intelligence system’s error” is legally invalid; for the duty of care that an expert lawyer must display necessitates that each precedent decision and statutory provision relied upon in petitions be audited and verified over official sources (UYAP, Official Gazette, Court of Cassation Decision Bank, etc.).
3.3-) Legal Status of Artificial Intelligence: “Trainee / Auxiliary Staff” Analogy
Artificial intelligence systems are not an independent subject or authority of liability in legal practice. In legal terms, artificial intelligence should be evaluated in the status of a kind of “trainee lawyer” or “auxiliary staff” (TBK Art. 116 – Liability for acts of auxiliary persons).
Just as a senior lawyer is personally and primarily responsible for all damages and disciplinary violations that arise when signing and submitting a petition prepared by a trainee or legal researcher in their office to the court without reading it at all and without auditing the precedents; a lawyer using artificial intelligence output without auditing it is equally responsible. The fact that artificial intelligence generates erroneous information does not eliminate the lawyer’s personal duty of care.
4-) THE SOLUTION IS NOT REJECTING TECHNOLOGY: RIGHT INTEGRATION AND SAFE USAGE GUIDE
All these described risks and liabilities do not require us to ignore the massive potential offered by artificial intelligence technology to the legal sector or to reject technology completely. The encountered hallucination cases stem from the rule-defying and unsupervised use of technology rather than technology itself.
On the contrary; an artificial intelligence architecture directed with correct commands, utilizing comprehensive data scanning capability, and most importantly whose output is passed through the lawyer’s professional filter adds an invaluable efficiency to legal practice, from time management to complex contract analysis, from routine documentation processes to strategic brainstorming.
Artificial Intelligence Usage Principles for Lawyers:
- Obligation to Verify Precedents: All precedent decisions, chamber and docket/decision numbers suggested by artificial intelligence must definitely be confirmed over UYAP or official decision banks.
- Confidentiality and KVKK Compliance: Personal data, commercial secrets belonging to the client, or confidential information in the case file must never be entered (as a prompt) into public artificial intelligence models.
- Critical Filter: Legal characterizations presented by artificial intelligence should be accepted not as a final legal opinion, but merely as a preliminary draft.
CONCLUSION
Artificial intelligence is not a threat that will take the place of the lawyer in the legal world, but on the contrary, a powerful “auxiliary element” that increases the lawyer’s capabilities. The issue is not rejecting technology out of fear or surrendering to it blindly; but being able to integrate it into our business processes in a cautious, conscious, and constructive manner within the framework of professional ethics, the duty of care commanded by Article 34 of the Attorneys’ Law, and technical audit mechanisms.