Is It Safe to Use AI for Legal Advice?

Using ChatGPT, Claude or Gemini for legal advice is not “safe” in the way a layperson means it. Two distinct risks stack up. First, reliability: a general-purpose chatbot can invent law — decisions, docket numbers and citations that don't exist. Second, confidentiality: there is no attorney-client privilege when you describe your problem to a consumer chatbot; what you say can be retained, reviewed, and even demanded in court. AI can help rough out a general question, but it doesn't replace a lawyer and creates no privilege. The only certain fix: anonymize the facts and identities before describing your situation.

Risk #1: AI can invent law

A language model produces plausible text, not verified truth. In legal matters, that shows up as “hallucinations”: court decisions, docket numbers and case excerpts that are entirely fabricated, presented with full confidence. The textbook case is Mata v. Avianca, Inc., before the federal court for the Southern District of New York: a lawyer filed a brief relying on six non-existent decisions generated by ChatGPT, including rulings attributed to real airlines that were never actually issued.

The most instructive detail: before filing, the lawyer asked the chatbot whether the cases were “real” — and the tool insisted they genuinely existed. On June 22, 2023, the judge sanctioned both lawyers and their firm with a fine, finding they had failed their duty to verify. If legal professionals get fooled, a non-lawyer has no way to spot the invented citation.

Risk #2: no privilege with a chatbot

When you talk to a lawyer, your exchanges are covered by attorney-client privilege: they can't be produced against you. There's nothing of the kind with a consumer chatbot. OpenAI CEO Sam Altman acknowledged this publicly in July 2025: with a therapist, a lawyer or a doctor there's legal privilege, but “we haven't figured that out yet” for conversations with ChatGPT. As things stand, he said, if there's a lawsuit OpenAI could be legally compelled to produce those exchanges — there is “no legal confidentiality” for users' conversations.

A federal court confirmed this in practice. In United States v. Heppner (Southern District of New York, ruling of February 10, 2026), the judge held that documents from a conversation with an AI — here a litigant using the consumer version of Claude for legal research — are neither covered by attorney-client privilege nor protected as attorney work-product. The reasoning is clear: an AI tool is not a lawyer. It holds no license to practice, owes no duty of loyalty, forms no attorney-client relationship and is bound by no professional rules.

Worse: these platforms' privacy policies allow data collection, use for training the models, and disclosures to authorities. So there is no “reasonable expectation of confidentiality.” And forwarding your exchange to a lawyer after the fact does not make it confidential — a document that wasn't privileged when it was created can't become so retroactively.

What can be demanded in court

In practice, legal advice from a consumer chatbot is discoverable: in litigation, logs containing your prompts and the model's responses can be demanded by the opposing party as long as they're non-privileged, relevant and proportionate to the case. What you describe — a contract, a dispute, personal facts — becomes a potentially demandable exhibit. Lawyers' guidance for sensitive matters is blunt:

• Avoid consumer AI tools for discussing a genuinely sensitive matter.
• Reserve the confidential discussion for a real lawyer — the only one offering privilege.
• Never share an already-privileged communication with a third-party platform: it can waive the original privilege.

The fix: anonymize before describing your situation

Both risks resolve in the same place: the content of your prompt. An AI can genuinely help you understand a legal concept, prepare the right questions or rough out a general situation — as long as you don't pour identifiable data into it. Before describing your case, strip out what exposes you:

1. 1Replace names, addresses and contact details with neutral tokens.
2. 2Mask the references that tie the matter to you: contract, case and account numbers.
3. 3Neutralize precise amounts and dates that make the facts recognizable.
4. 4Frame your question in general terms, then verify any cited legal reference against an official source or a lawyer.

Stay honest about the limit: once the question is roughed out, real advice — binding, confidential, enforceable — goes through a lawyer. AI sets the stage; it doesn't replace it.

That's exactly what ONYRI Sanitize is for: the engine spots names, addresses, case numbers and amounts, and replaces them with reversible tokens before sending. Detection and the token↔value mapping stay in your browser; only anonymized text reaches the AI. Whether the conversation is retained, reviewed or demanded in court, it contains only tokens — not your real file.