AI data leakage

What is AI data leakage?

AI data leakage is personal or confidential information ending up in an AI tool, or coming out of one, without the organisation having agreed to it: a pasted customer email, an uploaded file, an assistant connected to a mailbox or drive. From then on the data is stored or processed outside your control.

How it differs from a classic data breach. A classic breach usually starts with an attacker or a slip: a hack, a lost laptop, an email to the wrong address. An AI data leak usually starts with someone doing their job, and the recipient is a service the organisation never assessed. The legal outcome can be the same. If personal data reaches a party that should not have it, the GDPR calls it a personal data breach.

How it differs from shadow AI. Shadow AI is AI use the organisation has not decided on yet. A lot of AI data leakage happens there, but the two are not the same. An approved Copilot can surface a file that was shared too widely, and an unapproved tool used without personal or confidential data leaks nothing.

One more distinction, because search results mix them up. In machine learning, "data leakage" means information from the test data slipping into the training data, so a model looks better in testing than it is. That is a modelling error, not a privacy problem, and not what this page is about.

How data leaks into AI tools

Data reaches AI tools along six routes. Only in the first does someone put it in themselves. The others run through settings, permissions and features nobody looks at any more, and those apply to every chat and every user at once.

AI data leakage examples: real incidents

Each of these cases was documented by the organisation involved, a regulator or a reputable news outlet, and the source is listed with it.

At the end of 2025 the AP told the Dutch financial daily FD that it had received dozens of notifications that year of incidents in which sensitive information was shared through public AI tools (Binnenlands Bestuur, 30 December 2025). Notifications to the AP are not public, so the cases above are only the ones that came out.

Is it a data breach? Consequences and reporting under the GDPR

The AP draws a clear line. When employees use a chatbot on their own initiative and against the employer's agreements, and enter personal data, that is a data breach. When chatbot use is part of the organisation's policy, it is not a breach, but often not lawful either. Organisations have to prevent both. If it does go wrong, reporting to the AP and to the people affected is "in many cases mandatory" (AP, 6 August 2024).

The rules themselves are in Articles 33 and 34 of the GDPR:

• Notify the regulator within 72 hours of becoming aware of the breach, unless it is unlikely to result in a risk to the people involved (Article 33). If you do not have all the facts yet, you can provide them in phases.
• Inform the people affected without undue delay when the risk to them is likely to be high, for example with medical data or data about children (Article 34).
• Record every breach in your internal breach register, including the ones you decide not to report, with the facts, the effects and what you did about it (AP).

Whether a particular leak has to be reported depends on the kind of data, how much of it, how easily people can be identified and who received it. The AP explains that weighing on Datalek: wel of niet melden (in Dutch) and advises reporting when in doubt. Failing to report a breach that should have been reported can lead to a fine.

With AI tools, the extent is the hardest part to establish. You often cannot see what was entered, how long the provider keeps it or who could read it. Eindhoven could not establish how many files had gone out, and asked OpenAI to delete them; in June 2026 it told Nieuwsuur that OpenAI had not confirmed this (NOS via Omroep Brabant). Record that uncertainty in the notification and the register, and do not wait for certainty you may never get: the 72 hours start when you become aware of the breach.

If you are the privacy officer who has to make this call, see also AI use and personal data in view for the DPO.

How to tell whether data has already leaked

Usually nobody reports it. Someone pasted a customer file months ago and has long forgotten. There are four places to look.

• Network traffic. Proxy, DNS or firewall logs show which AI services are visited from work devices, and how often. Eindhoven found its leak through a review of outgoing data traffic and a sample.
• Business accounts you manage. Copilot interactions can be searched with Microsoft Purview, Gemini chats on Workspace accounts with Google Vault, and ChatGPT Enterprise offers a Compliance API. What an admin can see differs per tool and plan; can admins read AI chats? sets it out.
• Shared links, extensions and connections. Look for chats shared through a public link, browser extensions with access to every site, and assistants connected to mail or drives. They keep passing data on, even when nobody remembers who set them up.
• People. Personal accounts are out of your reach: you cannot see what a colleague typed into their own ChatGPT. Ask, and make it safe to answer honestly.

Expect gaps. Providers delete data after their retention period, and an old conversation may no longer exist anywhere you can reach. Write down what you checked and what you could not. If it turns out to be a breach, that record belongs in the register.

How to prevent AI data leakage without banning AI

A ban moves the problem to phones and personal accounts, where you see nothing at all (why blocking AI creates shadow AI). What works is a set of ordinary measures, each aimed at one of the routes above.

1. Find out which AI tools are in use, including personal accounts, browser extensions and AI features inside tools you already have.
2. Decide per tool: allowed, allowed with conditions, or not allowed, and record why in an AI register.
3. Offer a business account for the work people already do with AI. Business plans do not train on your data by default and come with a data processing agreement (free vs business accounts).
4. Write down what never goes into a prompt: client and patient data, national ID numbers, credentials and anything under professional secrecy. One page is enough (AI policy example).
5. Give priority to the teams that handle the most real data: support, HR, finance and legal work with customer and employee details every day.
6. Clean up sharing permissions before switching on Copilot or any other assistant that searches your files (Copilot and SharePoint).
7. Give agents and MCP connections their own account, with only the permissions the task needs and tokens that expire (why an agent needs permissions).
8. Mask or remove personal data before it goes into a prompt, for example by replacing a name with [NAME1]. The prompt redaction guide shows three ways.
9. Agree what happens when it goes wrong: who assesses a leak, who reports it within 72 hours and where it is registered. Tell people that a quick report is worth more than a perfect one.

BeeSensible helps with the first step and with masking. It shows which AI tools are used in the organisation, and sensitive details are highlighted in the text while someone types in an AI tool, email or chat, so they can mask or remove them before sending. The organisation can also switch on automatic masking on paste or send. Detection runs in working memory and the checked text is deleted straight away. How that works.