GeminiA support agent at a logistics company opens Gemini to draft a reply to an angry customer. She pastes the full complaint thread - name, address, order history, and the customer's bank details from a refund request - and asks Gemini to "write a calm, professional response." The draft is excellent. She never notices the small line of text beneath the prompt box: Gemini can make mistakes, so double-check it. Your conversations may be reviewed.
She is not using her company's Google Workspace. She is signed into her personal Google account, in the free consumer app, where a subset of conversations is read by human reviewers and reviewed chats can be kept for up to three years.
This is the part of Gemini that catches organisations out. The risk is not a single careless prompt. It is that Gemini does not feel like a separate tool you visit - it sits inside Gmail, Docs, Drive, and the phone in your pocket - and most people have no idea which version of it they are actually talking to.
Is Gemini safe for business data?
Gemini can be used safely at work, but the answer depends almost entirely on which Gemini. Google ships the brand under two very different products with two very different sets of promises.
The free consumer app at gemini.google.com is built around product improvement: Google samples conversations for human review and uses them to train its models, and it tells you so. Gemini in Google Workspace is built around enterprise commitments: Google states in writing that it does not use your Gmail, Docs, or Drive content to train its generative models and that your data is not reviewed by humans outside your domain without permission.
Same name. Same interface, more or less. Completely different exposure. The privacy risk in Gemini is rarely the model being "unsafe" - it is an employee placing customer or employee data into the consumer app while believing they are protected by their employer's contract with Google.
How Gemini handles your data
| Consumer Gemini (gemini.google.com) | Gemini in Google Workspace | |
|---|---|---|
| Account | Personal Google account | Managed work account |
| Used to train Google's models | Yes (a subset, with human review) | No - contractually excluded |
| Human review of conversations | Yes, a subset | No, without permission |
| Retention after you delete | Reviewed chats kept up to 3 years | Controlled by your administrator |
| Data processing agreement | No | Yes (Workspace terms) |
| Admin control | None | Full, via Admin console |
Consumer Gemini displays Google's own warning: "Please don't enter confidential information that you wouldn't want a reviewer to see or Google to use to improve our services, products, and machine-learning technologies." That is not a footnote - it is Google telling you, directly, what the service is for. A subset of chats is reviewed by trained reviewers, and those reviewed chats (plus related signals such as language, device type, and location) are retained for up to three years, even if you delete your activity. Turning off Gemini Apps Activity stops new chats being sampled; otherwise chats are kept for 72 hours to operate the service.
Gemini in Workspace is the version an organisation can actually govern. Google commits that customer data is not used to train or improve its generative models without permission, is not reviewed by humans outside your domain, and is not used for advertising. Data residency can be restricted to the EU, and client-side encryption is available. Crucially, an administrator decides whether Gemini is on at all, what it can reach, and how long conversation history is kept.
The trouble is that nothing on the screen makes the difference obvious to the person typing.
The biggest privacy risks in Gemini
1. Employees using the consumer app instead of the managed one
This is the headline risk. An employee signs into their personal Google account - on a work laptop, or on their phone - and uses the free Gemini. Anything they paste is governed by the consumer terms: sampled for review, used for improvement, retained. The company's Workspace protections do not apply, because the company was never in the loop.
2. Gemini reaching into Gmail, Docs, and Drive
The feature that makes Gemini in Workspace useful - "summarise this thread," "find the contract in my Drive," "draft from my notes" - is also a wide data surface. When Gemini is enabled with access to Workspace content, a single question can pull together information from across your mailbox and files. That is powerful and, under Workspace terms, contained. But it means access scope is now an admin decision, not a per-document one.
3. Acting on your phone apps - even with activity tracking off
In July 2025, Google emailed Android users that Gemini would be able to "help you use Phone, Messages, WhatsApp, and Utilities" whether Gemini Apps Activity is on or off. The change decoupled app access from the data-logging toggle: turning off activity no longer stopped Gemini from interacting with those apps. It was switched on by default and announced by an easily-missed email. Gemini does not train on these interactions while activity is off - but the access itself is the point, and many users never realised it had changed.
4. Prompt injection through summarised content
In July 2025, a researcher at Mozilla's 0DIN programme demonstrated that hidden text in an email - white, zero-size, wrapped in a fake <Admin> tag - could hijack Gemini's "summarise this email" feature. When the recipient asked for a summary, Gemini followed the hidden instruction and appended a fake security warning with an attacker's phone number. No link, no attachment. Google says it has seen no real-world abuse and is hardening its defences, but the technique turns any incoming newsletter, ticket, or CRM message into a potential injection vector.
5. Confidential data in a prompt that did not need it
Even in a fully governed Workspace, the safest data is the data you never paste. Summarising a performance review does not require the employee's date of birth and phone number to be in the prompt. The model will happily accept them - and the more identifiable data flows through prompts, the larger the footprint that any future misconfiguration, export, or access review has to cover.
Each of these has caused real concern in organisations that assumed "we use Google Workspace" was the whole answer.
What is actually at stake: consequences
Under GDPR, putting identifiable personal data - a name with an account number, health information, a customer's financial details - into an AI service is a processing event. If that processing happens in the consumer Gemini app, there is no data processing agreement behind it, no controlled retention, and a documented possibility of human review. For a regulated organisation, that is difficult to defend.
Fines under GDPR can reach €20 million or 4% of global annual turnover, whichever is higher, and the liability rests with the organisation as controller. The supervisory authority must be notified within 72 hours of becoming aware of a breach that poses a risk to people's rights and freedoms.
Regulators have already shown they are watching Google's AI data practices closely. Beyond fines, there is the quieter cost: a customer or employee learning that their sensitive details were typed into a consumer chatbot is a trust problem no settings page can repair.
Verified incidents
July 2025 - Gemini gains access to apps with activity off
Google notified Android users that, from 7 July 2025, Gemini could interact with Phone, Messages, WhatsApp, and Utilities even when Gemini Apps Activity was turned off. The change was on by default; users had to opt out per app. It was widely covered as a meaningful shift in how much of the phone Gemini could touch by default. Sources: Android Authority, June 2025; Malwarebytes, July 2025.
July 2025 - "Phishing for Gemini" prompt injection
Marco Figueroa, of Mozilla's 0DIN GenAI bug bounty programme, disclosed that hidden HTML in an email could make Gemini's email-summary feature output an attacker-controlled phishing message. Google stated it had observed no in-the-wild exploitation and that mitigations were being deployed. Sources: 0DIN, 10 July 2025; BleepingComputer, 13 July 2025.
September 2024 - Irish DPC inquiry into Google's AI data
Ireland's Data Protection Commission opened a statutory inquiry into whether Google should have completed a data protection impact assessment before processing EU personal data to develop its foundational AI model. The inquiry concerned the model underlying Google's AI development, and signalled that EU regulators expect GDPR diligence before, not after, AI training. Source: Silicon Republic, September 2024.
September 2025 - CNIL €325m fine over Gmail
France's data protection authority fined Google €325 million, in part for inserting ads between emails in Gmail without valid consent - a practice tied to the "smart features" personalisation setting. While not a Gemini case, it shows how Google's in-product data use and consent design remain under active regulatory scrutiny. Source: CNIL, 1 September 2025.
The common thread is not one broken feature. It is the gap between how protected people assume Google's AI is, and how the specific product in front of them actually behaves.
Settings that help
1. Standardise on Gemini in Workspace, and disable the consumer app where you can The single most effective control is ensuring employees use the managed, governed version. In the Google Admin console, under Generative AI, administrators decide whether Gemini is on, what Workspace content it can reach, and how long conversation history is retained.
2. Turn off Gemini Apps Activity (consumer accounts) On a personal account, go to gemini.google.com, open Activity (or Settings → Gemini Apps Activity), and choose Turn off - or Turn off and delete. This stops new chats from being saved and sampled for human review.
3. Review Gemini's app connections on Android In the Gemini app, open your profile → Apps (or Extensions) and turn off connections you do not want, such as Messages, WhatsApp, Phone, and Utilities.
4. Set an auto-delete window Consumer Gemini lets you auto-delete activity after 3, 18, or 36 months. Choose the shortest period that works, and remember it does not reach chats already pulled for human review.
5. Restrict data region in Workspace For organisations with EU data-residency requirements, Workspace can restrict Gemini processing to the EU, with client-side encryption available for stricter key control.
Steps verified in June 2026; Google has been relabelling "Gemini Apps Activity" as "Keep Activity," so menu wording may differ.
What settings do not solve
Settings decide what Google may do with the data. They do not decide what an employee types, or which Gemini they type it into.
No admin policy reaches the moment a support agent, on her own phone, pastes a customer's bank details into the consumer app. No data-residency setting un-sends a prompt that contained a date of birth that never needed to be there. No retention control helps if the sensitive value was avoidable in the first place.
The most common Gemini privacy failures are not model failures or configuration failures. They are ordinary moments - a quick summary, a fast draft - where someone places more personal data into a prompt than the task required, in whichever Gemini happened to be open.
How BeeSensible helps before you send
BeeSensible checks personal data in text fields - including the Gemini prompt box - as you type. Through the desktop app, detection runs entirely on the device and no text leaves the machine. For browser-only use, the browser extension sends the text to BeeSensible's EU detection service, where analysis runs in working memory and the text is discarded after detection. When sensitive content appears in a prompt, BeeSensible marks it inline and shows a panel listing what it found and how severe it is. The user can delete it, replace it with a placeholder, or mask the value before sending.
