Control over AI
Blog
AI data leakage 12 min read

Is Gemini safe for work data? It depends on the account

Gemini on a personal account and Gemini in Google Workspace follow different rules. What Google keeps and reviews, what your admin sees in Vault and what to set.

Gemini
Can you draft a polite reply to this customer?
Of course. Paste the details below and I'll turn them into a courteous reply.
BeeSensible puts its highlights right in the text.

The example above is interactive. Click a highlighted value to see what you can do with it.

Quick answer

It depends on the account. On a managed Google Workspace account, Google does not use your prompts to train its models or have them reviewed outside your organisation without permission. On a personal account, chats can be used for training while Keep Activity is on, and chats a reviewer has read are kept for up to three years. With a work account, your admin can search and export Gemini app chats in Google Vault, including deleted chats under a hold. A personal account is beyond that reach.

01

Gemini on a personal account and Gemini in Google Workspace fall under different data terms

02

With Keep Activity on, personal chats can be used to train Google's models, and chats a reviewer has read are kept for up to three years

03

Turning Keep Activity off stops training, but Google still keeps chats for 72 hours and can have them reviewed for safety

04

On a work account, admins can retain, hold, search and export Gemini app chats in Google Vault, including chats the user deleted

05

On Android, Gemini can use Phone, Messages and WhatsApp even when Keep Activity is off

A support agent at a logistics company opens Gemini to draft a reply to an angry customer. She pastes the full complaint thread (name, address, order history and the customer's bank details from a refund request) and asks Gemini to "write a calm, professional response". The draft is excellent.

She is not using her company's Google Workspace. She is signed in to her personal Google account, in the free consumer app, where a subset of conversations is read by human reviewers and reviewed chats can be kept for up to three years. Her employer has no view of that chat; Google's consumer terms decide what happens to it.

This is the part of Gemini that catches organisations out. Gemini does not feel like a separate tool you visit. It sits inside Gmail, Docs, Drive, Chrome and the phone in your pocket, and most people have no idea which version of it they are actually talking to.

Which Gemini are your employees using?

Google uses the Gemini name for two products with very different promises, and the account decides which one applies. A personal Google account gets the consumer terms, built around improving Google's products. A managed work or school account gets the organisation's Workspace agreement. The name and the interface are nearly the same; what happens to the data is not.

So the privacy risk in Gemini is rarely the model being "unsafe". It is an employee putting customer or employee data into the consumer app while believing they are protected by their employer's contract with Google.

How Gemini handles your data

Consumer Gemini (gemini.google.com)Gemini in Google Workspace
AccountPersonal Google accountManaged work or school account
Used to train Google's modelsYes, while Keep Activity is onNo, not outside your domain without permission
Human review of conversationsYes, a subset; review for safety continues with Keep Activity offNo, not outside your domain without permission
Retention18 months by default; reviewed chats up to 3 years, even after you delete themSet by your admin; Vault can keep chats users delete
Can your employer see it?NoYes, with Google Vault
Data processing agreementNoYes (Cloud Data Processing Addendum)
Admin controlNoneAdmin console

Consumer Gemini carries Google's own warning: "Please don't enter confidential information that you wouldn't want a reviewer to see or Google to use to improve our services, including machine-learning technologies." Keep Activity (formerly Gemini Apps Activity) is on by default for anyone 18 or over. While it is on, Google uses your chats to improve its services, including training its models, and a subset is read by trained reviewers. Reviewed chats, plus related data such as language, device type and location, are kept for up to three years, even if you delete your activity. Turning Keep Activity off, or using a temporary chat, stops future chats from being used for training, as long as you do not send feedback. It does not stop review altogether: Google keeps those chats for 72 hours and can still have them reviewed to protect its users and services. All of this is in the Gemini Apps Privacy Hub.

Gemini in Workspace is the version an organisation can actually govern. The Generative AI in Google Workspace Privacy Hub states that your content is not reviewed by humans or used for model training outside your domain without permission, and that the Gemini app on a work account falls under the Workspace agreement and the Cloud Data Processing Addendum. An administrator decides whether Gemini is on at all, whether it can use content from Gmail, Drive and other Workspace apps, and how long conversations are kept.

Three parts of Gemini follow rules of their own:

  • Gemini Notebook (formerly NotebookLM). In Workspace, uploads, questions and answers are not reviewed by humans or used for training, and prompts and answers are not kept after the session. Files you add from Drive become copies stored outside Drive, and your organisation's sharing and data region settings do not apply to them. On a personal account, chats with your notebooks follow your Keep Activity setting, human review included.
  • Gemini in Chrome reads the page content and address of the tab you share, plus any other tabs you add, and some of that content may not be visible to you. On a personal account, page content is logged to your Google account temporarily. In Workspace it is not reviewed or used for training outside your domain, but Google states that it does not yet fall under its ISO and SOC certifications. For now, Workspace users only get it when they are signed in to Chrome in the US with US English as the language.
  • Gems are saved instructions, sometimes with files, that people reuse and share. Google Vault does not retain or export a Gem attached to a prompt. Google has announced that skills will replace Gems in 2027.

Can your employer see your Gemini chats?

On a work or school account, yes, if your organisation has set it up. Google Vault, the eDiscovery tool in Workspace, can retain, hold, search and export the prompts you type into the Gemini app and the answers it gives. A hold or retention rule takes precedence over the admin's history setting and over your own deletions: a chat you delete disappears from your screen but stays searchable in Vault. Vault is included in Business Plus, in Enterprise and Frontline (both Standard and Plus) and in every Education edition. Business Starter and Business Standard do not include it.

Without Vault, the admin still decides how long Gemini app conversations are kept: 3, 18 or 36 months, with 18 as the default, or not at all, in which case Google holds chats for up to 72 hours. Conversations in the Gemini side panel of Gmail, Docs and Drive have their own retention setting, from 90 days to indefinitely. Audit logs show who used Gemini, when, in which app and for what kind of task, such as a conversation or a summary.

Vault does not reach everything. According to Google's list of supported data, it does not retain, hold, search or export Gems and media files attached to a prompt in the Gemini app, or media Gemini generates in its answers there.

Gemini conversations inside Workspace apps, such as the side panel in Gmail, Docs and Drive, used to be out of Vault's reach altogether. On 22 October 2026 Google announced that admins can now export them from Vault, filtered by user, organisational unit and date range, across Gmail, Drive, Docs, Sheets, Slides and Meet. This covers conversations held while conversation history was on. Retention rules and holds do not apply to them yet, so how long they last is set by the separate side panel retention setting. Checked 23 October 2026.

A personal Google account is outside all of this. Vault, audit logs and the history settings only reach accounts your organisation manages. That does not make a personal account private: the consumer terms apply, human review included. Our article on whether your admin can read your AI chats compares this with ChatGPT and Copilot and explains what the GDPR asks of an employer before looking.

Five places where it goes wrong

1. Employees using the consumer app instead of the managed one. This is the main risk, and exactly what happened to the support agent above: she worked in a personal Google account, so everything she pasted fell under the consumer terms and none of the Workspace protections applied. The same split runs through ChatGPT and Claude; our comparison of free and business AI accounts puts them side by side.

2. Gemini reaching into Gmail, Docs and Drive. The feature that makes Gemini in Workspace useful ("summarise this thread", "find the contract in my Drive") also gives it a wide reach into company data. Gemini only retrieves content the user already has access to, but one question can pull together information from across a mailbox and a Drive, including files that were shared more widely than anyone remembers. Under Workspace terms that stays inside the organisation. It does mean sharing settings carry more weight: a file shared too widely can turn up in the answer of anyone who can open it.

3. Phone apps that stay connected with activity off. In June 2025 Google emailed Android users to say that from 7 July Gemini would help them use Phone, Messages, WhatsApp and Utilities whether Gemini Apps Activity was on or off. The access was on by default; users who did not want it had to disconnect each app themselves. Google's privacy hub still lists Phone, Messages, WhatsApp and Device assistance as available on Android whether Keep Activity is on or off. Gemini does not use those chats for training while the setting is off, but the access remains, and many users never noticed the change. Sources: Android Authority, 26 June 2025; Malwarebytes, July 2025.

4. Hidden instructions in an email Gemini summarises. In July 2025, a researcher reported to 0DIN, Mozilla's bug bounty programme for generative AI, that hidden text in an email (white, set to zero font size, wrapped in a fake <Admin> tag) could hijack Gemini's "summarise this email" feature in Gmail. When the recipient asked for a summary, Gemini followed the hidden instruction and added a fake security warning with an attacker's phone number. No link, no attachment. Google said it had seen no abuse in the wild and was rolling out countermeasures, but any incoming newsletter, ticket or CRM message can carry the same kind of hidden instruction. Sources: 0DIN, 10 July 2025; BleepingComputer, 13 July 2025.

5. Personal data in a prompt that did not need it. Even in a fully managed Workspace, the safest data is the data you never paste. To summarise a performance review, the prompt does not need the employee's date of birth and phone number. The model will happily accept them, and the more identifiable data flows through prompts, the more any future misconfiguration, export or access review has to cover.

What it means under the GDPR

Putting identifiable personal data (a name with an account number, health information, a customer's financial details) into an AI service is processing under the GDPR. If that happens in the consumer Gemini app, there is no data processing agreement behind it, no retention the organisation controls and a documented possibility of human review. For a regulated organisation, that is difficult to defend. Fines can reach €20 million or 4% of global annual turnover, whichever is higher, and the liability rests with the organisation as controller. A breach that poses a risk to people's rights and freedoms has to be reported to the data protection regulator within 72 hours of the organisation becoming aware of it.

Regulators are already looking at how Google handles data in and around its AI. In September 2024, Ireland's Data Protection Commission opened an inquiry into whether Google should have carried out a data protection impact assessment before using EU personal data to develop its PaLM 2 model; by October 2026 it had not published a decision (DPC, 12 September 2024). In September 2025, France's CNIL fined Google €325 million for showing ads between emails in Gmail without consent and for placing advertising cookies at account creation without valid consent (CNIL, 1 September 2025). Neither case is about the Gemini app, but both show how closely consent and impact assessments around Google's products are watched.

Beyond fines, when a customer or employee learns that their details were typed into a consumer chatbot, that is a trust problem no settings page can repair.

The Gemini settings that make a difference

On a personal account

1. Turn off Keep Activity, or use a temporary chat On gemini.google.com, click Activity, then On, and choose Turn off or Turn off and delete activity. Future chats are no longer used for training, although Google still keeps them for 72 hours. For a single sensitive question, a temporary chat works the same way without changing the setting.

2. Disconnect the apps you do not want Gemini to use In the Gemini app on your Android phone, tap Menu, then Customize (under More if you do not see it) and Connectors. Next to an app, tap More options and then Disconnect. Phone, Messages, WhatsApp and Device assistance stay available even with Keep Activity off, so this is the place to turn them off. They only appear in the Android app, not on gemini.google.com.

3. Shorten the auto-delete period Activity is deleted after 18 months by default. You can choose 3 or 36 months, or turn auto-delete off. Choose the shortest period that works, and remember it does not reach chats already selected for human review.

In Google Workspace: a checklist for admins

  • Steer people to the work account. The single most effective control is that employees use Gemini with their managed account. In the Admin console, under Generative AI and then Gemini app, Service status decides who gets the Gemini app.
  • Set conversation history. On the same page, Gemini conversation history keeps chats for 3, 18 or 36 months, or not at all. The side panel in Gmail, Docs and Drive has its own setting under Gemini for Workspace.
  • Decide what Vault keeps. Check whether your edition includes Vault, and set a retention rule for the Gemini app that matches your policy. A hold keeps chats that users delete. Side panel conversations can be exported from Vault but not held, so their own retention setting is what counts.
  • Know where the audit logs are. Under Reporting, Audit and investigation, Gemini for Workspace log events shows who used Gemini, when and where.
  • Decide on Gemini Notebook and Gemini in Chrome separately. Gemini Notebook can be turned on or off per group in the Admin console; Gemini in Chrome can be turned off with the GeminiSettings policy in Chrome Enterprise.
  • Keep data in Europe where you need to. From Business Standard up, Data Regions can store Gemini prompts and answers in Europe. Keeping the processing in Europe as well takes Enterprise Data Regions (part of Enterprise Plus, or an add-on) or Assured Controls. Gemini Notebook falls outside Data Regions. For stricter requirements, see our article on data sovereignty and AI prompts; client-side encryption keeps selected files out of Gemini's reach.
  • Tell employees what you can see. Explain whether Vault keeps their Gemini chats, for how long and who can search them.

Checked in October 2026 against Google's Gemini Apps Privacy Hub, the Generative AI in Google Workspace Privacy Hub, the Vault documentation and the Workspace admin help. Menu names can differ by device and app version.

Seeing what is in the prompt, while you type

Settings decide what Google may do with the data. They do not decide what an employee types, or which Gemini they type it into. No admin policy reaches the support agent who, on her own phone, pastes a customer's bank details into the consumer app. The same pattern runs through every AI tool; our overview of AI data leakage shows where else it happens.

BeeSensible checks text fields for personal data as you type, including the Gemini prompt box. The browser extension sends the text to BeeSensible's detection service in the EU, where it is analysed in memory and discarded after detection. When sensitive details appear in a prompt, they are highlighted in the text: yellow, or red for the most sensitive data. The user can mask the value with a placeholder or remove it before sending.

Gemini
Can you draft a polite reply to this customer?
Of course. Paste the details below and I'll turn them into a courteous reply.
BeeSensible puts its highlights right in the text.
Hover over or tap a highlighted value to mask or delete it before the draft reaches anyone.

The content of messages is not kept. What happens to a highlight is up to the user, unless the organisation has set masking on paste or on send to On by default or Always on. With Keep context, the extension puts the real values back into the answer on the employee's screen, while Gemini only ever sees the placeholders.

For Gemini, this matters most just before a customer's IBAN, a patient's identifier or an employee's date of birth is sent. A highlight does not hold the message back, and it is no substitute for choosing the managed Workspace version over the consumer app. It shows the person typing what is in the prompt, while there is still time to change it.

FAQ

Common questions

Is Gemini safe to use at work?

It depends on the account. With a managed Google Workspace account, Gemini falls under your organisation's Workspace agreement: Google states that your content is not reviewed by humans or used to train its models outside your domain without permission. With a personal account, the consumer terms apply: chats can be used for training while Keep Activity is on, and a subset is read by human reviewers. The first question is always which account an employee is signed into.

Can my employer see my Gemini chats?

On a work or school account, yes, if your organisation uses Google Vault. Vault can retain, hold, search and export the prompts and answers in the Gemini app, including chats you deleted, when a hold or retention rule applies. Audit logs show who used Gemini, when and in which app. Since 22 October 2026, admins can also use Vault to export Gemini conversations from Gmail, Docs, Drive and other Workspace apps, filtered by user and date range, though they cannot yet put a retention rule or hold on them. Gems and media files attached to Gemini app prompts are outside Vault. A personal Google account is outside your employer's Vault and admin settings.

Is it safe to connect Gemini to Google Workspace?

On a work account, Gemini only retrieves Workspace content you already have access to, and that content stays under your organisation's Workspace terms. Your admin decides whether the Workspace app in Gemini is available. The risk lies in your own sharing settings: a file shared more widely than intended is also within Gemini's reach for everyone who can open it. On a personal account, content Gemini pulls from Gmail or Drive becomes part of the chat and follows the consumer terms, including human review while Keep Activity is on.

Does Google use Gemini conversations to train its AI?

On a personal account, yes, while Keep Activity is on: Google uses your activity to improve its services, including training generative AI models, and a subset of chats is read by human reviewers. Google warns against entering confidential information for that reason. In a temporary chat, or with Keep Activity off as long as you send no feedback, future chats are not used for training. On a Workspace account, Google commits that your content is not used for training outside your domain without permission.

Does Gemini keep my data after I delete it?

On a personal account, chats already selected for human review are not deleted when you delete your activity: Google keeps them for up to three years. With Keep Activity off, chats are still kept for 72 hours. On a work account, your admin sets how long conversations are kept, and a Google Vault hold or retention rule keeps chats even after you delete them.

Is Gemini safe for company or regulated data?

Regulated personal data (health information, identification numbers, financial details) should only be processed in a managed Google Workspace environment with a data processing agreement in place, not in the free consumer app. Even then, it is safest to keep personal data in any prompt to a minimum, because the protection covers how Google handles the data, not whether it needed to be there at all.

How do I stop Gemini from accessing my phone apps?

Open the Gemini app on your Android phone, tap Menu and then Customize (under More if you do not see it), and then Connectors. Next to the app, tap More options and then Disconnect. Do this for Phone, Messages and WhatsApp if you do not use them: on Android these stay available even when Keep Activity is off. They only appear in the Android app, not on gemini.google.com.

See how BeeSensible works

Sensitive data gets a marking while you type, in the tools your team already works in. You decide what happens before you send.

Portrait of Rens Timmermans from BeeSensible

Prefer someone to walk you through it?

Rens, BeeSensible

I'm happy to give you a twenty-minute tour. Leave your details and I'll find a time that suits you. You're also welcome to call or email.

Prefer to talk to someone first?

Want a live demo, or a quote for 100+ users? Leave your details and we'll be in touch within one working day.