A tender specifies that processing must happen locally. Three vendors tick the box. For the first, the model runs on the laptop. For the second, it runs in an isolated environment in a European data centre. For the third it runs locally, unless the local component is unavailable, in which case it goes to the cloud.
All three can explain why their answer is correct. Which is exactly why the distinction is worth pinning down before it appears in a contract.
The actual difference
In cloud processing your text travels to a server. There it is decrypted, processed and usually stored, with the consequences that follow: a retention period, the possibility of human review, and a third party in possession of your text.
In on-device processing the text stays on your machine. The model runs on your own processor or GPU. What leaves is at most a result, and sometimes nothing at all.
That difference is not cosmetic. Four things fall away at once:
- Nothing to intercept in transit, because there is no transit.
- No conversation history at a vendor.
- No third-party retention period applying to your text.
- Nothing that can fall under a preservation order at that vendor, as happened in the litigation over retained chat logs.
For a DPIA that removes a column. Processing that does not happen needs no justification.
Where it gets less simple
Three caveats that rarely appear in a pitch.
Your endpoint matters more. With processing and sometimes data sitting locally, risk shifts to disk encryption, screen lock, device management and theft. An unmanaged laptop doing local processing is not safer than managed cloud processing. It is a different risk, not automatically less risk.
Something usually still leaves. Models need updating, licences need checking, and most software reports usage statistics. All fine, provided that stream is value-free: counts and events, no text. This is the question most often skipped, and it is where the gap between "local" and "local, except" becomes visible.
What happens when the local component fails? The most important of the three. Some systems stop and say so. Others fall back to the cloud silently, and then the promise breaks at exactly the wrong moment. Ask explicitly about this behaviour and about how the user finds out.
Which task belongs where
The honest position is that the boundary runs per task and moves every year.
Local already works well for bounded work: detection, classification, filtering, pattern matching, short transformations. Tasks where a small model is competent and where latency matters more than depth.
Cloud is still needed for the heavy end: complex reasoning, long documents with a lot of internal reference, and anything where you want the largest available model. A model with hundreds of billions of parameters does not fit on a laptop, and that will not change next quarter.
What does change annually: small models get better and laptops ship better accelerators. Tasks that were impossible locally two years ago now run comfortably.
What to ask a vendor
Four questions that separate "local" from "local in the brochure":
- What data leaves the device, and at what moment?
- What happens when the local component is unavailable? Fail, or fall back?
- Does the user notice the difference, and how?
- What telemetry is transmitted, and does it contain text or only counts?
A vendor who answers those four has described an architecture. A vendor who only says "we process locally" has offered a slogan.
We had to make this choice ourselves, and our answer is configurable per organisation: detection can run entirely on the device, with no text and not even a check value of that text leaving the machine, and the system reports when the local component is missing rather than quietly falling back. That is not a better choice than the cloud variant. It is a different trade-off: slower to deploy, less dependent on somebody else.
