AI that your data won’t leave
“And where does the data actually get sent?”
That question ends most discussions about AI in a company before they’ve properly started. Someone suggests it would be handy to have meeting minutes summarized or the mail pre-sorted — and then it dawns on everyone what that would mean for documents that aren’t supposed to leave the building.
There’s another way. A language model — the same kind of program that powers ChatGPT or Copilot — can also run on your own computer, with no connection to the outside. Not on every one: you’ll want something more like sixteen gigabytes of memory, and a dedicated graphics card helps. But it’s still an ordinary work laptop, not a data center.
Right alongside that, though, comes a sentence I don’t want to bury in a footnote: a model like that gives worse answers. The model itself has to fit into the memory of an ordinary machine, so it’s much smaller than the ones running in other people’s data centers on machines with dozens of graphics cards. It answers a complicated question less well, and now and then it gets something wrong where a big model would have gotten it right. There’s no trick to get around that — it’s the price of the data not traveling anywhere.
My own example, where online was out of the question
Recently I wrote a small tool for myself. At the morning meeting I’m supposed to say what I’ve been working on — and it goes much better when I know exactly, rather than fishing it out of my memory.
On computers with the hardware for it (Microsoft calls them Copilot+), a Windows feature called Recall can save screenshots in the background. It doesn’t do that on its own — it has to be switched on, and whatever it saves stays encrypted on the disk. I have it switched on, and I get a short summary put together from those screenshots.
The thing is, those screenshots contain exactly what I’ll never send to someone else’s service: unfinished code, notes, and sometimes things that belong to a client and have no reason to leave my computer. So for this tool, a big model out there wasn’t even an option.
That’s why the model runs right here on my machine, through a program called LM Studio. It works a bit like a media player: you download the model as a file, LM Studio loads it and plays it — except that what comes out is answers instead of music. The tool asks, the model answers, and the whole conversation never leaves my disk. For what I want from it — turning a pile of screenshots into a few sentences to go with my morning coffee — it’s more than enough. I don’t need a poem or a legal analysis from it; I need “this is what you did yesterday.”
What the client gets out of it
You don’t have to choose between AI and the peace of mind that your data stays at home. Where sensitivity matters more than how clever the answer is — summarizing internal notes, a first sort of text — a model running on your own machines tends to be the only option worth considering at all. And where it isn’t enough, you can decide case by case, not once and for all for the whole company.