Turned off the internet — and search still works
Ran local models on my Mac: embeddings as good as the cloud ones, Mistral extracts tasks from notes. Privacy is turning from a pretty word in the pitch into a feature we can actually build.
By Georg Malahov
A week and a half ago I ran Deep Research — OpenAI's deep-research mode — over the voice-notes market. The conclusions are sobering: there are many competitors, almost all of them American services with cloud storage. And that's exactly where our potential strength lies: for most of them the weak spot is privacy, while we sit in the German market, where "where does my data go" is the first question people ask. Plus a second idea, the one I like best: zero effort — an app with a single button where you dictate everything haphazardly, and it sorts it all out by itself, no manual input. Nobody really has that.
But "privacy" in a pitch is just a pretty word as long as the transcription and analysis run in someone else's cloud. Today I tested how realistic it is to move all that to the user. And I'll say it straight: I was blown away by the ecosystem that has already been built around local models.
I started with embeddings — numeric fingerprints of a text's meaning, which have powered my note search since December. I spun up a local model, re-vectorized all my notes in a local Postgres, and compared the search against my current cloud setup from OpenAI. The results are practically identical — so close that I honestly can't see a meaningful difference.
Next — text generation. I found Ollama — a tool that installs language models on a local machine almost like Docker images: ollama pull plus the model name. I installed Mistral; there's an open-source interface for it that looks like a familiar chat. And then I did the control experiment: turned off the internet, took my notes, gave them to the local model with the prompt "extract the tasks" — and it correctly formulated the completed tasks from my chunk of text. All of this on my laptop, without a single request leaving the machine. Pure joy.
As a bonus I browsed Hugging Face — a catalog of open models: image analysis, text-to-speech, you name it. Each one's quality has to be checked individually, but the fact itself stands: out of this you can assemble a local Frankenstein of models tailored to our tasks, and run it on your own server or right at the user's.
For the project this changes the picture in two places. Experiments are now faster and free — no counting cents for every API call. And privacy turns from a marketing word into an architectural decision you can actually build.
I immediately shared my excitement with my first co-founder — dictated him a voice message. The main task for the week: assemble the chain that extracts and updates tasks from notes, and show the team that local models are not a toy.