The Quiet Case for Running AI on Your Own Machine
A few months ago I found myself in a slightly absurd situation. I was sitting in a café somewhere between Zurich and nowhere in particular — one of those spots with good coffee and catastrophically bad WiFi — and I needed help from an AI assistant with something I was working on. My usual tools were just gone. Not slow. Not broken. Gone. Because everything I relied on lived on someone else’s server, and that server might as well have been on the moon.
I have to admit this is not the kind of problem that ruins a day. But it stuck with me in a way I didn’t immediately understand.
The appeal of running AI completely locally has been obvious in theory for a while. Nothing you type leaves your machine. No company is logging your queries. No API goes down on you at the worst possible moment. You’re not subject to a usage cap, a terms-of-service update, or someone’s decision to deprecate the endpoint you built a workflow around. You run the model. It runs on your hardware. That’s the whole story.
However, the gap between understanding why this is appealing and actually getting there has always been real. I spent a weekend earlier this year setting up a proper local AI environment on my MacBook Pro — Ollama, a handful of models, a terminal workflow that felt good to use — and what surprised me most was how capable the whole thing was. Apple Silicon, it turns out, is genuinely suited for running local language models. We’re not talking about toy outputs. Reasonably large models, running fast, completely offline, on a laptop that fits in a bag.
The problem is that getting to that point requires navigating a maze most people have no reason to know how to navigate. Which model do you download? What is the difference between a 7B and a 14B? What does quantization actually mean and why should you care? Ollama is a great piece of software, but it assumes a certain comfort with the terminal, and most people I know — smart, technical people, some of them — don’t have that baseline. They’d like to try this. They just don’t know where to start.
I kept explaining it, and I kept noticing that the explanation was getting longer and more involved than it should be for something that, once you’re past the setup, is genuinely simple.
So I built something.
Shell Buddy (shellbuddy.tschuebi.com) is a small web app I put together to lower that barrier. The idea is not complicated: guide someone through installing Ollama and getting a local model running, step by step, in a way that feels manageable rather than intimidating. There’s a terminal sandbox where you can practice commands without worrying about breaking anything. A model picker that helps you figure out which model makes sense for your machine. A decoder for the moments when an error message stops being readable. And — because I find this kind of thing genuinely motivating — streaks, stars, and levels to make the whole process feel like progress rather than a chore.
I did not have a rigorous business case for building this. The honest reason is that I wanted it to exist.
The bigger point, though, is something I keep coming back to: there’s something worth thinking about in the idea of owning your AI setup. I use cloud AI constantly, and I’ll keep using it. But having a local fallback changes your relationship with the tool. You understand what’s running. You can use it on a flight over the Atlantic, in a cabin with no signal, in a café where the WiFi has given up. The models are good enough now. The hardware — at least if you’re on Apple Silicon — is good enough now. The missing piece has been a gentle on-ramp for people who aren’t already at home in a terminal.
That’s what I was trying to build. Whether I succeeded is something I’ll find out from people who use it.
Shell Buddy is in beta right now. If you’ve been meaning to look into local AI and haven’t quite gotten around to it, I’d genuinely love to have you try it. You can sign up at shellbuddy.tschuebi.com. It’s free to get started, and the setup takes about ten minutes if everything goes smoothly — which, in my testing, it usually does.
In seven months I suspect a lot more people will be doing this. Might as well start now.
Stay safe,
Raffael
