What an 8 GB MacBook Taught Me About AI
A weekend building a local AI setup on an aging MacBook - and why the failures taught me more than months of reading.
When I first tried ChatGPT in January 2023, it felt like a glimpse into the future. Over the months that followed, that future arrived faster than I expected. As each new generation of models leapfrogged the last, my usage grew from occasional experimentation to an essential part of my daily workflow. I subscribed to GPT-4 as soon as it became available, tested nearly every frontier model that followed, and eventually settled into a routine where AI became a genuine collaborator. Today, much of my professional work lives inside Claude’s Cowork and Code environments - a level of adoption I never would have predicted three years ago.
I consume an unhealthy amount of tech content. Over the past few months, my feeds have been filled with people building fascinating projects using agent frameworks like OpenClaw and Hermes, open models like Llama and Kimi, and surprisingly modest hardware - Mac minis, old laptops, and even everyday MacBooks.
They seemed to be having an incredible amount of fun.
Before long, I had accumulated a list of project ideas of my own. Rather than letting them sit in a notes app indefinitely, I decided to dedicate a few weekends to working through them one by one.
The first idea was simple: I had an aging MacBook Pro M1 gathering dust, and I wanted to see if I could turn it into a completely air-gapped writing environment-Wi-Fi off, cloud disconnected, everything running locally.
It was a slightly ambitious goal for a machine with only 8 GB of RAM.
The experiment had three objectives. First, I wanted to experience an open-weight LLM for the first time. Second, I wanted to get hands-on with an agent framework. Third, I wanted to understand, firsthand, the practical limits of consumer hardware. Yes, I realize many people solved these problems months ago, and yes, I am late to the party.
After some research, I settled on the Llama 3.2 3B model paired with Hermes Agent. On paper, it looked like a combination that should fit comfortably within the MacBook’s constraints.
A few hours later, I was deep in terminal windows, troubleshooting dependency conflicts, chasing down 404s, and editing configuration files. I’ll spare you the installation play-by-play. Besides, the troubleshooting was part of the fun.
Instead, let me jump straight to what I learned.
The lesson was the dependency chain.
I didn’t walk away from the weekend with a working application. More on that in a moment. What I did gain was something more valuable: an understanding of how the hardware, model, and agent layers constrain one another in practice, not just in architecture diagrams.
Before this experiment, I understood these dependencies intellectually. Afterward, I understood them in my hands.
The first lesson was that RAM sets the ceiling.
On an 8 GB MacBook, that memory has to be shared between macOS, the display, background processes, and the model itself. To avoid constant crashes, I also needed to keep more than 15 GB of disk space available for swap. Everything else flows from that constraint. It’s the first domino.
That memory ceiling effectively limits me to smaller models. In my case, a 3B model.
And that’s where the second domino falls.
A 3B model can chat surprisingly well, but it struggles with something far more important for an agent: tool calling. An agent’s entire purpose is to take actions on your behalf—save a file, update a document, execute a command, retrieve information. To do that reliably, the underlying model must be able to emit structured tool calls.
This was the moment that no demo video had prepared me for.
I would ask Hermes to write a profile and save it to disk. It would happily generate an excellent profile. Then, instead of saving the file, it would print a list of the tools available to it.
The reasoning was there. The action wasn’t.
After some digging, I discovered the issue wasn’t Hermes at all. The limitation was the underlying model. The Llama 3.2 3B model is effectively chat-only. Reliable tool use requires larger models—something closer to Gemma 4 31B—which in turn demands roughly 24 GB of RAM or more.
At that point, the dependency chain became obvious.
The hardware limits the model.
The model limits the agent.
And when the model can’t reliably call tools, the agent quietly degrades into a chatbot.
So my “agent” ended up doing what chatbots do: generating content. Meanwhile, I handled the file operations myself with a couple of Terminal commands.
Ironically, the part I expected to be difficult turned out to be the easiest.
Hermes is configured to use a paid cloud endpoint by default, so pointing it to a local Ollama instance required some configuration. But once that was done, creating an air-gapped environment was almost laughably simple.
Change the endpoint to ‘localhost’.
Turn off Wi-Fi.
Done.
Worth it — even if I never use it again
I won’t pretend this is a setup I’ll keep using.
The Terminal occasionally glitches into overlapping text. The 3B model has obvious limitations. Every new session begins with a 30-to-60-second wait while the model loads into memory. There is a very real chance I never open this setup for writing again.
And yet, the experiment was absolutely worth it.
For months, I had consumed videos, blog posts, benchmarks, and hot takes about local models and agent frameworks. I understood the concepts. I could explain the architecture. But what I retained was mostly the shape of the thing.
One afternoon spent chasing 404 errors, editing configuration files, and wrestling with hardware constraints taught me more than dozens of polished explainers ever had.
Because explainers show you the path that works, they rarely show you why it works.
And they seldom show you what breaks when one layer becomes the bottleneck for another.
That’s the last mile of understanding.
It’s messy. It’s frustrating. It’s where most of the learning happens.
~~~
What’s the setup you keep meaning to test on the machine you already own? Reply and tell me — I read everything.
The Last Mile of AI publishes one new learning every Friday. If you’d rather learn by watching someone else hit the 404 first, subscribe.