I Was Losing Context Across 100 Chats, So I Built My Own Memory

I had somewhere around a hundred open chats spread across ChatGPT and Claude, and I couldn't tell you what was in half of them.

Every new conversation started from zero. If I wanted the model to remember something from three chats ago, I had to go find that chat, or just re-explain everything from scratch. That led to duplicates. I'd end up with three or four conversations that were basically the same thing, created because I couldn't remember which one already had the context I needed.

Then there's the built-in memory features these tools ship with. Even when they work, they cost tokens and time. Every request has to pull from memory, and that retrieval isn't free. It adds latency and it adds to the bill.

And on the other end of the spectrum: sometimes I just want to ask one thing. A single question I'll never need again, that I don't want folded into a permanent memory system or sitting on a server tied to my account.

What I actually wanted was simple to describe and apparently hard to find: something available whenever I sat down to work, that didn't lose context between sessions, that didn't tax every request just to remember things, and that let a one-off question stay a one-off question.

So I built mybutler.

The pitch in one line

Ask anything, privately. No account, no cloud, no limits.

mybutler lives in your menu bar. Click it, ask a question, get an answer. The model answering runs locally through Ollama, so nothing you ask ever leaves your machine.

The actual problem with the tools I was using

It wasn't just privacy, though that was part of it. It was the whole shape of how these tools handle memory and context:

• Context lives in the chat, not with you. Start a new conversation and you're starting over, unless you go dig up the old one.
• Duplicate chats pile up. Not knowing where a piece of context lives means recreating it somewhere new.
• Memory has a cost. Retrieving relevant history from a large chat memory takes tokens and time, every single request.
• No room for a true one-off. A single private question either gets folded into your account's history or you have to consciously avoid using the tool for it.

mybutler fixes this by not tying memory to a conversation at all. Facts live in a local store, not inside a specific chat, so there's nothing to lose track of and nothing to duplicate.

What it actually does

It answers privately. Every question is handled by a model running locally through Ollama. Nothing typed into mybutler is sent anywhere else.

It remembers what matters, on its own, outside of any single chat. As you use it, mybutler builds a memory of facts and scores each one by relevance, recency, and how often it actually gets used. Open the Ask window and you'll see the back-and-forth for that session, same as any chat. Close it and that thread is gone, on purpose. What survives is the memory underneath it, so the context is still there next time, whether that's later today or a month from now, without you re-explaining anything or digging up an old window.

It only retrieves what's relevant, not everything. Instead of feeding a whole memory or chat history into the model on every request, mybutler searches for what's actually relevant to the current question and compresses that into a short summary first. That keeps retrieval itself cheap, usually well under a second, so context doesn't cost you tokens or time recalling things that have nothing to do with what you asked. The answer itself still takes as long as the local model needs to generate it: that part is your hardware's speed, not something mybutler can shortcut.

It admits what it doesn't know about you. Ask something that depends on a fact about you and nothing in memory actually answers it, and mybutler says so instead of guessing. General-knowledge questions still get answered the normal way, memory or not, same as talking to any model.

A one-off question doesn't turn into a permanent transcript. There's no chat log piling up somewhere you have to manage or delete. mybutler still looks at everything you ask for anything worth remembering, small talk and pure lookups usually just leave nothing behind, but there's no per-message toggle for it, that judgment call is the extraction step's, not yours.

It stays out of the way. One click from the menu bar. No tab to dig up, no app to remember to open.

How it works under the hood

mybutler runs as an Electron menu bar app. When you ask something, it searches local memory two different ways at once: keyword search using SQLite FTS5, and semantic search using sqlite-vec. The results from both get merged.

That merged set of facts gets compressed into a short, focused summary before it's handed to the local model, qwen3:14b by default, which is the one that writes the actual answer, along with the last few turns of that session's conversation so follow-ups still make sense. This is the part that solves the token problem on the retrieval side: the model sees a short, relevant summary, not your entire memory or every chat you've ever had.

The embedding side runs on nomic-embed-text, also through Ollama. Both are hardcoded right now, there's no settings UI for swapping models yet, just two constants in the source if you want to point it at something else.

Getting it running

bun install
bun run start

You'll need Ollama installed and running locally, with qwen3:14b and nomic-embed-text pulled. Once that's done, mybutler shows up as a small icon in your menu bar. Click Ask and you're talking to it.

The only point where mybutler touches the internet at all is that initial setup. After that, it's fully offline.

Where it's headed

mybutler is provider-agnostic by design. Local is the default and stays the default. Cloud is something you turn on, never something that happens quietly in the background.

Two things are planned next:

• Claude via API, for anyone who wants to bring their own key.
• Claude via a local Claude Code process, so people who already pay for a Claude subscription get frontier-model answers without a second bill.

Neither changes the core behavior. Memory still lives locally, retrieval still stays cheap, and nothing gets sent anywhere without you choosing it.

The questions people actually ask

Is it free? Yes. It runs on your own hardware with your own local models.

Does it need the internet? Only once, to install Ollama and pull a model. After that, it works fully offline.

What models does it support? qwen3:14b and nomic-embed-text are what it ships with. Swapping either for a different Ollama model means editing two constants in the source, there's no settings UI for that yet.

Is my data really private? Yes. Nothing you ask it, and nothing it remembers, leaves your machine unless you explicitly turn on a cloud provider.

Does it replace my chat history? No, it's not trying to be a chat client. It's a memory layer and an assistant that doesn't need a hundred open tabs to know what you told it last week.

Why I built it this way

The actual trigger for building mybutler wasn't some grand privacy stance. It was frustration with the mechanics of using AI tools day to day. A hundred chats and no idea which one had the context I needed. Memory features that cost tokens and time just to work. Questions I wanted to ask once and never think about again, but had nowhere private to put.

mybutler is my answer to all three at once. Memory that isn't trapped in a single conversation. Retrieval that only pulls what's relevant instead of everything. And a private, local model underneath it so a one-off question can stay exactly that.

It's early. Twelve stars and a handful of commits, not a company. But it does the thing I actually needed.

If you want to try it or dig into the code: github.com/alexcloudstar/mybutler