The local-model router.

The expensive default

Most “autonomous agent” setups reach for a frontier model on every step. New message? Frontier model. Routine status update? Frontier model. Obvious nothing-to-do? Frontier model, just in case. It works in a demo and it quietly bankrupts you in production, because the vast majority of what an agent does all day isn't hard — it's bookkeeping.

The whole game is deciding what doesn't need the expensive model. And that decision — “is this trivial or does it need real help?” — is itself easy. Easy enough for a 3B model running on a laptop, for free.

The router does one thing

Every incoming item hits a small local model first. It doesn't do the work. It makes one call: handle / escalate / ask.

• handle — the genuinely trivial: an ack, an obvious confirmation. The little model answers it for free and moves on.
• escalate — the default for anything real. Needs a fact, a tool, reasoning, code, judgment? It goes up to the big model. When in doubt, escalate.
• ask — when intent is unclear, or the action is irreversible, outward-facing, or touches money. It sends me one short question and waits.

// the only call the small model makes — illustrative, not my prompt
const decision = await localModel.chat({
  format: "json",     // force a structured decision, never prose to parse
  keep_alive: 0,      // unload the model right after -> RAM freed, $0
  messages: triagePrompt(item),
});
// decision = { route: "handle" | "escalate" | "ask", confidence: 0..1 }

switch (decision.route) {
  case "handle":   replyCheaply(item); break;
  case "escalate": runBigModel(item);  break;   // the hard ~10%
  case "ask":      askHuman(item);     break;
}

Two flags carry the economics. format: "json" means you branch on a clean field instead of parsing English. keep_alive: 0 means the model loads, answers, and unloads every single time — no idle GPU, no warm process, no bill. It's free because it's never sitting there.

The prompt is humility, not capability

Here's the part people get backwards. A 3B model is not smart, and you should not try to make it smart. The prompt that makes it safe to put in front of your money is written around what it can't do, not what it can.

The principle, in plain terms (the exact wording stays in my repo): you are small; you have no tools, no memory of the business, no reliable facts; you are good at reading intent and judging “trivial vs needs-help” and bad at everything with consequences; if you are not sure, assume you are wrong and escalate — escalating is cheap, acting wrong is expensive.

That's the whole trick. The small model isn't trying to be right about the work. It's trying to recognize when it's out of its depth and hand off. A router tuned to be humble fails safe; a router tuned to be clever fails expensive.

Run it, then die

There's no daemon holding memory. A scheduler (launchd, cron — your pick) runs one process on an interval, and it exits when the tick is done. A tiny durable cursor on disk remembers where it left off, so the next run wakes up oriented instead of confused. “Slow is fine” is a feature: low priority, one thing at a time, nothing lingering between ticks.

// one tick, then exit — the scheduler owns the cadence, not a loop
lock()                       // skip if a previous tick is still running
const items = gatherInputs(cursor)   // new messages + a periodic scan
for (const item of items) act(route(item))   // route -> handle / escalate / ask
saveCursor(cursor)           // tiny JSON: where we left off
unlock()                     // process ends; local model unloads; RAM freed

Escalate sparingly — and sign everything

When the router escalates, the hard item goes to the big model non-interactively, on the subscription I already pay for — one at a time, never in parallel. The frontier model only ever sees the slice that's genuinely hard. That's the line item that collapsed; the token bill is the proof the routing works.

And because this thing acts while I sleep, every decision and action gets a cryptographic signature (ed25519) over a small canonical envelope — who did it, what, against what, a hash of the content, a timestamp, a nonce. Change any field and the signature breaks. I can replay a night of work and verify each step actually came from my own agents, not from something slipped into an email one of them read.

// every action is tamper-evident — generic envelope, sign with the agent's key
sign({
  agent, action, target,
  contentHash: sha256(content),
  ts: now(), nonce: random(),
});  // verify later with the agent's public key

Why I build it this way

Everyone is selling a bigger model. The cheaper, better bet is a smarter harness around a small one: put the intelligence in the routing, not the weights. A humble local model in front, the expensive model only for the hard 10%, a scheduler that runs and exits, and a signature on everything. It costs almost nothing per tick and it gets sharper every time I tighten a rule.

Across 35 shifts it monitored ad accounts, triaged my inbox, found prospects, and drafted outreach — and it did everything except press send. That part stayed mine. Which turned out to be the lesson: agents don't remove you from the loop. They shrink it until the only thing left inside is your decision.