As you know, I've been working towards something I call the "Netscape moment" of consumer facing, edge based agent systems. It's been a ride. Over the last two years, I've been carving out an entirely different philosophical space than the rest of the industry. It's not good or bad. I guess it is what it is.
When the AI browsers dropped, and I realized how bad they are... I decided that it's something that's not going to come from the big labs. We're going on two years. Being gracious and waiting for them to figure this stuff out is too much to ask.
This weekend, I had a breakthrough.
The current side project is called Busy. I'm working on the 36th iteration of it. Past versions of the app have experimented with form factors and theories about the way agent systems need to work. But now, I've settled on a really interesting ergonomic that I think puts it in or close to a final form.
Getting back to the original concept. It's a Godelling agent system that asks and answers the question: "What do I need to do in order to exist?" And it goes from there.
The target is consumer hardware.
Latest breakthrough involves exposing confidence scores in Llama.cpp. Turns out, everything a model says is calculated. You can spot hallucinations and guesswork by evaluating the confidence score of any given response.
You know the other thing llama.cpp allows you to do?
Orchestrate the thought process and reasoning chain of any model.
So I wrote a loop that lives inside the big agentic loop that powers the process. I told the system to evaluate this score on three core states, low medium and high, each triggering different dumb processes that affect the reasoning state of the machine. Low certainty works out to gather more information. Medium means, spend more time thinking. High, continue about your business.
All of a sudden, you can code with 4b models... entirely without retraining.
Not understanding why this isn't a thing in agent systems.
Seems obvious to me.
Wait til you see my next trick.
More soon.