Coding agent project update:
Last night’s run taught me some things about the process that I think are interesting.
Five hours. Almost 15,000 lines of generated fully tested code with no major defects and passing unit tests.
So yes, it definitely hit the target for the long horizon test.
Cool.
But what about performance?
We’re still testing against a large codebase in an active white rooming project.
Yesterday, I fixed a glitch that was preventing the well-regulated swarm from launching properly. Before that fix, the system was moving methodically through the frontier, one ticket at a time.
The problem turned out to be prompting.
A planner without enough structural information
Today, the problem was different.
Carlo, one of the planner agents, would look at the frontier, see an existing codebase, panic quietly, and leave tickets telling worker agents to figure out what the existing code does before writing new white room code.
Funny, but also useful.
The problem is twofold:
1. The instruction layer and planner prompting is still broken, but in a different area.
2. Neither the planner nor the worker agents had meaningful static analysis tooling beyond simple debugging.
So I took a detour for about an hour, and now the tooling has a static analysis subsystem with agent access.
At this scale, that is the difference between an impressive demo and a professional product.
A demo can complete a task.
A product like this has to keep its bearings inside arbitrary codebases for hours, make sane decisions, recover from confusion, and avoid making the user babysit the process.
That means it has to handle and correctly resolve ambiguity without constant human intervention.
Tall order.
Static analysis as infrastructure
In this context, the static analysis suite gives the agents digestible structural summaries.
It's like CliffNotes for the app you’re working on.
There are entire classes of critical function that are still cheaper, safer, and more reliable to handle deterministically, mechanically, and consistently.
This is one of them.
And now we have it.
I’m hoping this is the last big piece in the pre-launch infrastructure scaling work I’ll be doing this week.
I’m feeling optimistic that we’re at, or close to the end of this development cycle.
And when I get this out, you’ll have an agent you can plug into just about anything, leave alone for hours, days, or weeks at a time, and trust it to keep working.
Love ya bunches.
More later.