Project update:
The long-horizon coding agent project is in what I hope is the final round of evals.
To recap, I’m building a state-of-the-art coding agent designed to solve long-horizon coding problems. It will be usable from the cli or over MCP. It’s good for things like porting and white rooming jobs that are still too big to reliably automate any other way.
Latest details:
I made another round of changes to better handle scale, large repositories, and swarm management. The new schema gives us an 8x performance boost over the original method.
The dependency graph in the current version looks really nice. I am quite proud of it.
My personal feeling is that the new ANN-based memory system beats the current state of the art by about a year. The trick was realizing that agent memory should be treated less like a database and more like a state machine.
The change really helped me understand how to see the correct mental model. Agent-directed memory never should have been treated like a filing cabinet, but rather an active component in a dynamic system.
The shape of the requirement is quite different.
I packaged that piece as a tool called Kilo-SDK, which should be landing on GitHub this week for anyone who wants to take it for a spin. It’s written in Go, and it should support both backend integration and tool-based client-side injection methods.
The Pizza Empire benchmark
The current eval is called Pizza Empire, because if you’re going to test long-horizon coding agents, you might as well get it with extra cheese.
The assignment is simple on paper: build a fully featured custom pizza delivery website, including customer ordering, kitchen tools, and delivery management, in 20,000 lines of code or less, without human supervision.
It’s a mean benchmark.
A real pizza delivery system has configuration logic, pricing rules, cart state, checkout flow, order lifecycle management, kitchen workflow, delivery dispatch, admin controls, status updates, permissions, and enough edge cases to expose whether a swarm of agents can reason across a whole product, without losing their pepperoni.
This version intentionally omits OAuth and payment processing. Those are important production concerns, but they’re not what I’m testing here.
Assuming it gets through Pizza Empire tonight or tomorrow, I’m hoping to send the project back to Samuel Goff for UI revision and UX optimization.
Once that’s done, we’re basically ready for release.
Looks like we’ll be skipping beta and going public with it.
I keep saying this, but I really do think we’re in the home stretch.