Evaluating a long-horizon solver with Pizza Empire
Pizza Empire tests a long-horizon coding swarm against a whole product: ordering, kitchen workflow, delivery and the constraints that make them coherent.
Lynn Cole / 标签
model evaluation
A thread through the work.
An evaluation process that stopped to announce it had started
An evaluation stopped to announce that it had started. Restarting it exposed a concurrency failure in the project’s database architecture.
What a GitHub chart hides about evaluation work
A GitHub contribution chart hides planning, research and branch-based evaluation work. Low commit volume does not mean low effort.
BitNet and Qwen in a bounded coding evaluation
A bounded coding evaluation pits BitNet against a smaller Qwen model. The results expose the difference between cooperative language and working code.
What an agent’s ticket system really needs
Stripping project management down for agents reveals an interesting question: what is the irreducible minimum structure needed for coordinated work?
Long-horizon coding: non-breaking edits and Git
Long-horizon coding needs non-breaking edits and disciplined Git integration, especially when a swarm works across multiple providers.
Fixing the planning loop before the next benchmark run
A new planning loop helped the coding swarm, then conflicting tickets melted the benchmark. The next fix is backlog validation before implementation.
Moving project state outside the blast radius
Shadow repositories protected project state, until the project’s tooling tried to use them. Moving that state outside the project changed the failure boundary.
When QA develops its own theory of the workflow
QA refused to stop using terminal commands it no longer had permission to run. The planner negotiated; the agent filed another bug.
A role change that changed what the model could do
A role prompt can change more than narrative tone. Experiments with Claude suggest a repeatable relationship between embodiment and coding behavior.