A status update on training the recursive Golang coding agent.
For the past few weeks I've been doing developmental training on a Qwen coding model running on a DGX Spark. The model participates in its own development: it expresses interests, a stronger model turns those into bounded programming tasks, a verifier decides whether the code works, another model provides criticism, and the child gets one chance to revise or keep its answer. Saying “I understand” earns nothing. Writing better code does.
The first major child, v125, showed signs of better engineering taste in blind comparisons between multiple correct implementations. Not just “does this pass?” but “which implementation would I actually want in the codebase?” Then we recursively trained v125 on a second corpus. The first attempt went badly: at the original 2e-5 learning rate, the child dropped to 66/120 solves and lost the taste advantage.
The learning-rate ablation
So we ran a clean ablation: same parent, data, updates, seed, LoRA geometry, and evaluation. One change: 2e-5 → 5e-6.
The regression disappeared: low-rate child 74/120, exact base 72/120, v125 parent 70/120, previous SFT 68/120. The recovery was broad. On HTTP handlers, the full-rate child had collapsed to 17/30; the low-rate child recovered to 22/30. Training dose matters.
Correctness and taste diverged
But then it got weird. The child got better at being correct while apparently getting worse at choosing how to be correct. On the broad taste set, parent and child were close: 18 vs. 16. But on the preregistered strict subset, where all four models were correct and distinct:
Base: 16
v125 parent: 10
Previous SFT: 6
Low-rate child: 3
That failed our gate. So we stopped. No fresh promotion benchmark. No moving the goalposts because 74/120 looked nice.
This may be the key result: competence and engineering judgment appear to be partially independent. Can you solve the problem and survive the verifier? Separately, when several approaches work, which one do you choose? What structure do you preserve? What complexity do you refuse to introduce? When do you stop?
What the training signal emphasized
Corpus 2 contained 559 ordinary solution trajectories, 89 revision pairs, and 115 comparative preference records. But those preference records were evidence, not direct SFT targets. So most of the gradient still said, “Here is another solution.” Much less of it said, “Here are two solutions that work. This one is preferable, and here is why.”
Before training that signal, we need to validate the ruler. Next is a critic-stability study using fixed correct solutions, blinded label permutations, an independent critic seed, and no solver resampling.
A few weeks ago the question was whether developmental play could improve a coding model. Now it's: can competence and engineering judgment be trained as separate developmental axes, then improved together without one erasing the other?
That feels like the experiment.