Round 1 of the play-training experiment is complete.
The question was simple:
Can we train engineering taste into a coding model without materially damaging its ability to solve hard problems?
The training run
We trained Qwen on a Go-first corpus built from play, critique, revision, authoring, and solution trajectories.
The final training run was surprisingly small:
- 92 eligible challenges
- 536 attempts
- 349 solver conversations
- 263 authoring conversations
- 1 epoch
- 44 training steps
- 21.7 GB Q4_K_M GGUF export
Correctness on the promotion benchmark
Then we ran an independent 120-task promotion benchmark against both the exact base model and the previous SFT.
Hard correctness:
- New candidate: 71/120
- Exact base: 70/120
- Previous SFT: 73/120
So: essentially the same capability band. The new model did not beat the previous SFT on correctness, so it was retained but not promoted.
Blind engineering-taste evaluation
The more interesting result came from blind engineering-taste evaluation.
The critic was only allowed to compare tasks where at least two models produced distinct, objectively correct implementations.
Across 72 valid comparisons:
- New candidate preferred: 28
- Previous SFT: 23
- Exact base: 18
- No preference: 3
Among the 47 clean three-way comparisons:
- Candidate: 18
- Previous SFT: 17
- Base: 11
- No preference: 1
What the results show
The careful interpretation is:
Conditional on producing a correct solution, the newly trained model was more frequently preferred by the blind engineering critic than the base model.
That is not yet proof of globally superior engineering judgment. One seed, one critic family, modest sample size.
But it is evidence that the training procedure may be changing HOW the model engineers, without substantially changing WHETHER it can solve the problem.
And that is exactly what we were trying to test.
Total active experiment time: about 71 hours on a DGX Spark.
The replication run
The next step is deliberately boring science: reproduce the exact training run with a different seed.
All of the same experimental parameters, other than that. Base, dataset, recipe, and benchmark all stay the same.
If the same pattern appears again, then things get much more interesting.
After that, we test whether the child model can help construct its own next curriculum.