I took a couple of days off from the recursive agent training to internalize and contemplate what just happened in the second corpus.
Super interesting stuff.
In the first round, I learned that while Qwen-3.6 can recursively train, its self-assessment of training signals was only useful about 25% of the time.
When we were able to get useful training signals from self-assessment, they worked.
The model got significantly better at coding, especially around ground truth and code reasoning, which is exactly what I wanted.
But it was a lot of churn, and it was hard to do in a controlled, predictable way.
My solution: Let GPT-Sol discuss the task with Qwen to improve the training signals in the data.
We did improve the training signals significantly, and in a way that was legitimately useful.
New problem: The tradeoff is that Qwen now can't reliably reason about its own capabilities at all, and it fell dramatically below baseline when estimating tasks on its own.
Cool, right?
So for Corpus 3, I think there needs to be another step in the training process. I'm going to use a handful of public corpora to pretrain the model on code reasoning and software estimation.
And I'm going to use Sol to create synthetic training exercises based on examples from each corpus.
My hope is that this makes Qwen a stronger estimator before the recursive self-training kicks in, and improves the training signals in the exercises it chooses to complete.
More soon.