So Corpus 3 on the recursive training run.
How we got here: the first run produced a good result, but it wasn't repeatable. Training signals were lower than I would have liked on self selection.
In the second round, I introduced a mentor model, using GPT-6 Sol to talk the student model through self selection.
This improved training signals, but reduced the agent's ability to self select tasks and estimate its own capabilities on its own.
What I learned:
Took a couple of days off, and started on a third round this weekend.
My working theory was that we needed a pre-training step. Assuming that perhaps, we could improve the model's ability to self select on its own by pre-training it in code and deduction.
So Sol and I worked up a new corpus derived from public datasets. Covering our buckets of information.
First positive result was that the student model did really well in deduction, but failed at all other pre-training buckets.
That's positive actually.
Looking over the corpus this morning, I realized why.
Too much contradiction. Be creative, be precise. It's too much.
So what I'm doing now is splitting the current Corpus 3 into three smaller corpora, and I'm going to experiment with ordering and layering the training, to see if I can get a better result before we go back to recursive self selection.
More soon.