Man, this play training setup I came up with is taking a long time. I decided to read three studies on the subject, and then without any preparation, just start doing the most challenging play training system anyone's ever done.
What? It's just three models and a go parser assembling a corpus from scratch, evaluating results, counting mistakes, and providing feedback.
It's gonna be so easy. I'll automate the whole thing.
We'll get this done in maybe a day, and then run the backward pass for maybe a couple of days, and it'll be great because then I'll have a happy little coding agent that specializes in Go, and punches above its weight class.
The dataset so far
Well, four days in, here's where we are on it.
- 🏐 72/250 training challenges, about 29% of the target dataset
- 🏐 61/120 independent promotion tasks, about 51% of the promotion bank
- 🏐 52 capability families
- 🏐 241 accepted solution trajectories
- 🏐 271 solver conversations
- 🏐 203 authoring conversations
- 🏐 30 revision pairs
- 🏐 53 explicit comparative preferences
I actually think the method is going to work.
Results so far are positive!
It just turns out that isolating training signals from exercises is a very slow process.
This one's going to take a minute.
When I'm done, it gets a snappy name, and I'll publish the weights and a paper about how I did it on Huggingface.
Yes yes.