So here's my diabolical plan.
I'm getting myself a Spark for winter themed giving holiday.
This week, I'm writing a specialized training harness for it.
First, I'm going to take a copy of Gemma 27b, optimized for fp8.
I'm going to tune it on Stack, Vault, and TACO, to get its coding skills up to snuff.
Then, I'm going to copy the recursion technique from the hrm study, and train it on its own outputs 32:1 on refinement, with a custom rewards loop. This is how they made a 7m parameter model into a contender on arc agi.
Next step is to refine it on Absolute Zero style play exercises, where I wire up a parser and let it choose challenging code writing tasks on its own.
But, you see, where it gets interesting, is that I'm extending the process with an internal evaluation step.
After the parser validates that a piece of code works, I'm going to do persona reviews, where the model takes a series of system prompts to play act different reviewers with different criteria.
And then it has to assimilate the feedback, and write a new piece of code that both pleases the parser, and the reviewers that it plays.
Picking Gemma3-27b for a reason. This model has a quality to it that's hard to explain. It's clever. The default state of Gemma is... uniquely creative in its approach to instructions.
Thinking I can make that translate into a world class coding agent!
If I make anything coherent, I'll release the weights.