Audited the app today.
Claude's brain melted again.
We made a lot of progress, but Claude created four competing implementations, multiple forward passes, two different kv-cache implementations, ropes, bitstream and non-bitstream implementations.
We were looping.
Broadly, when that happens, it means I've been vibing too long, and that I've lost focus. A fair assessment.
Still having dequant issues. That particular process shouldn't be happening in the app at all. It's part of the philosophy. The design calls for compute based on compressed weights. I've got a novel way of getting there.
I think I figured out the problem.
I haven't been strict enough with my constraints.
I let myself drift into "wouldn't it be nice if"
Which, as you know, breaks one of my own core tenets.
It's not a total wash though. I'm learning a lot about AI inference firmware.
I'm conceptually managing to hold the whole thing in my head. Even I find this surprising. Valuable though. You can't get this kind of training in school, especially at this pace. I've just compressed an entire PhD path into about a week.
Every time I go through a cycle like this, it feels like exercise. It's that same feeling you get when you overwork your body, except you get it behind your eyes.
Feels good.
A reflection system and a cleaner stack
Sidenote: I also built a functional reflection system that works in real time with zero overhead. A sort of MRI for any AI you put into the system. Traces layers, reasoning paths, and thought processes. This is not difficult to do, and I don't understand why the lab guys keep saying they can't do it.
Also, might have designed a parameter enhancing upscaler for 4-bit Quantized weights. Not sure if it'll work yet. I need a cleaner build to know for sure.
Would you believe I'm still having fun?
First things first, though.
Aggressive cleanup on the inference stack tonight.
Then I'm going to bed.
We start clean tomorrow.