This weird custom transformer I just wrote in Go *almost* works. It's totally new science around compute and memory management. We're not sure why it's broken yet. Results are nonsense, but the math is sound. All of the things are talking to all of the other things, and the slick custom tokenizer works on round trips. Looking at logits, softmax, and how I'm shaping the data. But man, when this thing is done... if I can finish it, it's going to be VERY cool.
So what have I done so far?
Wrote a complete Simd kernel in go and assembly (see my GitHub, it screams).
Wrote a CPU specific tokenizer
Create a rope implementation
Wrote a new fp8 guff loader
Invented a new memory management scheme
And wrote... probably two thirds of a go native transformer.
And all but the last piece are fully functional.
I mean, you guys, this isn't that hard.
Oh god damnit... it's the vocabulary. duh.
Okay, fixed the vocab issue. Tokenizer is working vocabs are loading... still nonsense! Ha! But, this is variable nonsense. So checking our norms... we're between (mean=3.84, max=15.3) mwhahaha *poof*. Almost there guys.
Narrowing it down. Looks like a... kv cache initialization issue? Maybe? Ve shall see.
Fuck, is it working?
If it is... it's slooooooooowwww.
This is fun.
I highly recommend this activity.
22 layers loaded, 47 seconds... and we have <uncs>!
hahah,
Still not there... but we're getting there.
Oh....
I should probably think about layer normalization, huh
So we just fixed a cluster of problems around q8 kv-cache and rsnorms. And... random words baby!
It's better than nan errors
And now... we're in the land of multi-token debugging. *spooky*
Lol, duh. Attention layer is locked.
Yeah that would do it.
I'm breaking my models, because I forgot to track sequence position. See? This is why we test, people.
Prompt: Once upon a time there was a
Response: rizona zahl ała ublic inoltre veget athedral Italy
Nice!
Still working on it
Looks like the forward pass is screwed up again.