If anyone's following this, we hit almost 90% of theoretical with this latest version of the side project over the weekend.
This time I wrote the whole thing with Fortran (but almost entirely without all caps yelling at the compiler).
It's actually much simpler to write something like this in a language like Fortran than it is in C or Go, Assembly, or Python. The benchmarks speak for themselves. Feel free to peer review, or show me the error of my ways.
GPU and mesh networking are still mostly mocked, but I labelled all the mocks. Metal support is coming next, because everyone I know wants to test this on a macbook. Arm64 and Nvidia/AMD GPU support will be in this weekend.
Take a look at the way memory management works. The same kinds of tricks should work on GPU, because they have the same bottleneck, we just... don't really think about it. Far as I can tell, this is the only sane way to do it.
Heterogeneous device and node orchestration are the answer to what ails ya.
I am convinced of this obvious reality.
Name of the project is changing next weekend, because everybody calls everything Sparkle.