Shmoosh: a faster packed-attention path
An opt-in TF32 mode makes Shmoosh’s packed-attention path faster while keeping the fidelity policy explicit. Here are the measurements and the remaining tradeoffs.
Lynn Cole / 标签
Shmoosh
A thread through the work.