The letter

One letter a week, in your inbox.

The signal of the week, what shipped, what to try, and the editor's note. No tracking, no ads, nothing else.

We keep your address, your language and the date you joined, nothing else. Every letter has a one-click unsubscribe link that deletes the record.

← Accept All   Archive
PyTorch BlogTools

Low Precision Flash Attention 4: End-to-End Block-Scaled Attention for Blackwell

September 16

TL;DR We extend FlashAttention-4 [1] with MXFP8 forward and backward, reaching 2.85 PF/s forward and 2 PF/s backward on LLM shapes.

Chips & compute
Read at PyTorch Blog ↗

Related

More from PyTorch Blog on Accept All.