FlashMask

PaddlePaddle releases FlashMask as a standalone attention library for long-context training, with PyTorch support

PaddlePaddle published this on Oct 16, 2026 (source: github.com).

PaddlePaddle published FlashMask as a standalone repository: an attention kernel library for flexible sparse masks that encodes a 2D mask as per-column row ranges, cutting mask memory from O(N^2) to O(N), with Hopper and Blackwell kernels written in CuTe DSL and both PaddlePaddle and PyTorch interfaces. PaddlePaddle says it runs up to 1.55x faster than FlashAttention-4 and supports context parallelism with load balancing and compute-communication overlap. The FlashMask-3 paper, which the team says was accepted at NeurIPS 2026, reports 45-141% higher TFLOPs than the original FlashMask on an H100 across 12 masks.