Resonetic Lab | AI · Code · Market

Triton Blackwell NVFP4 × Dense Matmul, Part 3 of 3 — From plain dtype coverage to a routed, scale-swizzled persistent matrix with scatter, gamma, and K-ragged variants The first two articles in this series examined an invalid tile decision in Triton’s Blackwell matmul planner. The affected path combined: NVFP4-scaled activation × dense BF16 or FP16 operand + persistent execution + an expanded operand staged in TMEM The automatic planner could select: block_n = 256 That tile looked reasonable for: N = 256 because one tile could cover the full output width. But the persistent lowering needed Tensor Memory for two live structures: output accumulator + expanded operand The combined allocation exceeded Blackwell’s 512-column TMEM limit. PR #11171 repaired the automatic plan by capping the default: block_n → no larger than 128 for the affected persistent NVFP4 × dense path. The code change was small. The regression test was not simply: Run one 256 × 256 × 128 matmul. Two top-level Ca...

Triton Blackwell NVFP4 × Dense Matmul, Part 1 of 3 — When a persistent accumulator and an expanded operand competed for the same TMEM budget Larger matrix-multiplication tiles are often associated with better GPU performance. A larger tile can: reduce the number of program instances increase arithmetic work per tile improve data reuse amortize scheduling and launch overhead But a tile is not free. Every increase in tile size also increases the amount of state that the kernel must keep alive. On NVIDIA Blackwell, some persistent matrix-multiplication paths place more than the accumulator in Tensor Memory, or TMEM. They may also stage an expanded operand there. That creates a hard resource question: Can the accumulator tile and the additional operand representation fit inside the same 512-column TMEM budget? Triton PR #11171 fixed a case where the answer was no. The affected path combined: Blackwell GPU persistent matmul NVFP4-scaled left-hand activation dense BF16 or FP16 right-ha...

Triton Blackwell NVFP4 × Dense Matmul, Part 2 of 3 — Planner-owned defaults, explicit overrides, and sharing one TMEM rule without sharing every tuning policy Part 1 examined why Triton’s automatic planner could select an impossible persistent matmul tile on Blackwell. The affected path combined: NVFP4-scaled activation × dense BF16 or FP16 operand + persistent execution + an expanded operand staged in TMEM The default planner could select: block_n = 256 But the kernel needed Tensor Memory for both: the output accumulator and the expanded operand The combined request exceeded Blackwell’s 512-column TMEM limit. In the validation matrix added by PR #11171: 8 executions requested 640 columns 4 executions requested 576 columns All twelve failed with Tensor Memory exhaustion. The fix capped the automatic N tile: default block_n → no larger than 128 The same twelve generated cases then passed. A blunt implementation could have done one of two things: Force block_n = 128 for every re...