NVIDIA CUTLASS #3017 — Why load() and store() Mapped the Same Tile Coordinate to Different Addresses


CUTLASS #3017, Part 1 of 3 — An asymmetry in address calculation and the patch that fixed it

In February 2026, while reading through NVIDIA CUTLASS’s RegularTileIterator<layout::PitchLinear> implementation, I noticed an odd asymmetry.

load() and store() accepted the same tile coordinate, yet they calculated the corresponding memory address differently.

The difference was a single line.

load() divided the contiguous coordinate by kElementsPerAccess.

store() did not.

When each memory access handled only one element, the difference remained invisible.

But in vectorized paths, where a single access handles multiple elements, the same logical tile coordinate could resolve to two different physical addresses.

I reported the issue as CUTLASS Issue #3017. The missing division was later added to store() in PR #3049.

CUTLASS and tile iterators

CUTLASS is NVIDIA’s open-source library for high-performance CUDA linear algebra.

It provides C++ template abstractions for composing GPU operations such as matrix multiplication from smaller building blocks, including data layouts, memory movement, tile shapes, element types, and execution policies.

A GPU does not process an entire large matrix as one indivisible object.

The matrix is divided into smaller blocks called tiles. Groups of threads load their assigned fragments, perform the computation, and write the results back to memory.

RegularTileIterator is one of the abstractions responsible for calculating which memory locations each thread should read from or write to.

In the PitchLinear specialization where this issue appeared, coordinates are expressed along two primary dimensions:

  • the contiguous dimension, where elements are laid out next to one another in memory

  • the strided dimension, where movement occurs at a fixed interval

When load() and store() receive the same tile coordinate, they should resolve it against the same base address.

Otherwise, the iterator cannot reliably write data back to the location from which the corresponding tile was read.

The two calculations were actually different

The affected load() path calculated the contiguous component of the tile offset like this:

tile_offset.contiguous()
    * Shape::kContiguous
    / ThreadMap::kElementsPerAccess

Before the fix, the corresponding store() path omitted the final division:

tile_offset.contiguous()
    * Shape::kContiguous

Both paths eventually call either load_with_pointer_offset() or store_with_pointer_offset().

Inside those functions, the supplied offset is multiplied by the size of the element type to produce the final byte address.

The downstream byte conversion was the same.

The offset passed into it was not.

When kElementsPerAccess was greater than 1, the same logical tile coordinate could therefore point to different memory locations in the two paths.

The difference becomes obvious with numbers

PR #3049 included a simple example that makes the mismatch clear.

Assume the following:

  • contiguous tile size: 64

  • element type: float

  • size of one float: 4 bytes

  • kElementsPerAccess: 4

load() divides 64 by 4 before converting the result into bytes:

(64 / 4) × 4 bytes
= 64 bytes

Before the patch, store() skipped that division:

64 × 4 bytes
= 256 bytes

For the same logical tile coordinate:

  • load() resolved to a 64-byte offset

  • store() resolved to a 256-byte offset

After the fix, store() applied the same division:

(64 / 4) × 4 bytes
= 64 bytes

The read and write paths once again agreed on the base address.

Why was the problem easy to miss?

kElementsPerAccess represents the number of elements handled by a single memory access.

In many ordinary SIMT configurations, this value can be 1.

When that happens:

64 / 1 = 64

The presence or absence of the division makes no numerical difference.

Even though load() and store() contain different formulas, both still resolve to the same address.

The code was not correct in the general case.

A specific condition was simply hiding the difference.

Once kElementsPerAccess becomes 2, 4, or 8 in a vectorized path, the asymmetry becomes a real address mismatch.

The original issue noted that the problem could remain dormant in common SIMT configurations and become visible only when vectorized access was used.

Part 2 of this series examines that condition in more detail.

How the issue became an actual patch

I opened CUTLASS Issue #3017 on February 11, 2026.

The issue raised two separate concerns.

The first was the address-calculation asymmetry between load() and store() described above.

The second was a unit-contract ambiguity around add_pointer_offset(): the interface was documented in terms of elements, while the affected implementation added the value directly to a uint8_t*, making it behave like a byte offset.

Ten days later, on February 21, another contributor submitted PR #3049 to fix the first problem.

The patch itself was small.

Its central change was adding the missing term to the contiguous offset calculation in store():

/ ThreadMap::kElementsPerAccess

The PR was merged into NVIDIA CUTLASS’s main branch on April 25, 2026, and Issue #3017 was closed in connection with that fix.

One line restored symmetry in the address calculation

Viewed only as a diff, the patch added a single division.

But what it restored was not merely an arithmetic expression.

It restored a shared contract between load() and store():

Given the same logical tile coordinate, both paths must resolve to the same base memory location.

Reading and writing are opposite operations.

Their interpretation of the address space should not be opposite as well.

The coordinate system must remain consistent even when the access width changes.

This kind of asymmetry is difficult to notice when reading only one side of the implementation.

The load() formula looks reasonable by itself.

The old store() formula is not syntactically invalid either.

The mismatch becomes visible only when both paths are placed side by side and tested against the same input.

What the patch fixed — and what should still be considered separately

PR #3049 fixed a clearly defined problem:

  • it added the missing kElementsPerAccess division to store()

  • it aligned the address calculations used by load() and store() for the same tile coordinate

  • it closed the address mismatch that could appear in vectorized paths

However, the patch did not redesign the broader Element-versus-Byte contract raised in the same issue.

If an internal caller converts an offset into bytes before passing it to a function backed by a byte pointer, the current internal path may still produce the correct address.

But if the public description says the argument is expressed in elements while the implementation behaves as though it were expressed in bytes, how another caller should interpret that value remains a separate question.

That distinction is the subject of Part 3.

Closing an issue does not necessarily mean that every abstraction concern discussed within it was resolved by the same pull request.

The same coordinate should be interpreted under the same contract

This problem was not found inside an unusually complex GPU equation.

It appeared when two paths that should have been symmetric were compared side by side.

load() and store() received the same tile coordinate.

Only one of them accounted for the access width.

The common condition kElementsPerAccess = 1 could hide that difference for a long time.

Vectorization did not create the bug.

It exposed an asymmetry that was already present.

When two paths read from and write to the same logical coordinate, their address calculations must follow the same contract.

If a difference is intentional, the reason should be declared in the code or documentation.

If an undeclared difference disappears only under a particular condition, it may be a boundary error that has not yet become visible.

Related material

This is Part 1 of a three-part series on CUTLASS #3017.

https://resoneticlab.blogspot.com/2026/08/nvidia-cutlass-kelementsperaccess-why.html

Part 2 examines how kElementsPerAccess = 1 masked the problem, and why vectorized access exposed the hidden addressing error.

#NVIDIA #CUTLASS #CUDA #GPUProgramming #OpenSource #CodeAnalysis #SoftwareArchitecture

Popular posts from this blog

AMD Is Trying to Turn Kernel Diversity Into One Software Asset

The first broad integration was rolled back, but the direction of the software stack is difficult to ignore The GPU software race is often described as a contest over who can write the fastest kernel. That is only part of the problem. A company may have several teams capable of producing high-performance matrix-multiplication kernels. Some may work through template libraries. Others may use compiler-generated code, handwritten assembly, or model-specific optimization stacks. The existence of those kernels does not automatically create one coherent platform. They still need to be: discovered validated matched to the correct problem given the correct arguments launched through a stable runtime and delivered through a reliable software package Recent public code changes inside AMD’s software ecosystem showed an attempt to connect these layers more directly. The central idea was to allow hipBLASLt’s GEMM selection system to choose kernels created by multiple internal kernel technologi...
Read more

PyTorch #188031 — Why Did Forward Use 64-Bit Indexing While Backward Still Used uint32_t?

PyTorch Spatial Softmax, Part 1 of 3 — An asymmetric indexing contract and the patch that aligned it A recent PyTorch change fixed a 64-bit indexing problem in the spatial softmax backward path. The bug was not in the softmax formula itself. It was in the width of the integer used to calculate memory locations. For the same large tensor, the forward path already selected either 32-bit or 64-bit indexing based on the input size. One backward path, however, continued to calculate sizes, strides, and offsets using uint32_t . That created a possible split in behavior: Forward → 64-bit address calculation → accesses the intended elements Backward → 32-bit address calculation → offset wraparound → gradients read from or written to different locations A tensor could therefore complete the forward pass correctly while the corresponding backward kernel could no longer represent every address in that tensor. The changes described in PyTorch #188031 were later landed in PyTorch’s main branch. S...
Read more