PyTorch 2.13 Changed the Boundary, Not the Behavior — So Why Did Some C++ Extensions Break?

How PR #179063 replaced StorageImpl’s copy-on-write-specific path with a general materializer hook

PyTorch 2.13 includes a change that replaced copy-on-write-specific handling inside StorageImpl with a general materializer hook.

According to the PyTorch maintainers, the behavior of copy-on-write itself did not change.

Yet some precompiled C++ and CUDA extensions built against an earlier version of PyTorch could no longer be loaded after the update.

How can the behavior stay the same while the binary still breaks?

The answer is that the meaning of the feature remained intact, but the boundary that external code depended on had changed.

What copy-on-write does

Imagine two tensors sharing the same underlying memory.

As long as both tensors only read the data, there is no need to keep two identical copies. They can refer to the same storage.

The problem begins when one of them tries to modify the data.

If the shared memory were changed directly, the other tensor would see that change as well. To prevent this, a separate copy is created only when a write is about to occur.

That is copy-on-write, commonly shortened to COW.

In simple terms:

Share the data while it is only being read.
Create a private copy when someone needs to write to it.

Inside PyTorch, StorageImpl is a core object responsible for managing state related to a tensor’s underlying storage.

Before this patch, StorageImpl knew whether a storage object used COW and could invoke COW-specific logic before mutable access.

In other words, the core storage object knew the internal details of one particular memory strategy.

What changed in PyTorch 2.13

PR #179063 moved that COW-specific knowledge out of StorageImpl.

The previous design exposed COW-specific entry points such as:

  • StorageImpl::is_cow()

  • StorageImpl::maybe_materialize_cow()

  • cow::materialize_cow_storage()

In PyTorch 2.13, that dedicated path was replaced by a more general MaterializeFn hook.

Under the new design, StorageImpl no longer needs to determine whether a storage object is using copy-on-write.

Instead, a backend can register a materializer when some form of preparation is required before the storage is modified.

The registered function runs once, when mutable access is first requested, and is then cleared.

Copy-on-write becomes one consumer of this general mechanism rather than a special case built directly into the central storage object.

The existing COW semantics — including lazy cloning, shared references, and copying immediately before mutation — remain in place.

What changed was the entry point through which that behavior was reached.

Why replace a dedicated path with a general hook?

If copy-on-write were the only concern, PyTorch could have kept the old specialized logic.

But other backends may also need to perform work immediately before storage becomes mutable.

For example, a backend might need to:

  • finalize a deferred memory allocation

  • move data from a device to host memory

  • wait for an asynchronous operation to complete

  • turn a placeholder buffer into real storage

  • materialize symbolic or deferred tensor data

If every new case required another condition inside StorageImpl, the core object would gradually accumulate knowledge about many different backends and memory strategies.

The patch moved in the opposite direction.

StorageImpl now provides a common interception point. Backends that require special preparation can register their own function through that interface.

Responsibility moved from a central object that understood every special case to individual backends using a shared extension point.

The PR describes potential uses involving accelerator backends, delayed allocation, device-to-host movement, and graph compilers operating in eager mode.

That does not mean every possible backend has already adopted the new mechanism. The patch created an extension point; it did not prove that all of its potential use cases are already deployed.

If the behavior stayed the same, why did an extension break?

The problem was not the behavior of copy-on-write.

It was the old C++ symbol.

During the change, the previous materialize_cow_storage entry point was removed or replaced by the new mechanism.

An extension rebuilt against the new PyTorch source can detect the change during compilation and migrate to the new API.

A precompiled binary is different.

That binary still contains a reference to the symbol that existed when it was built. If the new PyTorch library no longer exports that symbol, the extension can fail at import time with an undefined-symbol error.

A downstream report from an NVIDIA PyTorch container update showed this exact failure mode. A precompiled FlashAttention extension attempted to resolve the previous materialize_cow_storage symbol against the updated PyTorch build.

In that environment, roughly 600 FlashAttention tests failed with an undefined symbol error.

The affected extension had to be rebuilt against the new PyTorch version.

This does not mean every FlashAttention installation encountered the same problem. It was an ABI compatibility failure observed in a specific combination of an older precompiled extension and a newer PyTorch build.

Source compatibility and binary compatibility are different

The impact becomes clearer when source code and precompiled binaries are considered separately.

Rebuilding from source

Code that directly calls removed methods such as is_cow() or maybe_materialize_cow() may fail during compilation.

The developer receives an error and can migrate the code to the new materializer API.

Loading an existing binary

A precompiled extension does not go through that compilation step.

Instead, it can fail later, during loading or import, because the symbol embedded in the binary no longer exists in the current PyTorch library.

The same patch can therefore have different effects on:

  • source compatibility

  • binary compatibility

  • user-visible feature behavior

Preserving the semantics of copy-on-write did not guarantee that previously compiled binaries would continue to load.

Was it really an internal implementation detail?

During review, the PR was initially labeled not user facing.

At first glance, it looked like a cleanup of PyTorch’s internal storage implementation.

That interpretation changed after a real downstream undefined-symbol failure was identified.

The PR was subsequently marked as a backward-compatibility-breaking change, the not user facing label was removed, and the migration was documented in the PyTorch 2.13 release notes under backward-incompatible changes.

This raises an important question:

When does an internal implementation detail stop being truly internal?

A symbol may not be part of a broadly documented public API. But if external projects are already linked against it, and removing it breaks deployed binaries, it has begun to function as a contract in practice.

The declared visibility of a boundary is not always enough to determine its real impact.

You also have to ask who is already depending on it.

Who may be affected?

Most users running ordinary Python model code are unlikely to notice this change directly.

The PyTorch release notes describe it as a C++-specific change.

The affected group is more likely to include:

  • external extensions that call COW-related StorageImpl symbols directly

  • out-of-tree backends coupled to PyTorch internals

  • custom accelerator backends

  • precompiled C++ or CUDA wheels built against an older PyTorch version

  • binaries linked to the removed materialize_cow_storage symbol

In PyTorch 2.13, code using the previous COW-specific path needs to migrate to the new materializer boundary, including APIs such as set_materializer(), has_materializer(), and clear_materializer().

The feature did not change. The boundary did.

This patch did not introduce copy-on-write to PyTorch.

It preserved the existing behavior while changing who owns the knowledge of that behavior, who invokes it, and where other backends are allowed to intervene.

StorageImpl became less dependent on COW-specific logic and gained a more general extension point.

That is a cleaner and more flexible architecture for future memory strategies and backend integrations.

But improving the internal design did not eliminate the compatibility cost of removing the old C++ boundary.

Some external binaries were already connected to that boundary and therefore had to be rebuilt.

The broader lesson is that the phrase “internal implementation detail” is not enough to determine compatibility risk.

Even an undocumented symbol can become a practical contract once downstream software begins to depend on it.

The size of a patch is therefore not defined only by the number of changed lines.

It is also defined by how much responsibility the patch moved — and how wide the affected dependency boundary had already become.

Related material

This article was written after reviewing the public code changes, discussion history, and the official PyTorch 2.13 release notes.

#PyTorch #OpenSource #CodeAnalysis #SoftwareArchitecture #CPlusPlus #AIInfrastructure

Popular posts from this blog

AMD Is Trying to Turn Kernel Diversity Into One Software Asset

The first broad integration was rolled back, but the direction of the software stack is difficult to ignore The GPU software race is often described as a contest over who can write the fastest kernel. That is only part of the problem. A company may have several teams capable of producing high-performance matrix-multiplication kernels. Some may work through template libraries. Others may use compiler-generated code, handwritten assembly, or model-specific optimization stacks. The existence of those kernels does not automatically create one coherent platform. They still need to be: discovered validated matched to the correct problem given the correct arguments launched through a stable runtime and delivered through a reliable software package Recent public code changes inside AMD’s software ecosystem showed an attempt to connect these layers more directly. The central idea was to allow hipBLASLt’s GEMM selection system to choose kernels created by multiple internal kernel technologi...
Read more

PyTorch #188031 — Why Did Forward Use 64-Bit Indexing While Backward Still Used uint32_t?

PyTorch Spatial Softmax, Part 1 of 3 — An asymmetric indexing contract and the patch that aligned it A recent PyTorch change fixed a 64-bit indexing problem in the spatial softmax backward path. The bug was not in the softmax formula itself. It was in the width of the integer used to calculate memory locations. For the same large tensor, the forward path already selected either 32-bit or 64-bit indexing based on the input size. One backward path, however, continued to calculate sizes, strides, and offsets using uint32_t . That created a possible split in behavior: Forward → 64-bit address calculation → accesses the intended elements Backward → 32-bit address calculation → offset wraparound → gradients read from or written to different locations A tensor could therefore complete the forward pass correctly while the corresponding backward kernel could no longer represent every address in that tensor. The changes described in PyTorch #188031 were later landed in PyTorch’s main branch. S...
Read more

NVIDIA CUTLASS #3017 — Why load() and store() Mapped the Same Tile Coordinate to Different Addresses

CUTLASS #3017, Part 1 of 3 — An asymmetry in address calculation and the patch that fixed it In February 2026, while reading through NVIDIA CUTLASS’s RegularTileIterator<layout::PitchLinear> implementation, I noticed an odd asymmetry. load() and store() accepted the same tile coordinate, yet they calculated the corresponding memory address differently. The difference was a single line. load() divided the contiguous coordinate by kElementsPerAccess . store() did not. When each memory access handled only one element, the difference remained invisible. But in vectorized paths, where a single access handles multiple elements, the same logical tile coordinate could resolve to two different physical addresses. I reported the issue as CUTLASS Issue #3017. The missing division was later added to store() in PR #3049. CUTLASS and tile iterators CUTLASS is NVIDIA’s open-source library for high-performance CUDA linear algebra. It provides C++ template abstractions for composing GPU ope...
Read more