ONNX Runtime PR #32666: Why WebGPU Inputs Could Be Cleared After Upload

ONNX Runtime PR #32666 fixes a single-tensor copy path that received an execution stream and discarded it before calling a plugin execution provider.

The missing argument could change the order of operations on the destination buffer. Issue #32643 describes a WebGPU input-binding failure in which a buffer-clear command was queued during allocation, but the subsequent CPU-to-GPU upload reached the plugin without its stream. The upload could execute first. The pending clear then replaced the uploaded input with zeros.

The fix forwards the caller's stream through plugin_ep::DataTransfer::CopyTensorImpl(), matching the existing batched-copy path. Synchronous calls continue to supply a null stream.

The change merged on September 17, 2026. Its production edit is two lines; the accompanying test checks the argument received by the plugin callback.

The copy needed to follow the allocation-time clear

An execution provider, or EP, implements operations for a particular execution backend. A plugin EP receives tensor-transfer requests through the OrtDataTransferImpl::CopyTensors callback.

That callback accepts arrays of source tensors, destination tensors, and execution streams. The API contract allows a provider to use a supplied stream for asynchronous copies and calls for synchronous copying when a stream is unavailable.

In the reported WebGPU case, BindInput() connected a CPU tensor to an input consumed on the GPU. Allocating the destination also queued a command to clear its buffer. The upload needed to follow that initialization work:

clear destination buffer
        ↓
upload CPU input into that buffer
        ↓
consume the uploaded input

Removing the stream from the copy request removed the context needed to preserve that relationship. The issue describes this possible execution order:

upload CPU input
        ↓
execute the previously queued clear
        ↓
consume a zeroed input

For an illustrative input [2, 3, 5], clearing before uploading leaves [2, 3, 5]; uploading before clearing leaves [0, 0, 0]. Both operations can finish successfully while their order changes the result.

Synchronous completion of an upload alone does not establish that an earlier clear on an omitted stream has already executed. The relevant dependency includes both operations.

This failure sequence is reported in the linked issue. The public patch does not include a WebGPU command trace or an end-to-end reproduction of the zeroed input.

The stream was lost inside the single-copy adapter

The adapter's header already distinguished its two single-tensor entry points:

Status CopyTensor(const Tensor& src, Tensor& dst) const override {
  return CopyTensorImpl(src, dst, nullptr);
}

Status CopyTensorAsync(const Tensor& src, Tensor& dst,
                       Stream& stream) const override {
  return CopyTensorImpl(src, dst, &stream);
}

The asynchronous caller therefore supplied the stream correctly.

Inside CopyTensorImpl(), however, the parameter was marked unused. The implementation wrapped the source and destination tensors in OrtValue objects, then constructed this callback argument:

OrtSyncStream* stream_ptr = nullptr;

The callback was invoked with the address of that pointer:

impl_.CopyTensors(&impl_, &src_ptr, &dst_ptr, &stream_ptr, 1);

The distinction between streams and streams[0] matters here. The callback received a valid one-element array address, but its sole stream entry was null. A provider inspecting that entry could not recover the execution context passed into CopyTensorAsync().

The merged implementation names the parameter and uses it:

OrtSyncStream* stream_ptr =
    reinterpret_cast<OrtSyncStream*>(stream);

The callback invocation, tensor wrappers, no-op tensor deleters, and status conversion remain unchanged. The patch restores information that the adapter already possessed.

The batched path already preserved the same information

DataTransfer::CopyTensors() builds a stream array alongside its source and destination arrays. For each pair, it forwards:

reinterpret_cast<OrtSyncStream*>(src_dst_pairs[i].src_stream)

The single-copy implementation is labeled as an optimized version of that path. It uses local wrappers and pointers for one transfer instead of constructing the vectors used for a batch.

That specialization should change preparation overhead while preserving the callback's meaning. Before this fix, selecting the single-copy entry point also erased the stream.

The existing batched conversion and the plugin stream header establish the representation used at this boundary: OrtSyncStream is the C API handle for ONNX Runtime's Stream abstraction. This is not a conversion of the backend's raw native stream handle.

The distinction is visible in the regression test. Its Stream object has a null native handle, but the object itself has an address. An asynchronous call must forward that object address; a synchronous call must forward a null pointer.

No new stream is created, and the patch adds no stream flush or global synchronization. The provider receives the original context and remains responsible for implementing the copy under its API contract.

What the regression actually checks

The new PluginCopiesForwardStreams test constructs a small fake OrtDataTransferImpl. Its callback records the number of tensors submitted and the first stream pointer it receives, then returns success.

The test calls the real plugin adapter three times:

Adapter call	Expected callback stream
`CopyTensorAsync(source, destination, stream)`	The supplied `Stream` object's address
`CopyTensor(source, destination)`	Null
`CopyTensors({{source, destination, &stream}})`	The supplied `Stream` object's address

The cumulative tensor counts must be one, two, and three. These checks establish that each entry point invokes the callback and that synchronous and asynchronous stream handling remain distinct.

Under the old implementation, the first pointer comparison would fail: the asynchronous single-copy path always supplied null. That conclusion follows directly from the old assignment and the new assertion; it is not a locally executed before-and-after result from this review.

The fake callback never copies tensor contents. The tensors use a CPU allocator, and the test performs no GPU command submission. Its evidence is callback argument forwarding, not successful asynchronous data transfer on WebGPU.

The batched entry point is exercised with one pair. The new test does not cover a multi-element batch with different streams or a mixture of null and non-null entries. It also calls the adapter directly rather than exercising provider selection through DataTransferManager or input binding through a session.

Those limits keep the result precise: the test targets the location where the stream was lost.

What this means for plugin implementers

For a plugin copy callback, inspect the stream associated with each transfer, not just whether the outer stream-array pointer exists. In this bug, the array was present while the relevant entry was null.

When a copy path works in batches but fails for one tensor, compare all arguments reaching the provider, including execution context. An optimized wrapper can preserve tensor addresses and sizes while dropping the information that orders their use.

For this reported failure, the scoped static-review conclusion is that the patch repairs the adapter's stream-forwarding contract. It also preserves the existing null-stream behavior of synchronous calls.

The reviewed discussion contains a human approval and a request to consider release-branch backports. It does not establish which distributed package first contains the fix. A deployment assessment should check the actual ONNX Runtime build rather than equate a merge to main with an installed release.

This review did not build ONNX Runtime, run the new test, or reproduce the WebGPU failure. No performance measurements accompany the patch. The strongest evidence is the direct correspondence between the discarded parameter, the restored callback argument, and the regression assertion.

The practical lesson is specific: a single-tensor fast path must preserve the caller's stream just as the batched path does. The bytes being copied and the order in which they become usable belong to the same transfer request.