ONNX Runtime PR #32315: Why a WebGPU Copy Error Terminated the Entire Process
A bad WebGPU tensor copy should return an error to the caller.
Before ONNX Runtime PR #32315, some copy misuse could instead terminate the entire process.
The internal WebGPU buffer manager correctly rejected operations such as copying a GPU buffer onto itself. It reported that misuse with ORT_ENFORCE, which throws a C++ exception. The exception then crossed a noexcept callback used by ONNX Runtime’s C ABI.
C++ does not allow an exception to escape a noexcept function. The runtime calls std::terminate.
The problem was not that BufferManager rejected the copy; the problem was that the rejection escaped through the wrong interface.
The internal operation and the public boundary used different error mechanisms
Inside the WebGPU provider, GPU-to-GPU copies eventually reach:
BufferManager::MemCpy(...)That function enforces several preconditions. One of them requires the source and destination buffers to be different:
ORT_ENFORCE(
src != dst,
"Source and destination buffers must be different.");Within ordinary C++ code, throwing is an expected way to leave the operation and carry an error message upward.
The plugin-facing data-transfer interface has a different contract.
WebGpuDataTransferImpl::CopyTensorsImpl is installed as a C ABI callback and declared noexcept:
static OrtStatus* ORT_API_CALL CopyTensorsImpl(...) noexceptIts result type already provides the correct error channel:
success
→ return nullptr
failure
→ return OrtStatus*Before the fix, the function handled failures returned as an internal Status, but it did not catch exceptions thrown below that level.
The two error paths therefore behaved differently:
called operation returns non-OK Status
→ convert to OrtStatus
→ caller receives an error
called operation throws
→ exception escapes noexcept
→ std::terminate
→ process endsA lower-level implementation detail determined whether the application received a recoverable status or disappeared.
Why returning OrtStatus* was not enough
The callback’s return type may make it look exception-safe.
It was not.
A function can promise to return an error object and still invoke code that throws before the return statement is reached.
The relevant call chain was:
public tensor-copy request
↓
WebGpuDataTransferImpl::CopyTensorsImpl
↓
shared WebGPU data transfer
↓
BufferManager::MemCpy
↓
ORT_ENFORCE
↓
C++ exceptionThe callback was the last C++ frame before the C ABI. That made it the required translation boundary.
An exception must be caught before it crosses that frame, converted into the ABI’s error representation, and returned normally.
The first repair also had to preserve status semantics
A local try/catch block could prevent std::terminate.
That alone would not guarantee correct behavior.
ONNX Runtime already has a standard exception wrapper for C API entry points:
API_IMPL_BEGIN
...
API_IMPL_ENDThe macro distinguishes several exception categories.
An OnnxRuntimeException becomes an OrtStatus that preserves the exception’s original ONNX Runtime error code. A NotImplementedException maps to ORT_NOT_IMPLEMENTED. Other standard exceptions map to ORT_RUNTIME_EXCEPTION, and unknown exceptions map to ORT_FAIL.
The initial review found that a custom wrapper duplicated this logic but changed its semantics. In particular, it collapsed OnnxRuntimeException into a generic runtime-exception status rather than preserving ex.Code().
The final patch uses the shared wrapper instead.
Consistency matters at an ABI boundary. Callers may inspect both the message and the status code, and two C API callbacks should not translate the same internal exception differently.
The fix separates the noexcept shell from the throwing implementation
The merged callback is intentionally small:
static OrtStatus* ORT_API_CALL CopyTensorsImpl(...) noexcept {
API_IMPL_BEGIN
return CopyTensorsOrThrow(...);
API_IMPL_END
}The original copy logic moves into a helper without noexcept:
static OrtStatus* CopyTensorsOrThrow(...) {
...
}This creates two explicit responsibilities.
CopyTensorsOrThrow performs the operation. It can return an OrtStatus*, and lower-level C++ code can still throw.
CopyTensorsImpl protects the C ABI. It guarantees that any supported exception category becomes an OrtStatus* before control returns to the caller.
The fix does not suppress the error; it translates it into the ABI’s expected result type.
That distinction is important. Swallowing the exception would make the copy appear successful. Converting it preserves the failure while keeping the host process alive.
The regression test checks survival, not only the message
A test that only expects an error message cannot observe a process that has already terminated.
The added regression test creates a WebGPU device tensor and requests a self-copy:
source device tensor
=
destination device tensorThe request reaches the src != dst enforcement in BufferManager::MemCpy.
After the fix, the test expects:
CopyTensor returns a non-OK status
error message contains "must be different"Then it continues using the same device tensor.
The test copies known values from CPU memory into the device tensor, copies them back into another CPU tensor, and verifies that the output equals the input.
That second half proves a separate property:
the misuse was reported
and
the process and transfer path remained usableThe author also reported checking the test’s sensitivity by temporarily removing the exception wrapper. In that form, the test process terminated instead of returning the expected failure, reproducing the original boundary error.
Why the test uses self-copy
The buffer manager can also reject an undersized destination or a buffer that is still mapped.
Self-copy is the cleanest deterministic trigger.
Source and destination resolve to the same WGPUBuffer, so the first enforcement condition fails without depending on allocator bucket sizes, timing, or mapped-state setup.
The review considered an undersized-buffer case as another possible regression. The author did not include it because allocator requests may be rounded up to bucket sizes, making “undersized” unreliable at the layer used by the test.
The self-copy case directly reaches the throwing branch the patch must translate.
Why the sibling callbacks were left unchanged
The same plugin wrapper also contains callbacks for checking whether a copy is supported and for releasing the transfer object.
The PR deliberately limits the change to CopyTensorsImpl.
CanCopyImpl only calls C ABI function pointers in the reviewed path and had no identified throwing operation that required translation.
ReleaseImpl returns void. If an exception were possible there, converting it into an OrtStatus would not be available because the function has no status return channel. Catching and discarding an exception would create a different policy decision.
This patch repairs a confirmed mismatch:
a callback that promises OrtStatus error reporting
calls code that can throw
without translating the exceptionIt does not introduce a general rule that every sibling callback should catch and hide all failures.
The Python reachability boundary
At the time described by the PR, this misuse path was not reachable through the existing Python copy_tensors behavior for WebGPU OrtValues.
The fix was still placed in the shared boundary because another change was preparing to expose supported WebGPU graph-capture I/O and tensor copies through Python.
That scope matters when interpreting impact.
The patch fixes a real C ABI correctness problem in the shared WebGPU data-transfer implementation. It does not establish that ordinary Python users had already been hitting the process termination in released builds.
Repairing the boundary before expanding reachability prevents a latent failure from becoming a public API behavior.
What the patch proves and what it does not
The merged code handles exceptions raised during CopyTensorsOrThrow with ONNX Runtime’s standard C API mapping.
The regression test reaches a known ORT_ENFORCE, verifies a returned error status and message, and performs successful transfers afterward.
The reviewed evidence supports that the self-copy misuse no longer escapes through the noexcept callback.
It does not show that every possible WebGPU transfer failure throws. Some paths still return Status values normally. It also does not prove that CanCopyImpl or ReleaseImpl can never throw under any future implementation.
The restored boundary contract is:
C++ implementation may return Status or throw
↓
noexcept C ABI callback translates both failure forms
↓
caller receives OrtStatus
↓
process remains aliveAt a language boundary, declaring a function noexcept is not an error-handling strategy. Every throwing operation below that boundary still needs an explicit route into the interface’s error type.
Sources
https://github.com/microsoft/onnxruntime/pull/32315
https://github.com/microsoft/onnxruntime/commit/c4231eecc4508248369f1f742a7614c914886fcb
https://github.com/microsoft/onnxruntime/blob/c4231eecc4508248369f1f742a7614c914886