ONNX Runtime #32048 — Why WebGPU Reused the Wrong Shader When the Cache Key Omitted the Activation

ONNX Runtime WebGPU Pipeline Cache, Part 1 of 3 — Generated WGSL, incomplete program identity, and a silent wrong result where the first compiled activation won

The tensor shapes were the same.

The generated shader was not.

ONNX Runtime’s WebGPU execution provider can compile a small matrix multiplication through:

MatMulNaiveProgram

That program does more than multiply two matrices.

When it is used by a fused convolution path, it can also embed an activation directly into the generated WGSL.

MatMul

→ optional bias

→ activation

→ output

For one graph, the generated shader may contain ReLU:

value = max(value, 0)

For another, it may contain Sigmoid:

value = 1 / (1 + exp(-value))

Those are different programs.

Before ONNX Runtime PR #32048, however, the pipeline cache hint for MatMulNaiveProgram did not include the activation.

It contained values related to:

input packing

output packing

number of results computed per invocation

but not:

which activation had been compiled into the shader

Two otherwise identical operations could therefore produce:

different WGSL

but

the same pipeline cache key

The first shader compiled under that key could then be reused for the second operation.

Conv + ReLU
        ↓
compile MatMulNaive shader with ReLU
        ↓
store pipeline under key K

Conv + Sigmoid
        ↓
request MatMulNaive pipeline with the same key K
        ↓
receive previously compiled ReLU shader

No compilation error was required.

No validation warning was required.

The reused pipeline was a valid WebGPU shader.

It was simply the wrong shader for the second operation.

The result could therefore be numerically incorrect without any visible runtime failure.

PR #32048 repaired the identity contract by adding the activation serialization to the cache hint at both MatMulNaiveProgram call sites.

Generated shader depends on activation

→ pipeline key now depends on activation

The same patch also closed two neighboring identity gaps involving activation-parameter serialization and layout-dependent bias indexing. Those are covered separately in Parts 2 and 3.

The first defect is the simplest form of the problem:

A compiled-program cache must distinguish every value that changes the generated program.

Why WebGPU uses a pipeline cache

WebGPU does not need to compile the same WGSL program from scratch for every invocation.

A runtime can create a compute pipeline once and reuse it when another operation needs an equivalent shader.

Conceptually:

Program description

        ↓

generate WGSL

        ↓

compile WebGPU pipeline

        ↓

store pipeline in cache

Later:

Equivalent program description

        ↓

calculate same cache key

        ↓

reuse existing compiled pipeline

This avoids repeated shader compilation.

The benefit is especially useful when:

one model runs the same operation many times,
several graph nodes share the same shader structure,
several sessions share one WebGPU context,
or model initialization creates many pipelines.

But the reuse is correct only when:

same cache key

means:

same compiled program semantics

If two program descriptions generate different WGSL while receiving the same key, the cache cannot know which one the caller intended.

The cache is doing exactly what it was asked to do.

The identity supplied by the program was incomplete.

A pipeline key does not need every runtime value

Not every input to an operation belongs in the pipeline key.

Suppose the same matrix-multiplication shader runs with different tensor contents.

Run 1

A values:
different

B values:
different

Run 2

A values:
different again

B values:
different again

The same compiled WGSL can still be valid.

The tensor values are runtime data.

They travel through buffers.

They do not change the shader source.

Likewise, dimensions passed through uniforms may not require a new pipeline when the generated code remains the same.

The useful distinction is:

Runtime state

Changes what one invocation computes

but

does not change the compiled WGSL

Examples can include:

tensor contents,
uniform values,
dispatch counts,
or dynamic dimensions supported by the same shader.

Compilation state

Changes the shader text,
specialized constants,
resource declarations,
or indexing logic

Examples can include:

activation kind,
embedded activation parameters,
compile-time layout decisions,
vector widths,
workgroup size,
and code-generation branches.

A pipeline cache key must cover the second category.

It does not need to serialize every byte in the first.

The activation was compilation state

MatMulNaiveProgram stores an Activation object as part of the program instance.

When the shader is generated, the program calls:

GetActivationSnippet(
    activation_,
    "output_value_t",
    "output_element_t"
);

That function returns WGSL code according to the selected activation.

ReLU

Conceptually:

value = max(
    value,
    output_value_t(output_element_t(0.0))
);

Sigmoid

Conceptually:

value =
    output_value_t(output_element_t(1.0))
    /
    (
        output_value_t(output_element_t(1.0))
        +
        exp(-value)
    );

Tanh

value = tanh(value);

Parameterized activations can embed constants as well.

Clip

HardSigmoid

LeakyRelu

The activation is not selected by a runtime branch inside one universal shader.

Its code is inserted while the WGSL string is being built.

Activation object
        ↓
GetActivationSnippet()
        ↓
literal WGSL text
        ↓
compiled pipeline

Therefore:

different activation
→ different shader

The activation belonged in the pipeline identity.

The old cache hint did not include it

Before the patch, the small matmul path constructed its cache hint approximately like this:

.CacheHint(
    std::to_string(components),
    std::to_string(a_components),
    std::to_string(output_number)
)

Those fields describe important code-generation choices.

`components`

How many values of the N dimension are processed together.

`a_components`

How many A values are loaded together.

`output_number`

How many output rows one invocation computes.

These values affect the generated loop and indexing code.

They belonged in the key.

But the hint did not include:

activation kind

even though the program’s WGSL generator used it.

The pre-fix identity was effectively:

MatMulNaive

+

packing configuration

+

tensor metadata

+

dispatch and uniform structure

The shader identity was actually:

MatMulNaive

+

packing configuration

+

tensor metadata

+

dispatch and uniform structure

+

activation

The key represented only the first expression.

The rest of the cache key could not recover the missing fact

ONNX Runtime’s WebGPU program key contains more than the custom hint.

It also includes information such as:

program name

workgroup size

requested subgroup size

dispatch dimensionality

uniform lengths

selected input metadata

selected output metadata

index rank

That protects many shader-generation dependencies.

But none of those fields necessarily distinguishes:

ReLU

from:

Sigmoid

when both operations otherwise use the same:

input types,
input ranks,
tensor shapes,
output types,
output ranks,
packing widths,
and dispatch structure.

The activation is not implied by the tensor shape.

same matrix dimensions
≠
same post-matmul activation

It is not implied by the program name either.

Both use:

MatMulNaive

It is not implied by the number of uniforms.

Both activation snippets can be generated without introducing a different runtime uniform layout.

The cache key had no remaining field from which the missing activation could be reconstructed.

This was an identity collision, not a hash-function failure

The word “collision” can suggest that two different key strings happened to produce the same hash.

That was not the central problem here.

The two program configurations were assigned the same semantic key before any low-level hash collision needed to occur.

Program A

WGSL:
MatMul + ReLU

Program B

WGSL:
MatMul + Sigmoid

Old key construction:

Program A key
=
K

Program B key
=
K

The cache did not confuse two accurately described programs.

The caller described both programs as the same cache entry.

A stronger hash function would not help.

same input key material

→ same cache identity

The missing state had to be added before hashing or lookup.

How a convolution reached `MatMulNaiveProgram`

The regression was not built from a standalone MatMul node alone.

It used a small 1×1 convolution.

A 1×1 convolution with:

unit stride

no padding

appropriate layout and shape

can be lowered to matrix multiplication.

The tested shape was designed so that the resulting matrix dimensions selected the naive WebGPU implementation.

The dispatch condition in the WebGPU matmul path is:

N < 8

and

K < 8

The regression used convolution weights shaped so that the lowered matmul had:

N = 6

K = 4

Therefore:

6 < 8

and

4 < 8

and the path entered:

MatMulNaiveProgram

The graph optimizer also fused the activation into the convolution.

The final execution path was approximately:

Conv

+

activation node

        ↓

fused Conv carrying activation metadata

        ↓

1×1 convolution lowered to small matmul

        ↓

MatMulNaiveProgram

        ↓

activation embedded into WGSL

This was the exact path whose cache key lacked the activation.

The minimum counterexample needed two shaders in one cache

A single operation cannot expose this class of wrong reuse.

Suppose a graph contains only:

Conv + ReLU

The runtime calculates the incomplete key and compiles a ReLU shader.

requested shader
=
cached shader

The result is correct.

Now suppose a separate process runs only:

Conv + Sigmoid

A fresh cache compiles a Sigmoid shader.

Again:

requested shader
=
cached shader

The result is correct.

The defect appears only when two distinct shader configurations occupy the same cache lifetime.

Configuration A
→ creates cache entry K

Configuration B
→ requests cache entry K

The regression therefore placed two convolutions in one graph.

Conv A
→ ReLU

Conv B
→ Sigmoid

Their relevant matrix dimensions were intentionally equivalent.

Their activations were intentionally different.

That made the key collision observable.

The first compiled shader could define the second operation

Assume the ReLU operation is encountered first.

Step 1

Generate ReLU WGSL

Calculate old key K

Cache miss

Compile ReLU pipeline

Store ReLU pipeline under K

Then the Sigmoid operation is processed.

Step 2

Sigmoid configuration requests old key K

Cache hit

Return pipeline already stored under K

The returned pipeline contains:

ReLU code

not:

Sigmoid code

The second operation can therefore produce:

ReLU(matmul output)

where the graph requires:

Sigmoid(matmul output)

Reverse the graph order:

Sigmoid first

ReLU second

and the Sigmoid shader may become the reused pipeline instead.

The merged regression exercises both orderings.

ReLU, then Sigmoid

Sigmoid, then ReLU

This prevents the test from accidentally protecting only one cache insertion order.

Why the result could be silently wrong

The reused pipeline remains structurally valid.

It can have:

compatible bindings,
compatible input types,
compatible output types,
compatible dispatch dimensions,
and compatible tensor shapes.

WebGPU does not know that the graph-level activation has changed.

It receives one already compiled pipeline whose resource interface fits the request.

No buffer necessarily has the wrong size.

No WGSL parser necessarily fails.

No device validation error necessarily occurs.

The incorrect behavior is inside the shader’s arithmetic.

expected:
Sigmoid

executed:
ReLU

The operation can complete successfully and return a tensor with plausible values.

That makes this class of cache error more difficult to diagnose than a compilation crash.

Runtime status
→ success

Tensor shape
→ correct

Tensor dtype
→ correct

Tensor values
→ wrong

The test verifies that both activations were actually fused

A regression can appear to test a shader path without reaching it.

For example, if the graph optimizer failed to fuse one activation:

Conv A
→ MatMulNaive without activation

Activation A
→ separate operator

then MatMulNaiveProgram would not embed that activation.

The cache-key defect would not be exercised.

The regression inspects the transformed graph and confirms that both activation kinds were absorbed into their convolutions.

Conceptually:

fused activation set

=
{
    ReLU,
    Sigmoid
}

Only then does the test proceed as evidence for two activation-specific MatMulNaiveProgram variants.

This is an important test boundary.

Graph contains activation nodes before optimization

does not prove:

activation was baked into the cached WGSL

The transformed graph must be checked.

Distinct weights prevented a different optimizer from collapsing the test

The two convolution branches use distinct weight and bias tensors.

That was not required by the cache bug itself.

It was required to keep the regression topology stable.

If the two branches were otherwise identical, another optimization might identify common work and merge or eliminate part of the graph.

same input

same weights

same convolution

→ possible common-subexpression opportunity

Then the test might stop producing two independent small-matmul programs.

Using distinct initializers ensures:

Conv A
→ remains a distinct computation

Conv B
→ remains a distinct computation

The only cache identity fact intended to collide was the missing activation.

The outputs were compared with a baseline

The test does not stop after proving that two fused nodes exist.

It executes the optimized WebGPU graph and compares the outputs with a baseline path.

This matters because a structural key test alone could show:

activation now appears in the hint

without proving that the previously incorrect numerical behavior was closed.

The end-to-end test checks the complete chain:

graph construction

→ activation fusion

→ Conv-to-matmul path

→ MatMulNaive selection

→ pipeline-cache lookup

→ WGSL execution

→ numerical output comparison

The patch also adds smaller unit tests around activation serialization, but the paired-convolution regression establishes the actual pipeline reuse boundary.

The fix adds activation identity at both call sites

The fused convolution path now constructs the cache hint approximately as:

.CacheHint(
    activation_.ToString(),
    std::to_string(components),
    std::to_string(a_components),
    std::to_string(output_number),
    std::to_string(is_channels_last)
)

The standalone small MatMul path uses the same identity schema:

.CacheHint(
    Activation().ToString(),
    std::to_string(components),
    std::to_string(a_components),
    std::to_string(output_number),
    std::to_string(is_channels_last)
)

For standalone MatMul, the default activation is:

None

For a fused convolution, it may be:

ReLU

Sigmoid

Clip

HardSigmoid

LeakyRelu

Tanh

The key now distinguishes:

MatMulNaive + no activation

from:

MatMulNaive + ReLU

and:

MatMulNaive + Sigmoid

The same program class no longer relies on its call site to remain isolated from other activation configurations.

Why both call sites needed the field

MatMulNaiveProgram is not owned exclusively by one graph operator.

It can be constructed from:

standalone MatMul

and:

small convolution lowered to matmul

Both instances use the program name:

MatMulNaive

A cache identity should describe the generated program itself, not assume that different callers always receive separate caches.

If one call site includes a code-generation field and another omits it, equivalent program instances can be keyed differently—or different program instances can still collide.

Using one complete identity schema at both call sites makes the contract local to the program.

Program depends on activation

→ every constructor path declares activation in the hint

`Activation::ToString()` includes both kind and parameters

The activation serialization contains:

activation kind

parameter 0

parameter 1

That supports activations such as:

LeakyRelu(alpha)

Clip(minimum, maximum)

HardSigmoid(alpha, beta)

For the first defect, the essential field is:

activation kind

ReLU and Sigmoid have no matching semantics even when their parameter storage is zeroed identically.

ReLU:

kind = ReLU
params = 0, 0

Sigmoid:

kind = Sigmoid
params = 0, 0

The serialized kind separates them.

The patch includes a unit test requiring:

ReLU key
!=
Sigmoid key

while identical activations must still produce identical strings.

That preserves useful reuse.

same activation
→ same cache identity

different activation
→ different cache identity

A complete key should not disable all reuse

One possible response to a cache correctness bug is to avoid caching the program.

That would prevent wrong reuse.

It would also discard the benefit of pipeline caching for every valid repeated configuration.

The better repair is to refine the equivalence relation.

Before:

same packing and tensor metadata
→ considered equivalent

After:

same packing

+

same tensor metadata

+

same activation

+

same relevant layout state

→ considered equivalent

The cache can still reuse a ReLU pipeline for another genuinely equivalent ReLU operation.

It simply cannot reuse that pipeline for Sigmoid.

Correct cache keys separate:

necessary misses

from:

incorrect hits

More cache entries are not automatically a regression

Adding activation identity can increase the number of compiled pipelines.

A model containing:

MatMulNaive + ReLU

MatMulNaive + Sigmoid

MatMulNaive + Tanh

now requires separate cache entries.

That is not redundant compilation.

The generated WGSL is different in all three cases.

The old single cache entry represented three programs only by executing the wrong one for two of them.

Fewer cache entries

because

different programs were incorrectly merged

is not a valid optimization.

The meaningful reuse boundary is:

same generated program

not:

same broad operator family

This was narrower than a general WebGPU cache failure

The confirmed defect belonged to:

MatMulNaiveProgram

and the compile-time state it failed to declare.

It does not establish that every ONNX Runtime WebGPU program had an incomplete key.

The PR notes that most comparable programs already included activation identity in their cache hints.

MatMulNaiveProgram was the holdout.

Likewise, this does not prove that the pipeline-cache implementation itself ignored supplied fields or returned arbitrary entries.

The cache received identical identities.

The error was at the producer boundary.

Pipeline cache
→ reused matching key

Program description
→ failed to distinguish different WGSL

The direct production path was intentionally narrow

The regression reaches MatMulNaiveProgram through a specific path:

1×1 Conv

unit stride

no padding

activation fused

N < 8

K < 8

The patch does not claim that every convolution uses the naive matmul shader.

Larger or differently shaped operations may dispatch to:

packed matmul,
subgroup-matrix implementations,
vendor-specific paths,
or other convolution programs.

The exact defect belongs to the branch that uses this program and shares its pipeline cache.

That narrow path was sufficient to produce a confirmed wrong result.

This was a compiled-state bug, not a tensor-shape bug

The two operations could have correct and identical tensor metadata.

input shape
→ valid

weight shape
→ valid

output shape
→ valid

The wrong result did not come from:

an out-of-bounds tensor,
an invalid broadcast,
an incorrect output allocation,
or a malformed dispatch size.

The failure occurred because the runtime reused machine-level program state created under a different compilation configuration.

Tensor contract
→ compatible

Compiled shader contract
→ incompatible

This distinction is why shape-only cache keys are insufficient for specialized kernels.

The first-use dependency can make the failure appear unstable

When two configurations share one invalid key, behavior can depend on which one populates the cache first.

Order A:

ReLU first
→ Sigmoid may execute ReLU

Order B:

Sigmoid first
→ ReLU may execute Sigmoid

Changes in:

graph traversal order,
session initialization order,
optimization order,
or concurrent setup

can therefore change which operation becomes incorrect.

The underlying defect is deterministic for a given cache-population order.

Its visible victim can move.

This kind of order sensitivity can make a cache bug appear nondeterministic at the application level.

Clearing the cache can temporarily hide the defect

If each configuration receives a fresh cache:

ReLU operation
→ compile ReLU
→ correct

cache destroyed

Sigmoid operation
→ compile Sigmoid
→ correct

Both tests pass independently.

That does not prove the key is complete.

It proves only that no two conflicting configurations coexisted during the observed cache lifetime.

A valid cache regression must preserve the cache between the two requests.

The single-graph design does that for the activation tests.

Part 3 examines this test-lifetime boundary in more detail, including why sequentially destroyed sessions were insufficient for the layout-dependent collision.

The patch repaired three forms of omitted generated state

The merged PR ultimately covered three distinct facts that changed MatMulNaiveProgram WGSL.

Activation kind

ReLU

versus

Sigmoid

Different snippets were assigned the same old key.

Activation parameter value

same activation kind

different embedded float constants

The activation existed in the key, but insufficient serialization precision could still merge different shaders.

Layout-dependent bias indexing

channels last
→ bias[col]

channels first
→ bias[row + i]

The layout flag changed shader indexing but was absent from the hint.

The first and third were omitted fields.

The second was a lossy field representation.

All three express one general requirement:

key identity
must be at least as precise as
generated-code identity

What PR #32048 directly changed for this first defect

The merged patch directly:

adds activation serialization to the MatMulNaiveProgram cache hint in the fused-convolution path,
adds the same activation field at the standalone MatMul call site,
retains existing packing and output-number hint fields,
adds WebGPU regressions containing two activation variants in one graph,
verifies that both activations were fused,
executes both activation orderings,
and compares the optimized outputs with the baseline results.

The same patch also adds:

higher-precision activation-parameter serialization,
is_channels_last to the key,
and a concurrent-session layout regression.

Those neighboring changes are not required to explain the ReLU-versus-Sigmoid collision itself.

What the patch does not establish

The public evidence does not establish that:

every WebGPU program had an incomplete pipeline key,
every ONNX Runtime model using ReLU and Sigmoid was affected,
large matmul or convolution paths used the same faulty key,
the cache hash function itself produced an accidental hash collision,
the wrong shader caused a device crash,
tensor shapes or resource bindings were invalid,
this was remotely exploitable,
or the patch provides a measured performance improvement.

The supported conclusion is narrower:

MatMulNaiveProgram embedded the fused activation into WGSL but omitted that activation from its custom cache hint. Two small matmul-backed convolutions with matching program metadata and different activations could therefore reuse one compiled pipeline and silently produce the wrong numerical result.

The complete failure chain

The first defect in #32048 can be written as:

Graph contains two small 1×1 convolutions
        ↓
different activations are fused
        ↓
both lower to MatMulNaiveProgram
        ↓
GetActivationSnippet emits different WGSL
        ↓
old CacheHint omits activation
        ↓
remaining cache-key fields are equal
        ↓
first pipeline is stored under shared key
        ↓
second operation receives that pipeline
        ↓
valid shader executes wrong activation
        ↓
silent numerical mismatch

The repaired path is:

Activation kind and parameters
        ↓
Activation::ToString()
        ↓
included in CacheHint
        ↓
ReLU and Sigmoid receive different keys
        ↓
each distinct WGSL program receives its own pipeline

The final lesson is that cache identity follows generated code

A program cache does not need to remember every fact in the application.

It needs every fact that can change the compiled result.

The boundary can be expressed as:

If state X can change WGSL,

then state X must either:

1. appear in the pipeline key,

2. become a runtime input to one shared shader,

or

3. prevent the pipelines from sharing a cache.

MatMulNaiveProgram chose compile-time specialization.

Its activation became literal WGSL.

That made the activation part of the pipeline identity.

Two operations are cache-equivalent only when the generated program cannot tell them apart.

ONNX Runtime #32048 repaired the key so that the cache finally made the same distinction as the shader generator.

Part 2: why six significant digits were not a canonical float identity

The next article examines a subtler collision.

After activation identity was added, Activation::ToString() still serialized parameters through the default std::stringstream precision:

6 significant digits

The WGSL generator used:

std::to_string

→ 6 digits after the decimal point

Two different float32 values could therefore produce:

the same cache string

but

different WGSL constants

The recorded counterexample was:

1000015.0

1000025.0

They were 160 float32 ULPs apart, yet both collapsed to the same old cache representation.

Part 2 examines why human-readable formatting was not enough to define compiled float identity and why the fix used max_digits10.

Part 3: why separate sessions could make the bug disappear

The final article examines cache lifetime and layout state.

is_channels_last changes generated bias indexing:

bias[col]

versus

bias[row + i]

But a sequential test creating and destroying one session at a time could receive a fresh WebGPU context and cache for each run.

The collision would never occur.

The regression had to keep two sessions alive concurrently so they shared the process-level WebGPU context and program cache.

Part 3 examines:

cache ownership,
context lifetime,
first-use ordering,
graph-local versus cross-session collisions,
and why a test must reproduce the lifetime of the state it claims to validate.

Related material

Patch status: Merged into ONNX Runtime main
Affected component: WebGPU MatMulNaiveProgram
Reachable path: Small matmul and 1×1 Conv-to-matmul lowering with N < 8 and K < 8
Generated-code dependency: Fused activation snippet
Old cache hint: Packing and output-width fields, without activation identity
Failure mode: Different WGSL programs received the same semantic cache key
Observed consequence: Wrong shader reuse with no error or warning
Minimum regression: Two small convolutions with matching program metadata and different fused activations in one graph
Order coverage: ReLU→Sigmoid and Sigmoid→ReLU
Repair: Include Activation::ToString() in the cache hint at both program call sites
Current status: The repaired hint remains present in ONNX Runtime main
Evidence boundary: Confirmed WebGPU numerical regression and merged fix; no claim that every WebGPU pipeline key was affected

This is Part 1 of a three-part series on ONNX Runtime WebGPU pipeline-cache identity.

This article examined why activation semantics compiled directly into WGSL had to be represented in the pipeline key.

Part 2 examines why activation presence alone was insufficient when two distinct float parameters could collapse to one six-significant-digit cache string.

Part 3 examines why layout-dependent WGSL and process-shared cache lifetime required a concurrent-session regression rather than two isolated runs.

#ONNXRuntime #WebGPU #WGSL #PipelineCache #ShaderCompilation #MatMul #Convolution #CacheKey #CompilerCorrectness #EdgeAI #CodeAnalysis