Intel OpenVINO PR #37605: Why an Uninitialized GPU Flag Changed Quantization Across Runs

Intel OpenVINO PR #37605 removes a GPU capability flag that could influence model transformation without ever receiving a value on the Level Zero path.

The field was:

supports_non_uniform_work_group

OpenVINO’s OpenCL device initialization queried and assigned it. Level Zero created a device_info object, filled many of its fields, and returned it without assigning this one.

The field looked like a hardware capability. On Level Zero, it had never been assigned a value.

OpenVINO later read that value while deciding whether grouped dynamic activation quantization was allowed. The same model could therefore take different transformation paths depending on an indeterminate byte in the device-information object.

The pull request reports corrupted, garbage, or degraded GenAI output across process launches. The merged fix removes the flag entirely and replaces the capability-dependent behavior with one backend-independent rule:

Every dispatched global work size must be divisible by its local work size.

The uninitialized value crossed several layers

The failure did not stay inside device discovery.

Before the patch, the data path was:

Level Zero device initialization
        ↓
device_info.supports_non_uniform_work_group
        ↓
GPU transformation pipeline
        ↓
grouped dynamic quantization enabled or rejected
        ↓
different model graph and kernel selection

The pre-fix device_info structure declared the field as a plain bool without a default member initializer.

The Level Zero initialization function began with:

device_info info;

It explicitly assigned support flags for FP16, FP64, subgroups, unified shared memory, command lists, images, and other capabilities. It did not assign supports_non_uniform_work_group.

The OpenCL path behaved differently. It queried:

CL_DEVICE_NON_UNIFORM_WORK_GROUP_SUPPORT

and stored the result.

That asymmetry meant one runtime supplied a real capability value and the other supplied no defined application value at all.

The flag controlled graph transformation

OpenVINO’s GPU transformation pipeline used the field in two related decisions.

First, it helped determine whether per-token INT8 dynamic quantization should use a group size of 128 on newer GPU architectures:

architecture >= Xe2
and
supports_non_uniform_work_group

Second, the transformation callback rejected a grouped dynamic-quantization candidate when the same flag appeared false.

The field therefore did more than select a minor kernel option. It could decide whether the graph received a grouped dynamic-quantization transformation.

That is why an initialization bug could appear as an accuracy problem rather than a clean runtime error.

One process could enter the grouped path. Another could leave the operation untransformed or choose a different quantization path. The PR reports that this changed model-level results between launches on the Level Zero backend.

The public record does not include the affected model names or exact benchmark scores. The source does establish that an uninitialized field controlled the transformation decision.

Why setting the flag to false was not the final fix

The smallest local repair would have been:

info.supports_non_uniform_work_group = false;

That would give the Level Zero path a deterministic value.

It would also keep grouped dynamic quantization disabled whenever the runtime did not advertise non-uniform work-groups, even though the relevant optimized kernel no longer needed that feature.

Initializing the flag would remove the indeterminate state. It would not remove the obsolete decision.

A previous OpenVINO change, PR #36624, had already modified the large-group dynamic-quantization kernel so its dispatch could be padded safely.

That earlier patch added two pieces.

The host code rounded the number of dispatched blocks upward so the global work size could be evenly divided by the local work size.

The kernel calculated:

is_valid_block = global_block_idx < TOTAL_BLOCK_NUM;

Padded work-items used neutral reduction values and skipped output writes when their block index was outside the real input range.

The extra work-items existed only to complete the final work-group. They did not represent additional tensor elements.

Global and local work sizes no longer needed a capability gate

GPU dispatch describes work with two related sizes.

The global work size states how many work-items are launched.

The local work size states how those work-items are grouped.

A non-uniform work-group capability allows a final partial group when the global size is not divisible by the local size.

OpenVINO’s dynamic-quantization dispatch no longer needed a partial group. It could round the global block count upward and let the kernel ignore the padded positions.

For example, a conceptual dispatch with:

real blocks: 35
local block group: 32

can be launched as:

dispatched blocks: 64
local block group: 32

The first 35 block indices perform real work. The remaining 29 fail the validity check and do not write tensor output.

The earlier patch made the kernel safe for padded work-items. It did not remove the capability gate.

PR #37605 completes that transition.

The merged patch makes aligned dispatch the invariant

The final patch does not preserve two policies for OpenCL and Level Zero.

It removes supports_non_uniform_work_group from:

device_info
kernel-selector EngineInfo
the device-to-kernel parameter copy
the OpenCL capability query
the graph-transformation gate
dispatch validation

The dynamic-quantization host code now always performs:

dispatch_block_num = Align(total_block_num, block_num);

The OV_GPU_WITH_ZE_RT conditional is gone.

KernelBase::CheckDispatchData() also changes from:

Check divisibility only when non-uniform groups are unsupported

to:

Always require GWS % LWS == 0

A supported non-uniform work-group is a device capability. OpenVINO no longer uses that capability as permission to emit unaligned dispatches.

This is a stronger design than initializing one boolean. It makes the legality of a generated dispatch a property of OpenVINO’s kernel contract rather than a branch on backend capability.

Grouped quantization is now selected consistently

The transformation pipeline now chooses the group-size-128 path from the device architecture alone:

arch >= gpu_arch::xe2

It also removes the callback that rejected grouped dynamic quantization when the old flag appeared false.

The result is consistent across OpenCL and Level Zero:

eligible architecture and graph
        ↓
grouped dynamic quantization selected
        ↓
optimized kernel receives an aligned dispatch
        ↓
padded work-items are ignored safely

There is no capability byte left to change the graph between process launches.

The patch does not claim that every GPU should use grouped quantization. Architecture, model pattern, group size, configuration, and other transformation conditions still apply.

It removes one condition because that condition no longer describes a requirement of the selected kernel.

The test checks the new execution rule

The modified GPU unit test uses a large grouped dynamic-quantization case whose dispatch previously depended on non-uniform work-group support.

The test already compares the selected implementation’s outputs with a forced reference kernel.

For integer quantization, it permits the existing small numerical threshold used by the test harness. It then records the selected kernel identifier.

Before this PR, the assertion for the optimized kernel was conditional:

if the device reports non-uniform work-group support
    require the optimized kernel

After the patch, the test unconditionally requires an _opt kernel.

That verifies two post-fix properties:

the aligned optimized path is selected regardless of the removed capability
and
its numerical output agrees with the reference for the tested shape

The new test proves the post-fix dispatch and numerical result. It does not reproduce uninitialized memory.

A reliable regression test should not depend on whatever value an uninitialized byte happens to contain. The structural removal of the field is the proof that this specific source of nondeterminism is gone.

Review expanded the change from initialization to removal

The first major review response was not “initialize the missing field.”

A reviewer pointed to PR #36624 and proposed removing the entire supports_non_uniform_work_group mechanism.

That prompted a broader question: did the flag still protect other OpenCL kernels that might legally rely on non-uniform work-groups?

The discussion examined the dynamic-quantization change and the role of CheckDispatchData. Reviewers concluded that aligned global and local sizes should now be the standard expectation.

The final patch therefore:

aligns the dynamic-quantization dispatch for every backend
enforces divisibility in common validation
removes the capability from both runtime representations
and removes transformation decisions based on it

Two public approvals were recorded before merge. A later tidy warning about braces in kernel_base.cpp was fixed before the final commit.

This review history matters because removing a capability flag is broader than fixing one uninitialized read. The accepted design depends on replacing that capability with a stronger invariant.

The removal is broader than the direct test coverage

CheckDispatchData() is common kernel-selector infrastructure.

After this patch, any kernel that supplies a non-divisible global and local size will fail validation even on an OpenCL device that supports non-uniform groups.

The PR description states that other users already produce divisible dispatches through a local size of one, explicit alignment, or a helper that selects a divisor.

The public diff does not add a repository-wide test proving that statement for every kernel.

The removal is broader than dynamic quantization. The direct test coverage is not.

That is the main review limitation. The change was approved after codebase analysis and CI, but the committed regression shown in this patch directly exercises the dynamic-quantization boundary.

What the evidence establishes

The merged code establishes that:

the Level Zero initializer previously left the field unassigned
the field previously gated grouped dynamic quantization
the optimized kernel can use aligned dispatch with guarded padding
the final patch removes the field and its consumers
the common validator now requires divisible dispatch sizes
the focused test always selects and validates the optimized path

The author also reports matching model benchmark accuracy between Level Zero and OpenCL on Windows and Linux after the fix.

This review did not independently execute those model benchmarks or the GPU unit suite. No public numerical table identifies the affected models or quantifies the pre-fix variance.

The strongest supported conclusion is narrower than “all OpenVINO quantization nondeterminism is fixed”:

An uninitialized Level Zero device field could control whether grouped dynamic quantization was applied. OpenVINO removed that field after making aligned dispatch the common kernel rule.

The contract restored by the patch

A compiler or runtime may branch on a hardware capability only when that capability has a defined value on every construction path.

There is a better option when the generated program can avoid depending on the capability.

The corrected sequence is:

select an eligible quantization transformation
        ↓
construct a divisible GPU dispatch
        ↓
pad the final group when needed
        ↓
guard padded work-items in the kernel
        ↓
validate the same rule on every backend

A graph transformation should not depend on a byte that one runtime never initialized.

OpenVINO PR #37605 removes that accidental authority and replaces it with a dispatch invariant the kernel can enforce.