ONNX Runtime PR #32489: Why Concurrent Startup Could Leave a Dangling Allocator

ONNX Runtime PR #32489 fixes an XNNPACK startup race that could leave the library holding a pointer to an allocator whose owner had been replaced.

The trigger was concurrent first-time session creation. Separate execution providers reached the same process-wide allocator through an unsynchronized null check. Both could decide to create it, while XNNPACK retained the context supplied during initialization.

The patch puts allocator creation, context publication, and xnn_initialize() inside one std::call_once callback. Its regression test starts in a fresh process, releases 32 competing calls together, and checks that XNNPACK can still allocate after the individual provider owners are gone.

The pointer crossed an ownership boundary

The XNNPACK adapter keeps two static objects in GetStoredAllocator():

static AllocatorPtr ort_allocator;
static xnn_allocator xnn_allocator_wrapper_ = { ... };

The first is a shared owner of an ONNX Runtime allocator. The second is a C callback table containing allocation functions and a raw context pointer.

The allocation callback casts that context back to IAllocator* and invokes Alloc(). The deallocation callback uses the same context to call Free().

The wrapper does not carry a shared_ptr. Copying its context therefore does not extend the allocator's lifetime.

The XNNPACK revision pinned by this ONNX Runtime commit makes that distinction concrete. Its initialization routine selects an allocator table and copies the structure into xnn_params.allocator through its own once-initialization mechanism. The copied raw pointer must remain valid for subsequent allocation and release operations.

An ownership error at startup can consequently surface much later. Issue #32461 reports faults during weight preparation, inference, and operator destruction, all reaching the allocator through the retained context.

Two sessions could both pass the empty check

The old provider code followed this order:

if (!stored_allocator) {
  stored_allocator = CreateAllocator(allocator_info);
}
xnn_allocator->context = stored_allocator.get();
xnn_initialize(xnn_allocator);

Making the storage static did not synchronize these later assignments.

An illustrative interleaving explains the lifetime failure:

Thread A sees an empty stored allocator.
Thread B also sees it empty and enters the creation branch.
Thread A creates allocator A and publishes its context to XNNPACK.
Thread B assigns allocator B to the shared owner.
XNNPACK still retains the context for A.

When no remaining shared owner keeps A alive, that retained context becomes dangling. Rewriting the external wrapper to point at B does not update the allocator structure that XNNPACK already copied.

The actual unsynchronized accesses are a C++ data race, so this sequence illustrates the ownership hazard rather than predicting one guaranteed execution. The reported crashes provide the separate runtime evidence.

There was also an ongoing publication problem: even after initialization, every new provider call wrote the shared wrapper's context again. Existing kernels could read that wrapper while the write occurred. Protecting only allocator construction would leave that access pattern in place.

The once callback covers the complete handoff

The patch moves all three operations into the same protected region. Omitting allocator-construction details, the new structure is:

static std::once_flag init_once;
std::call_once(init_once, [&]() {
  if (!stored_allocator) {
    stored_allocator = CreateAllocator(allocator_info);
  }
  xnn_allocator->context = stored_allocator.get();
  const xnn_status status = xnn_initialize(xnn_allocator);
  if (status != xnn_status_success) {
    ORT_THROW("XNNPACK initialization failed with status ", status);
  }
});

Every successful caller then receives a shared pointer to the stored allocator.

std::call_once permits at most one normally returning initialization for a given flag. Other successful callers return only after that initialization is complete, with its writes made visible to them. This supplies both exclusion during setup and publication to waiting callers.

After success, later provider creation no longer rewrites the callback context or repeats XNNPACK initialization through this path.

The scope is narrower than serializing session execution. The once flag protects shared setup; it does not turn every model operation into a global critical section.

Failed initialization can retry without replacing the owner

The status check stays inside the callback and throws when XNNPACK initialization fails. An exceptional call_once invocation does not mark initialization successfully complete, so a later call can retry.

The inner if (!stored_allocator) remains important. A failed attempt may already have constructed the owner. Keeping it avoids replacing an object whose pointer may have crossed the library boundary before completion.

This is a conservative lifetime safeguard, not a claim that every XNNPACK error occurs after context retention. In the pinned implementation, the hardware-support failure check precedes allocator capture.

Provider destruction also calls xnn_deinitialize(), but the pinned XNNPACK implementation simply returns success from that function. It does not clear the process-wide allocator. That supports keeping successful initialization complete across provider destruction and recreation.

The checked-in regression does not inject an initialization failure. Retry behavior is supported by the code structure and the C++ contract rather than an upstream failure-injection test.

The regression must begin before the first initialization

A concurrency test run after another XNNPACK test could start with the allocator already populated. It might exercise dozens of threads while completely missing the original empty-check race.

The new XnnpackEPDeathTest.ConcurrentAllocatorInitialization uses GoogleTest's threadsafe subprocess style and ASSERT_EXIT. On POSIX systems, that style re-executes the test binary for the selected test, instead of merely inheriting a warmed-up process image.

Despite the test-suite name, the expected result is normal exit with success.

Inside the child, the test constructs 32 provider objects before launching the workers. Each worker waits on a condition variable. The main thread releases them only after all 32 have reached the starting gate.

The calls being raced are specifically CreatePreferredAllocators(). The test is not constructing 32 complete inference sessions or executing 32 models.

After joining every worker, it checks that no exception was recorded and that every returned allocator has the same identity.

Identity is followed by a lifetime test

Equal pointers at the end of initialization would be an incomplete check. The object could still disappear when the providers release their references.

The test therefore destroys every provider and resets the allocator references retained by the test. It then calls:

xnn_create_workspace(&workspace);
xnn_release_workspace(workspace);

In the pinned XNNPACK source, workspace creation allocates the workspace descriptor through XNNPACK's allocator helpers, and release frees it through those helpers. This exercises the context retained by the library after the per-provider and test-held owners have been released.

The process-wide owner must keep the allocator alive.

Finally, the test creates another provider, checks that it receives the same allocator, and performs a 64-byte allocation and release through that allocator.

The test thus covers concurrent identity, retained-context use, and later provider creation. It does not need a large model to exercise the failed ownership boundary.

The negative controls make the reported passes meaningful

The public record includes runs with the new test retained but the provider fix removed. That matters because a timing-sensitive test can pass repeatedly without entering the vulnerable window.

Both sets of results below were reported on macOS arm64 with MallocScribble=1:

Report	Original provider behavior	Fixed provider behavior
PR author	12 failures in 20 fresh-process runs	0 failures in 100 runs
Original issue reporter	62 failures in 100 fresh-process runs	0 failures in 100 runs

The author reports that seven of the twelve negative-control failures were segmentation faults. Both reports also describe 20 enabled XNNPACK tests passing, with 12 existing tests disabled.

These are separate experiments, not a combined estimate of the application's crash probability. Their value is that the same focused test exposes failure with the old provider behavior and stops doing so after the synchronization change.

The contributor reports are not executions performed for this article.

The iOS follow-up has a different evidentiary role

The original reporter also rebuilt the iOS framework and restored concurrent loading in a React Native application.

They reported 28 successful cold launches of a stress harness and five successful launches of the real three-model wake-word engine. However, the unfixed iOS harness also passed its reported runs. Its loader threads did not have the same synchronized start as the native regression and rarely entered the vulnerable window.

Those iOS runs support functional compatibility of the patched library with that workload. They do not establish an on-device crash-versus-no-crash comparison.

The strongest negative control remains the macOS subprocess test. The distinction comes from the reporter's own qualification and should survive any summary of the PR.

What this patch establishes

The implementation covers the entire allocator handoff and keeps its owner stable after successful publication. The pinned XNNPACK code explains why that stability must outlast individual providers, and the regression deliberately releases those providers before using the retained context.

The evidence supports the fix for concurrent allocator initialization through this provider path. It does not establish a particular released-package version containing the patch, universal mobile validation, or new permission to run the same XNNPACK kernel concurrently. The provider still reports that concurrent kernel execution is unsupported.

No ONNX Runtime build or upstream test suite was executed for this article. A separate, dependency-free C++ check confirmed the normal and exceptional call_once behavior used in the reasoning; its source and output are in the Research Report. That check is not an ONNX Runtime reproduction.

The restored rule is straightforward: when a library retains a raw allocator context, the shared owner must remain stable for as long as that library can use it. Synchronizing allocation alone is insufficient if publication and retention remain outside the same protected initialization.