PyTorch #182948 — Why the Deadlock Regression Tests Had to Run in Separate Processes

PyTorch Lazy Initialization, Part 3 of 3 — Bounding a self-deadlock, validating two lifecycle paths, and separating similar XPU and MTIA code from a reproduced backend failure

The first two articles in this series examined a self-deadlock in PyTorch’s lazy GPU initialization.

The old torch.cuda._lazy_call() implementation could execute a callback while holding:

_initialization_lock

That lock was a non-reentrant threading.Lock.

If the callback invoked another GPU API that eventually re-entered _lazy_call(), the same thread attempted to acquire the same lock again.

Outer _lazy_call()
        ↓
Acquire initialization lock
        ↓
Execute callback under the lock
        ↓
Callback re-enters _lazy_call()
        ↓
Attempt to acquire the same lock
        ↓
Wait forever

The patch fixed two distinct routes.

Path 1

Backend already initialized
→ callback executed under the lock
→ nested _lazy_call deadlocked

Path 2

Native initialization complete
but queued callbacks still draining
→ _initialized remained False
→ initializer thread re-entered _lazy_call
→ nested lock acquisition deadlocked

The implementation fix was not enough by itself.

A regression test also needed to demonstrate that both paths now completed.

But testing a deadlock creates a special problem:

What happens if the regression test itself enters the old deadlock and never returns?

A normal assertion cannot execute after a function that never completes.

A timeout running inside the same interpreter cannot safely restore a thread permanently blocked on a non-reentrant lock.

PR #182948 therefore placed each scenario inside a fresh Python subprocess.

The parent test process remained outside the lock cycle.

It waited for a bounded amount of time, verified the child’s exit status, and required an explicit completion marker:

done

The regression contract was not merely:

The callback returned the expected value.

It was:

The process remained live,
completed the full initialization path,
reached the end of the script,
and exited successfully within a finite deadline.

A deadlock test must survive the deadlock

Most unit-test failures return control to the test runner.

Function returns wrong result
        ↓
Assertion fails
        ↓
Test runner reports failure

An exception also returns control.

Function raises
        ↓
Test runner catches the exception
        ↓
Failure is reported

A self-deadlock does neither.

Thread waits for its own lock
        ↓
No exception
        ↓
No return
        ↓
No later assertion

If the reproduction runs directly inside the main PyTorch test process, the process can remain blocked indefinitely.

The test runner cannot reach:

self.fail(...)

because the same execution context is trapped inside the call being tested.

The test therefore needs an observer that is not part of the deadlocked state.

Parent process
→ observes

Child process
→ may deadlock

That process boundary lets the parent retain control even when the child cannot make progress.

Why a normal timeout around the function was not enough

Python does not provide a general mechanism that can safely interrupt arbitrary code running in the same thread and restore every lock invariant afterward.

Suppose the test attempted:

Start timer
        ↓
Call reproduction directly
        ↓
Deadlock while holding _initialization_lock

Even if another mechanism noticed that too much time had passed, the blocked thread would still own the lock.

The process-local initialization state could remain:

_initialization_lock
→ permanently held

_initialized
→ possibly incomplete

queued callbacks
→ partially processed

CUDA/ROCm runtime
→ partially initialized from Python’s perspective

Continuing with later tests would not be reliable.

The test must not only detect noncompletion.

It must contain the noncompletion inside state that can be discarded as a whole.

A process gives that boundary.

Why a helper thread would still leave poisoned process state

A possible alternative would be:

Main test thread
        ↓
Start worker thread
        ↓
Worker runs deadlock reproducer
        ↓
Main thread waits with timeout

The main thread could notice that the worker did not finish.

But the deadlocked worker would still exist inside the same process.

It could still hold:

_initialization_lock

The CUDA/ROCm initialization module and its global variables would also be shared with the parent test thread.

_initialized

_queued_calls

_lazy_seed_tracker

_tls objects

initialization lock

native runtime state

The parent could report one failed test, but the process might no longer be usable for subsequent GPU tests.

Python also cannot generally force a blocked thread to release a lock and terminate cleanly.

A subprocess isolates both:

the blocked thread

and:

the process-wide lazy-initialization state

from the main test runner.

The test helper creates a fresh Python interpreter

PR #182948 added a helper shaped like this:

def _check_lazy_call_reentrant_no_deadlock(self, script):
    timeout_sec = 60

    try:
        proc = subprocess.run(
            [sys.executable, "-c", script],
            capture_output=True,
            text=True,
            timeout=timeout_sec,
        )
    except subprocess.TimeoutExpired as e:
        self.fail(
            "lazy_call reentrancy subprocess did not finish "
            "within 60s (likely deadlock)"
        )

    self.assertEqual(proc.returncode, 0)
    self.assertIn("done", proc.stdout)

Each scenario begins in a new interpreter.

New Python process
        ↓
Import torch
        ↓
Create fresh module-global initialization state
        ↓
Execute one reproduction
        ↓
Exit or time out

The helper checks three separate conditions:

Did the process finish within the deadline?

Did it exit successfully?

Did it reach the explicit completion marker?

The 60-second timeout is not the expected execution time

The successful scripts are short.

They import PyTorch, initialize the backend, execute one reentrant path, print done, and exit.

They are not expected to require sixty seconds during ordinary execution.

The source comment explains the purpose of the long deadline:

The passing path exits quickly.

The timeout exists only to bound failure detection.

The limit is generous for slow CI machines.

This distinction matters.

60 seconds
≠
acceptable normal runtime

It means:

If the child is still alive after this generous boundary,
the test classifies the scenario as likely deadlocked.

A shorter deadline might falsely fail on overloaded CI hosts.

An infinite wait would fail to bound the regression.

A timeout indicates noncompletion, not every possible root cause

The helper’s failure message says:

likely deadlock in torch.cuda._lazy_call

That wording is appropriately narrow.

A timeout proves:

The child did not finish before the deadline.

By itself, it does not universally prove:

The exact non-reentrant lock cycle was the only possible cause.

In these two scripts, however, the old implementation had a directly identified lock cycle, and the commit reports that both scripts timed out before the fix and completed afterward.

The diagnosis is therefore supported by:

known source-level deadlock

+

minimal scripts entering that path

+

old code timing out

+

fixed code completing

It is stronger than a timeout observed in an unrelated full-model test.

Why the test checks the process return code

Completion before the timeout is not sufficient.

The subprocess could terminate because of:

an unhandled Python exception,
a backend initialization error,
a crash,
an assertion,
an import failure,
or another abnormal exit.

The helper therefore requires:

proc.returncode == 0

If the process exits unsuccessfully, the test reports both:

stdout

stderr

The regression passes only when the script completes normally.

No timeout
+
zero return code

are separate requirements.

Why the test also requires the word `done`

A zero return code alone does not prove that the intended sequence reached its final line.

The scripts explicitly end with:

print("done")

The parent checks:

self.assertIn("done", proc.stdout)

This creates a simple completion receipt.

Backend initialized

Reentrant callback completed

Control returned to the script

Final marker printed

The test therefore distinguishes:

Process exited somehow

from:

Process reached the end of the intended reproduction

Regression test 1 covers the fully initialized path

The first script is:

import torch

torch.cuda.init()

state = torch.cuda.get_rng_state()

torch.cuda._lazy_call(
    lambda: torch.cuda.set_rng_state(state)
)

print("done")

Its lifecycle is:

Initialize CUDA/ROCm fully
        ↓
Capture current RNG state
        ↓
Enter outer _lazy_call()
        ↓
Outer callback calls set_rng_state()
        ↓
set_rng_state() submits another lazy callback
        ↓
Nested _lazy_call()

Before the fix:

Outer _lazy_call
→ held _initialization_lock

Nested _lazy_call
→ waited for that same lock

Result
→ subprocess timeout

After the fix:

is_initialized()
→ True

Outer callback
→ runs without lock

Nested _lazy_call
→ also sees initialized state

Result
→ callback completes
→ “done” printed

This test protects the stable post-initialization fast path.

Why `set_rng_state()` is a useful nested edge

set_rng_state() does not update the generator immediately in every state.

It creates a callback that eventually accesses the selected default generator and applies the requested state.

It then submits that callback through _lazy_call().

set_rng_state()
        ↓
construct generator callback
        ↓
_lazy_call(callback)

That makes it a compact, real framework path for re-entering _lazy_call().

The test does not invent recursion by having the callback call _lazy_call() directly.

It demonstrates recursion through an ordinary PyTorch CUDA RNG API.

Regression test 2 covers queue draining during initialization

The second script is:

import torch

torch.cuda._lazy_call(
    lambda: torch.cuda.manual_seed(42)
)

torch.cuda.init()

print("done")

Its lifecycle begins before initialization.

Call _lazy_call()
        ↓
Backend not initialized
        ↓
Queue outer callback

Then:

Call torch.cuda.init()
        ↓
_lazy_init() acquires initialization lock
        ↓
torch._C._cuda_init() completes
        ↓
_tls.is_initializing is installed
        ↓
Queued outer callback begins

The outer callback calls:

torch.cuda.manual_seed(42)

manual_seed() constructs another generator callback and submits it through _lazy_call(..., seed=True).

Queued outer callback
        ↓
manual_seed()
        ↓
Nested _lazy_call()

Before the fix:

_initialized
→ still False

Nested _lazy_call
→ attempted _initialization_lock

Same thread already owned lock through _lazy_init

Result
→ subprocess timeout

After the fix:

Initializer thread has _tls.is_initializing

Nested _lazy_call
→ executes callback immediately

Queue drain completes

_initialized becomes True

“done” is printed

This test protects the transitional initializer-thread path rather than the stable initialized path.

The two scripts are not interchangeable

Both failures involve:

callback
→ nested _lazy_call
→ same non-reentrant lock

But the state that permits the fixed callback differs.

First script

_initialized
→ True

The ordinary initialized fast path is sufficient.

Second script

_initialized
→ False

_tls.is_initializing
→ present on this thread

The thread-local transitional path is required.

A test of only the first script would not prove that queue draining was safe.

A test of only the second would not directly protect callback execution after full initialization.

The patch needed both because the implementation fix contained two distinct branches.

The tests also protect `run_now` indirectly

Consider a third timing possibility:

Thread A checks is_initialized()
→ False

Thread B completes initialization

Thread A acquires the lock
→ sees initialized state

The fixed code records:

run_now = True

inside the lock and executes the callback after releasing it.

The two added subprocess scripts are centered on deterministic same-thread reentrancy paths.

They do not construct a dedicated multi-thread interleaving that forces the run_now race.

The source directly protects that race structurally.

The new tests primarily protect:

initialized fast-path reentrancy

initializer-thread reentrancy

This is an evidence boundary worth keeping visible.

A fresh process also gives each script a clean lifecycle

Lazy initialization is process-global within the Python module.

Once one in-process test has called:

torch.cuda.init()

later tests cannot naturally return that module to its original uninitialized state merely by assigning one Boolean.

The runtime may have created:

device contexts,
default generators,
driver state,
caches,
queued callback history,
or other native resources.

Using a new subprocess lets the second test start from:

fresh import

_initialized = False

fresh queue

fresh initialization lock

fresh thread-local state

That is especially important for the queued-callback scenario, which must begin before initialization.

This clean-state benefit follows from the process design, even though the commit’s explicit reason for the subprocess is that a regression can deadlock the interpreter.

The tests verify liveness, not a returned numerical value

Neither test asserts a tensor result.

Their primary contract is liveness.

Callback re-enters initialization system
        ↓
Execution must still complete

The checks are:

finite completion

successful process exit

completion marker reached

This is appropriate because the old defect was not:

wrong RNG value

It was:

no forward progress

A regression test should match the observable contract that failed.

The parent process is a watchdog outside the lock graph

The deadlock graph exists entirely inside the child.

Child thread
→ owns initialization lock
→ waits for initialization lock

The parent process does not share that Python lock.

Parent
→ waits on child process completion

Child
→ runs PyTorch reproduction

The parent’s wait has its own timeout.

The child’s initialization lock cannot prevent the parent from reaching the timeout handler.

This is the core advantage of the process boundary:

The observer is outside the synchronization system being tested.

The regression test is stronger than relying on broad model execution

The issue was originally encountered while running larger models under torch.compile on ROCm.

A full-model test would have several disadvantages.

Many unrelated compiler paths

Large runtime cost

Multiple possible hang sources

Harder diagnosis

More environment dependencies

The two subprocess scripts isolate the minimum lifecycle edges.

Script 1
→ initialized callback reentrancy

Script 2
→ queued callback reentrancy

If either script times out, the investigation begins at a much narrower boundary than a full model hang.

The commit reports old-versus-new behavior

The commit states that both regression scenarios:

timed out before the fix

and:

passed after the fix

on a ROCm gfx90a host.

This is important because a newly added test that passes only on the fixed code does not automatically prove it would have caught the old defect.

The recorded negative control shows that the scripts reached the historical lock cycle.

The evidence is still bounded to the reported environment.

It does not establish identical behavior on every CUDA and ROCm system.

The timeout test does not prove every reentrant callback is safe

The two scripts cover two known call graphs.

set_rng_state
→ initialized path

manual_seed
→ queued initialization path

They do not enumerate every callback that can be submitted through _lazy_call().

A different callback could:

recurse indefinitely,
acquire another incompatible lock,
wait for another thread,
or block on unrelated GPU work.

The patch proves a narrower property:

These two documented callbacks no longer deadlock solely by reacquiring _initialization_lock through _lazy_call().

The timeout test does not prove every thread interleaving

The helper launches one reproduction per process.

It does not deliberately create multiple Python threads racing through initialization.

The run_now path exists for precisely such a race:

one thread checks before initialization

another thread completes initialization

the first thread acquires the lock afterward

The source handles that interleaving by moving callback execution outside the lock.

The new tests do not appear to force that exact schedule.

Thus the evidence matrix is:

Same-thread reentrant initialized path
→ runtime regression test

Same-thread reentrant queue-draining path
→ runtime regression test

Cross-thread run_now race
→ source-level implementation protection
→ no dedicated forced schedule shown here

The patch fixed only `torch.cuda`

The landed commit changes:

torch/cuda/__init__.py

test/test_cuda.py

It does not modify:

torch/xpu/__init__.py

torch/mtia/__init__.py

The commit explicitly notes that XPU and MTIA contain the same callback-under-lock pattern and would require follow-up work.

The current source snapshots inspected for this article still show structurally similar logic in those backends.

The current XPU pattern still executes callbacks under the lock

The current XPU implementation uses:

_initialization_lock =
    threading.Lock()

Its _lazy_call() contains the structure:

with _initialization_lock:
    if is_initialized():
        callable()
    else:
        queue(callable)

Its _lazy_init() also:

acquires the initialization lock

runs native XPU initialization

sets a thread-local initializing marker

drains queued callbacks while still holding the lock

sets _initialized afterward

That is structurally close to the old CUDA implementation.

The current MTIA pattern is also similar

The current MTIA implementation likewise defines:

_initialization_lock =
    threading.Lock()

Its _lazy_call() executes an initialized callback from inside:

with _initialization_lock:

Its _lazy_init() drains queued calls while holding that same lock and sets the public initialized state only afterward.

The source pattern therefore exists in three backend families:

CUDA/ROCm
→ pattern existed
→ concrete deadlock reproduced
→ fixed and tested

XPU
→ similar pattern visible
→ no concrete reproduction established here

MTIA
→ similar pattern visible
→ no concrete reproduction established here

Similar source is evidence of risk, not proof of the same runtime bug

It would be inaccurate to conclude:

XPU definitely deadlocks.

MTIA definitely deadlocks.

A deadlock requires more than the lock structure.

A callback must actually re-enter the same backend’s _lazy_call() while the lock is held.

For each backend, the complete chain would need to be demonstrated:

Outer backend _lazy_call
        ↓
Callback executes under backend initialization lock
        ↓
Callback invokes another backend API
        ↓
That API calls the same backend _lazy_call
        ↓
Nested lock acquisition
        ↓
Noncompletion

The source confirms the first two structural ingredients.

It does not yet confirm a concrete XPU or MTIA callback chain and runtime timeout.

The CUDA reproduction cannot simply be relabeled as XPU or MTIA

The CUDA tests use:

torch.cuda.set_rng_state()

torch.cuda.manual_seed()

Equivalent backend tests would need to confirm:

the backend exposes the corresponding operation,
the operation actually submits through that backend’s _lazy_call(),
the backend can initialize in the test environment,
the callback re-enters at the same lifecycle point,
and the scenario reaches the same lock cycle.

A matching function name alone is not enough.

The runtime route must be inspected and reproduced.

This is the same distinction used throughout the blog:

similar structure
≠
confirmed execution behavior

What a proper XPU follow-up would need

A strong XPU follow-up would require at least one minimal script that demonstrates the complete reentrant path.

Conceptually:

Initialize XPU

Enter XPU _lazy_call

Callback invokes an XPU API

That API re-enters XPU _lazy_call

Old code times out

Fixed code completes

A second case should cover queue draining if a concrete XPU callback can re-enter during _lazy_init().

The tests should again run in separate subprocesses.

They should not rely only on static similarity to the CUDA source.

What a proper MTIA follow-up would need

The same standard applies to MTIA.

Source pattern
→ already visible

But a concrete finding needs:

actual MTIA callback

actual nested _lazy_call

actual non-reentrant lock cycle

actual old-versus-new runtime result

If the available MTIA environment makes hardware execution difficult, the project could still consider a controlled test double or backend-initialization fixture.

But any such test would need to prove that it preserves the relevant lock and callback semantics.

That would be a future design, not evidence provided by PR #182948.

The strongest next step may be sharing the corrected state machine

CUDA, XPU, and MTIA maintain similar lazy-initialization structures.

That duplication creates repeated obligations:

same lock semantics

same queued-callback semantics

same transitional state

same reentrancy rule

same regression strategy

One possible architectural direction would be to factor the queue-or-run state machine into shared infrastructure.

That could reduce the chance that one backend receives a concurrency correction while another retains the older pattern.

This is an architectural inference, not a change made by the patch.

Any shared abstraction would still need to accommodate backend-specific initialization, error types, bad-fork handling, generator setup, and availability semantics.

Duplicated code also duplicates evidence requirements

Even if the implementations look almost identical, each backend can differ in:

which callbacks are queued,
when native initialization becomes safe,
which operations re-enter lazy initialization,
how generators are created,
how fork handling works,
and whether callbacks invoke backend code during queue draining.

A shared source pattern supports a shared review question.

It does not eliminate backend-specific runtime validation.

Common code smell
→ one investigation

Backend-specific execution
→ separate proof

The current evidence should be stated in layers

Claim	Current evidence
CUDA/ROCm had two `_lazy_call()` self-deadlock paths	Confirmed by source and reproductions
Both added scripts timed out before the fix	Reported by the landed commit
Both scripts completed after the fix on ROCm `gfx90a`	Reported by the landed commit
The CUDA fix executes callbacks outside `_lazy_call()`’s lock	Confirmed in landed source
XPU still has a similar callback-under-lock structure	Confirmed in current source
MTIA still has a similar callback-under-lock structure	Confirmed in current source
XPU currently reproduces the same deadlock	Not established
MTIA currently reproduces the same deadlock	Not established
A common shared fix has been implemented	Not established

This separation prevents one verified backend finding from being silently expanded into unverified claims about every similar file.

The subprocess test is also an evidence boundary

The test establishes:

The script completed within sixty seconds.

It exited successfully.

It printed the completion marker.

It does not produce a formal proof that no deadlock is possible for every callback graph.

Nor does it inspect the internal lock owner dynamically.

The negative control and source analysis connect the timeout to the known lock cycle.

The result is strong regression evidence for the documented paths, not a universal theorem about lazy initialization.

A deadlock regression needs both a positive and negative meaning

For the fixed code, the test must prove:

Completion occurred.

For the old code, it must prove:

The scenario would have failed to complete.

The commit records both.

Without the negative control, the script might pass because it never reached the defective branch.

Without the positive completion marker, the child might exit without completing the intended callback chain.

The full evidence is:

Old code
→ timeout

New code
→ zero exit
→ “done” printed

A passing path should complete far before the timeout

A useful deadlock test has a wide separation between:

normal completion time

and:

failure deadline

The scripts are small enough that a successful run should finish quickly.

The sixty-second timeout is deliberately much larger.

This reduces ambiguity.

Completes quickly
→ clear PASS

Never completes
→ deadline reached
→ clear failure

If normal execution routinely approached the deadline, the timeout would mix performance variability with deadlock detection.

The source comment explicitly avoids that design.

A timeout should not become an ordinary performance threshold

The test is not asserting:

PyTorch initialization must finish in under sixty seconds

as a product-performance contract.

It is asserting:

This minimal script must not wait indefinitely.

A system-level initialization benchmark would need a much tighter and better-calibrated threshold.

The regression test uses time only to bound nontermination.

Why the failure message includes the child command

On timeout, the helper includes:

e.cmd

in its failure report.

This matters because the same helper runs two different scripts.

A CI failure should identify which reproduction was launched.

Initialized set_rng_state route

or

queued manual_seed route

The command text helps distinguish them without requiring the parent to inspect a deadlocked child stack.

Captured stdout and stderr support non-deadlock diagnosis

If the child exits with a nonzero return code, the helper reports:

stdout

stderr

This distinguishes:

Timeout
→ likely nontermination

from:

Fast failure
→ exception, import problem, backend error, or crash

A test that classified every non-success as “deadlock” would erase useful evidence.

The helper preserves different failure branches.

The test boundary matches the failure boundary

The defect could trap:

one Python interpreter

one process-local lock

one process-local GPU initialization lifecycle

The test isolates exactly that unit.

One reproduction
→ one process

The process is the smallest practical boundary that can be abandoned safely if the interpreter stops making progress.

This is an example of a general testing rule:

Contain a nontermination defect inside a resource boundary that the test controller does not share.

Similar process isolation appears in other low-level tests for the same reason

GPU and compiler regression tests may also use subprocesses when a defect can:

deadlock a CUDA context,
block a runtime initialization lock,
terminate the interpreter,
poison process-global state,
or leave the main test environment unrecoverable.

The specific resource differs.

The common principle is:

Potentially unrecoverable child state
        ↓
separate process
        ↓
parent retains a finite decision path

What the patch directly added to regression coverage

Commit cec50d89 directly added:

a common subprocess helper,
a 60-second timeout,
return-code validation,
a done completion-marker check,
an initialized-path reproduction through set_rng_state(),
a queued-path reproduction through manual_seed(),
and source changes corresponding to both tests.

What remains outside the regression coverage

The landed tests do not directly cover:

a forced multi-thread run_now interleaving,
arbitrary recursive callback graphs,
unrelated lock-order deadlocks,
XPU callback reentrancy,
MTIA callback reentrancy,
performance of the fixed path,
every CUDA/ROCm backend and architecture,
or repeated initialization across forked-process scenarios.

The suite protects the two documented self-deadlocks.

The complete three-part structure

Part 1 — A callback ran while its own lock was held

Non-reentrant lock
+
callback under lock
+
callback re-enters _lazy_call
→ self-deadlock

The repair moved the initialized callback outside the critical section.

Part 2 — Initialization had a real transitional state

Native backend initialized

Queued callbacks still draining

_initialized still False

Initializer thread still owns lock

_tls.is_initializing granted only that thread permission to re-enter, while run_now handled another thread completing initialization during the lock race.

Part 3 — The test runner had to remain outside the failure

Potential interpreter deadlock
        ↓
fresh subprocess
        ↓
finite timeout
        ↓
return-code and completion-marker checks

The current source also shows similar XPU and MTIA lock patterns, but runtime failure remains unconfirmed for those backends.

The final lesson is about testing loss of progress

A numerical test asks:

Did the function return the right value?

A deadlock test must first ask:

Did the function return at all?

That question changes the design of the test.

The assertion cannot live behind the operation that may never finish.

The observer cannot share the lock that may never be released.

The environment should not remain contaminated after failure.

PyTorch’s solution was:

Child process
→ enters the risky lifecycle

Parent process
→ retains the deadline and verdict

A regression test for nontermination must preserve one execution path that the nonterminating code cannot block.

That is what the subprocess supplied.

The source correction repaired the callback boundary.

The process boundary made that repair testable.

PyTorch #182948 — Why torch.cuda._lazy_call() Deadlocked When Its Callback Re-entered It
PyTorch #182948 — Why is_initialized() Was Not Enough to Prevent _lazy_call() Deadlocks

Related material

Patch status: Landed in PyTorch main as commit cec50d89
Regression boundary: One fresh Python subprocess per deadlock scenario
Failure deadline: 60 seconds
Pass requirements: Zero exit status and done in standard output
Initialized-path test: Re-entry through torch.cuda.set_rng_state()
Queued-path test: Re-entry through torch.cuda.manual_seed() during _lazy_init()
Recorded validation: Both scenarios timed out before the fix and completed afterward on a ROCm gfx90a host
Current XPU status: Similar callback-under-lock source structure remains visible; runtime deadlock not established here
Current MTIA status: Similar callback-under-lock source structure remains visible; runtime deadlock not established here

This is Part 3 and the final article in the PyTorch lazy-initialization reentrancy series.

Part 1 examined how a callback executed under a non-reentrant lock could make one thread wait for itself.

Part 2 separated the fully initialized state, the cross-thread run_now race, and the thread-local transitional state used while queued callbacks were still being drained.

This final article examined why both known deadlock paths were isolated in fresh subprocesses, what the timeout and completion marker actually prove, and why similar XPU and MTIA source patterns should not be described as confirmed backend failures without their own reproductions.

#PyTorch #CUDA #ROCm #Python #Concurrency #Deadlock #LazyInitialization #Subprocess #RegressionTesting #XPU #MTIA #CodeAnalysis