PyTorch PR #196901: Why torch.compile Accepted Invalid Length Hints

PyTorch PR #196901 repairs a difference between ordinary Python execution and torch.compile: Dynamo could return a raw __length_hint__() result that CPython would reject or replace with a default.

A negative estimate could escape as a negative integer. A string could escape as a string. NotImplemented could become the answer instead of asking the caller to use its fallback value.

The problem was in Dynamo's handling of a Python operation, before any optimized tensor kernel was needed. The patch adds argument conversion, result validation, fallback handling, and iterator-state support. It landed through PyTorch MergeBot on September 19, 2026.

A documented exception remains for subclasses of TypeError. That boundary matters when assessing what the tests actually prove.

A length estimate still has rules

operator.length_hint(obj, default) first tries the object's actual length, then its __length_hint__() estimate, and finally the default. An estimate may differ from the number of elements eventually produced; the protocol exists to help with operations such as allocating container storage.

That flexibility does not allow an arbitrary return type or a negative value from __length_hint__().

Consider this small object:

import operator

class NegativeHint:
    def __length_hint__(self):
        return -2

operator.length_hint(NegativeHint())
# ValueError: __length_hint__() should return >= 0

The old Dynamo rule returned the result of the method call without the required post-processing. Returning -2 instead of raising ValueError changes which branch runs in a caller that catches the exception.

The same distinction applies to several other outcomes:

What the object does	CPython behavior
Returns `NotImplemented`	Return the supplied default
Raises `TypeError` from the hint method	Return the supplied default
Returns a string	Raise `TypeError`
Returns a negative integer	Raise `ValueError`
Returns an integer outside `Py_ssize_t` range	Raise `OverflowError`
Returns `True`	Return integer `1`, with exact type `int`

These are observable parts of the operation. A caller can inspect the return type or handle a particular error before performing any tensor computation.

The default is validated even when it is unused

One of the less obvious failures involved a successful hint and an invalid fallback argument:

class ValidHint:
    def __length_hint__(self):
        return 2

operator.length_hint(ValidHint(), "abc")
# TypeError

The object can supply a usable answer, but CPython's C entry point still converts the default before looking for that answer. A valid default must support integer-index conversion and fit in the platform's signed size type.

The old Dynamo rule could accept the call because it never needed the fallback value.

The landed implementation now applies pynumber_index, specializes the value where possible, and checks it with pylong_as_ssize_t before querying the object. An invalid unused default therefore remains an error.

The restrictions on a returned hint and on a default are not identical. CPython accepts a negative, in-range default when it must fall back, even though a negative value returned by __length_hint__() raises ValueError. The patch preserves that difference rather than applying one blanket nonnegative check.

The fallback path must handle exceptions in order

Dynamo also needed to preserve the order between __len__() and __length_hint__().

An object may provide both methods while reporting that an exact length is unavailable:

class EstimatedLength:
    def __len__(self):
        raise TypeError("exact length unavailable")

    def __length_hint__(self):
        return 7

CPython clears that TypeError, tries the estimate, and returns 7. The old tracing code returned directly from its length operation, without implementing that recovery step.

The new rule catches Dynamo's ObservedTypeError, clears the tracked exception state, and continues to the hint method. A TypeError raised by the hint itself instead selects the default. Other exception types continue outward.

For a returned hint, the rule handles NotImplemented first, then checks integer type, machine-size range, and nonnegative value. It finally constructs an ordinary integer result.

That last conversion matters for booleans. Python treats bool as an integer subtype, but the C implementation returns an ordinary integer representation. True == 1 can pass a value assertion while the caller still receives the wrong type.

Iterator state is part of the implementation

The patch adds a __length_hint__ method to Dynamo's base list-iterator representation. It reports the remaining item count, floored at zero, and reports zero permanently after exhaustion.

Deque iterators receive a related correction: when mutation causes iteration to raise, Dynamo first marks the iterator exhausted so a later hint reflects the cleared remaining count.

The class-method tracing path also recognizes unbound calls such as:

type(iterator).__length_hint__(iterator)

It delegates to the instance implementation only when the descriptor and exact instance type match. That restriction avoids replacing a base-class method call with a subclass override.

These additions make the fallback machinery reachable for built-in iterator forms rather than fixing only custom Python objects. The source still documents a narrower limitation for reversed-list iterators: their copied item state can miss a later shrink of the original list.

The tests compare errors as well as values

The new parameterized test wraps each operation in a function that returns either a successful result or the exception's type and message. It also performs x + 1 on a tensor.

It compares that function with:

torch.compile(fn, backend="eager", fullgraph=True)

The backend choice isolates Dynamo's tracing behavior from optimized kernel generation. fullgraph=True prevents an unsupported operation from quietly passing the comparison by falling back through a graph break.

The cases include defaults, invalid hints, exception fallback, missing methods, direct and unbound iterator hints, mutated deques, and invalid call signatures.

A separate boolean test checks type(hint) is int. That assertion covers something ordinary value equality would miss.

The patch also removes eight expected-failure markers for the imported CPython operator and iterator-length tests. Removing those markers changes what the suite is expected to pass; it is not independent evidence that every configuration was executed successfully.

One test deliberately records a remaining mismatch

The landed code catches ObservedTypeError, which Dynamo creates for TypeError itself. It does not apply the same fallback to every subclass.

For example:

class MyTypeError(TypeError):
    pass

class UnavailableHint:
    def __length_hint__(self):
        raise MyTypeError("boom")

CPython treats that exception as a TypeError match:

operator.length_hint(UnavailableHint(), 42)
# 42

The new PyTorch test explicitly expects compiled execution to propagate MyTypeError into the caller's exception handler instead. Its expected marker is:

("raise", "MyTypeError", "boom")

This test preserves a known compatibility difference; it does not demonstrate eager/compiled equivalence for that case.

The main repair is well targeted, but calling it complete CPython compatibility would exceed the implementation and its own tests. An application that depends on fallback from a custom TypeError subclass remains outside the repaired behavior at this commit.

Symbolic hints are specialized before rejection

The final code first tries specialize_symnode for both a hint and its default. A shape-derived value that becomes a Python constant can then receive the same type and range checks.

If a value still cannot be validated, Dynamo reports an unsupported case instead of returning unchecked data. The patch adds separate graph-break descriptions for a non-constant result and a non-constant default; the corresponding fullgraph=True tests expect Unsupported.

Consequently, it would be inaccurate to summarize the landed change as rejecting every symbolic hint. The deciding boundary is whether specialization makes validation possible. Specialization may narrow reuse across dynamic shapes, but the PR provides no performance measurement for that tradeoff.

What was verified for this review

I checked the landed implementation against CPython's PyObject_LengthHint source and ran 22 reference cases on CPython 3.13.5. Those checks confirmed the default-validation order, exception fallback, invalid-result errors, boolean-to-integer normalization, and the custom-TypeError reference behavior described above.

Those runs exercised CPython, not the patched Dynamo implementation. The available PyTorch 2.10.0+cpu installation did not contain this BuiltinVariable.call_length_hint method, so it was not a suitable before-and-after runtime for the reviewed change. I did not build the September landing or run its upstream suites.

The source review supports the targeted validation and fallback repair, with the explicitly retained subclass difference. No model-accuracy result or speedup is claimed.

For compiler debugging, the practical lesson is to compare the returned value, its exact type, and the exception path. A Python helper can change the program even when the tensors and their kernels remain untouched.