Skip to content

Commit

Permalink
PEP 747: Fix rules related to UnionType (T1 | T2). Contrast TypeExpr …
Browse files Browse the repository at this point in the history 
…with TypeAlias. Apply other feedback. (#3856)
  • Loading branch information
@davidfstr
davidfstr authored Jul 9, 2024
1 parent 8c02849 commit 1ad2288
Showing 1 changed file with 127 additions and 76 deletions.
203 changes: 127 additions & 76 deletions peps/pep-0747.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -254,9 +254,8 @@ A ``TypeExpr`` value represents a :ref:`type expression <typing:type-expression>
such as ``str | None``, ``dict[str, int]``, or ``MyTypedDict``.
A ``TypeExpr`` type is written as
``TypeExpr[T]`` where ``T`` is a type or a type variable. It can also be
written without brackets as just ``TypeExpr``, in which case a type
checker should apply its usual type inference mechanisms to determine
the type of its argument, possibly ``Any``.
written without brackets as just ``TypeExpr``, which is treated the same as
to ``TypeExpr[Any]``.


Using TypeExprs
Expand All @@ -278,7 +277,6 @@ or a variable type:
::

STR_TYPE: TypeExpr = str # variable type
assert_type(STR_TYPE, TypeExpr[str])

Note however that an *unannotated* variable assigned a type expression literal
will not be inferred to be of ``TypeExpr`` type by type checkers because PEP
Expand Down Expand Up @@ -352,7 +350,7 @@ not spell a type are not ``TypeExpr`` values.
::

OPTIONAL_INT_TYPE: TypeExpr = TypeExpr[int | None] # OK
assert isassignable(Optional[int], OPTIONAL_INT_TYPE)
assert isassignable(int | None, OPTIONAL_INT_TYPE)

.. _non_universal_typeexpr:

Expand Down Expand Up @@ -442,14 +440,29 @@ so must be disambiguated based on its argument type:
- As a value expression, ``Annotated[x, ...]`` has type ``object``
if ``x`` has a type that is not ``type[C]`` or ``TypeExpr[T]``.

**Union**: The type expression ``T1 | T2`` is ambiguous with the value ``int1 | int2``,
so must be disambiguated based on its argument type:
**Union**: The type expression ``T1 | T2`` is ambiguous with
the value ``int1 | int2``, ``set1 | set2``, ``dict1 | dict2``, and more,
so must be disambiguated based on its argument types:

- As a value expression, ``x | y`` has type equal to the return type of ``type(x).__or__``
if ``type(x)`` overrides the ``__or__`` method.

- When ``x`` has type ``builtins.type``, ``types.GenericAlias``, or the
internal type of a typing special form, ``type(x).__or__`` has a return type
in the format ``TypeExpr[T1 | T2]``.

- As a value expression, ``x | y`` has type equal to the return type of ``type(y).__ror__``
if ``type(y)`` overrides the ``__ror__`` method.

- When ``y`` has type ``builtins.type``, ``types.GenericAlias``, or the
internal type of a typing special form, ``type(y).__ror__`` has a return type
in the format ``TypeExpr[T1 | T2]``.

- As a value expression, ``x | y`` has type ``TypeExpr[x | y]``
if ``x`` has type ``TypeExpr[t1]`` (or ``type[t1]``)
and ``y`` has type ``TypeExpr[t2]`` (or ``type[t2]``).
- As a value expression, ``x | y`` has type ``int``
if ``x`` has type ``int`` and ``y`` has type ``int``
- As a value expression, ``x | y`` has type ``UnionType``
in all other situations.

- This rule is intended to be consistent with the preexisting fallback rule
used by static type checkers.

The **stringified type expression** ``"T"`` is ambiguous with both
the stringified annotation expression ``"T"``
Expand All @@ -466,71 +479,24 @@ New kinds of type expressions that are introduced should define how they
will be recognized in a value expression context.


Implicit Annotation Expression Values
'''''''''''''''''''''''''''''''''''''

Although this PEP is mostly concerned with *type expressions* rather than
*annotation expressions*, it is straightforward to extend the rules for
:ref:`recognizing type expressions <implicit_typeexpr_values>`
to similar rules for recognizing annotation expressions,
so this PEP takes the opportunity to define those rules as well:

The following **unparameterized annotation expressions** can be recognized unambiguously:

- As a value expression, ``X`` has type ``object``,
for each of the following values of X:

- ``<TypeAlias>``

The following **parameterized annotation expressions** can be recognized unambiguously:

- As a value expression, ``X`` has type ``object``,
for each of the following values of X:

- ``<Required> '[' ... ']'``
- ``<NotRequired> '[' ... ']'``
- ``<ReadOnly> '[' ... ']'``
- ``<ClassVar> '[' ... ']'``
- ``<Final> '[' ... ']'``
- ``<InitVar> '[' ... ']'``
- ``<Unpack> '[' ... ']'``

**Annotated**: The annotation expression ``Annotated[...]`` is ambiguous with
the type expression ``Annotated[...]``,
so must be :ref:`disambiguated based on its argument type <recognizing_annotated>`.

The following **syntactic annotation expressions**
cannot be recognized in a value expression context at all:

- ``'*' unpackable``
- ``name '.' 'args'`` (where ``name`` must be an in-scope ParamSpec)
- ``name '.' 'kwargs'`` (where ``name`` must be an in-scope ParamSpec)

The **stringified annotation expression** ``"T"`` is ambiguous with both
the stringified type expression ``"T"``
and the string literal ``"T"``, and
cannot be recognized in a value expression context at all:

- As a value expression, ``"T"`` continues to have type ``Literal["T"]``.

No other kinds of annotation expressions currently exist.

New kinds of annotation expressions that are introduced should define how they
will (or will not) be recognized in a value expression context.


Literal[] TypeExprs
'''''''''''''''''''

To simplify static type checking, a ``Literal[...]`` value is *not*
considered assignable to a ``TypeExpr`` variable even if all of its members
spell valid types:
A value of ``Literal[...]`` type is *not* considered assignable to
a ``TypeExpr`` variable even if all of its members spell valid types because
dynamic values are not allowed in type expressions:

::

STRS_TYPE_NAME: Literal['str', 'list[str]'] = 'str'
STRS_TYPE: TypeExpr = STRS_TYPE_NAME # ERROR: Literal[] value is not a TypeExpr

However ``Literal[...]`` itself is still a ``TypeExpr``:

::

DIRECTION_TYPE: TypeExpr[Literal['left', 'right']] = Literal['left', 'right'] # OK


Static vs. Runtime Representations of TypeExprs
'''''''''''''''''''''''''''''''''''''''''''''''
Expand Down Expand Up @@ -569,19 +535,35 @@ Subtyping
Whether a ``TypeExpr`` value can be assigned from one variable to another is
determined by the following rules:

Relationship with type
''''''''''''''''''''''

``TypeExpr[]`` is covariant in its argument type, just like ``type[]``:

- ``TypeExpr[T1]`` is a subtype of ``TypeExpr[T2]`` iff ``T1`` is a
subtype of ``T2``.
- ``type[C1]`` is a subtype of ``TypeExpr[C2]`` iff ``C1`` is a subtype
of ``C2``.

A plain ``type`` can be assigned to a plain ``TypeExpr`` but not the
other way around:
An unparameterized ``type`` can be assigned to an unparameterized ``TypeExpr``
but not the other way around:

- ``type[Any]`` is assignable to ``TypeExpr[Any]``. (But not the
other way around.)

Relationship with UnionType
'''''''''''''''''''''''''''

``TypeExpr[U]`` is a subtype of ``UnionType`` iff ``U`` is
the type expression ``X | Y | ...``:

- ``TypeExpr[X | Y | ...]`` is a subtype of ``UnionType``.

``UnionType`` is assignable to ``TypeExpr[Any]``.

Relationship with object
''''''''''''''''''''''''

``TypeExpr[]`` is a kind of ``object``, just like ``type[]``:

- ``TypeExpr[T]`` for any ``T`` is a subtype of ``object``.
Expand Down Expand Up @@ -623,11 +605,33 @@ Changed signatures
''''''''''''''''''

The following signatures related to type expressions introduce
``TypeExpr`` where previously ``object`` existed:
``TypeExpr`` where previously ``object`` or ``Any`` existed:

- ``typing.cast``
- ``typing.assert_type``

The following signatures transforming union type expressions introduce
``TypeExpr`` where previously ``UnionType`` existed so that a more-precise
``TypeExpr`` type can be inferred:

- ``builtins.type[T].__or__``

- Old: ``def __or__(self, value: Any, /) -> types.UnionType: ...``
- New: ``def __or__[T2](self, value: TypeExpr[T2], /) -> TypeExpr[T | T2]: ...``

- ``builtins.type[T].__ror__``

- Old: ``def __ror__(self, value: Any, /) -> types.UnionType: ...``
- New: ``def __ror__[T1](self, value: TypeExpr[T1], /) -> TypeExpr[T1 | T]: ...``

- ``types.GenericAlias.{__or__,__ror__}``
- «the internal type of a typing special form»``.{__or__,__ror__}``

However the implementations of those methods continue to return ``UnionType``
instances at runtime so that runtime ``isinstance`` checks like
``isinstance('42', int | str)`` and ``isinstance(int | str, UnionType)``
continue to work.


Unchanged signatures
''''''''''''''''''''
Expand Down Expand Up @@ -662,12 +666,32 @@ not propose those changes now:

- Returns annotation expressions

The following signatures accepting union type expressions continue
to use ``UnionType``:

- ``builtins.isinstance``
- ``builtins.issubclass``
- ``typing.get_origin`` (used in an ``@overload``)

The following signatures transforming union type expressions continue
to use ``UnionType`` because it is not possible to infer a more-precise
``TypeExpr`` type:

- ``types.UnionType.{__or__,__ror__}``


Backwards Compatibility
=======================

Previously the rules for recognizing type expression objects
in a value expression context were not defined, so static type checkers
As a value expression, ``X | Y`` previously had type ``UnionType`` (via :pep:`604`)
but this PEP gives it the more-precise static type ``TypeExpr[X | Y]``
(a subtype of ``UnionType``) while continuing to return a ``UnionType`` instance at runtime.
Preserving compability with ``UnionType`` is important because ``UnionType``
supports ``isinstance`` checks, unlike ``TypeExpr``, and existing code relies
on being able to perform those checks.

The rules for recognizing other kinds of type expression objects
in a value expression context were not previously defined, so static type checkers
`varied in what types were assigned <https://discuss.python.org/t/typeform-spelling-for-a-type-annotation-object-at-runtime/51435/34>`_
to such objects. Existing programs manipulating type expression objects
were already limited in manipulating them as plain ``object`` values,
Expand Down Expand Up @@ -711,12 +735,38 @@ assigned to variables and manipulated like any other data in a program:
``TypeExpr[]`` is how you spell the type of a variable containing a
type annotation object describing a type.

``TypeExpr[]`` is similar to ``type[]``, but ``type[]`` can only used to
``TypeExpr[]`` is similar to ``type[]``, but ``type[]`` can only
spell simple **class objects** like ``int``, ``str``, ``list``, or ``MyClass``.
``TypeExpr[]`` by contrast can additionally spell more complex types,
including those with brackets (like ``list[int]``) or pipes (like ``int | None``),
and including special types like ``Any``, ``LiteralString``, or ``Never``.

A ``TypeExpr`` variable looks similar to a ``TypeAlias`` definition, but
can only be used where a dynamic value is expected.
``TypeAlias`` (and the ``type`` statement) by contrast define a name that can
be used where a fixed type is expected:

- Okay, but discouraged in Python 3.12+:

::

MaybeFloat: TypeAlias = float | None
def sqrt(n: float) -> MaybeFloat: ...

- Yes:

::

type MaybeFloat = float | None
def sqrt(n: float) -> MaybeFloat: ...

- No:

::

maybe_float: TypeExpr = float | None
def sqrt(n: float) -> maybe_float: ... # ERROR: Can't use TypeExpr value in a type annotation

It is uncommon for a programmer to define their *own* function which accepts
a ``TypeExpr`` parameter or returns a ``TypeExpr`` value. Instead it is more common
for a programmer to pass a literal type expression to an *existing* function
Expand Down Expand Up @@ -891,8 +941,9 @@ The following will be true when
`mypy#9773 <https://github.com/python/mypy/issues/9773>`__ is implemented:

The mypy type checker supports ``TypeExpr`` types.
A reference implementation of the runtime component is provided in the
``typing_extensions`` module.

A reference implementation of the runtime component is provided in the
``typing_extensions`` module.


Rejected Ideas
Expand Down

0 comments on commit 1ad2288

Please sign in to comment.