Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PEP 747: Fix rules related to UnionType (T1 | T2). Contrast TypeExpr with TypeAlias. Apply other feedback. #3856

Merged
merged 11 commits into from
Jul 9, 2024
Merged

PEP 747: Fix rules related to UnionType (T1 | T2). Contrast TypeExpr with TypeAlias. Apply other feedback. #3856

Changes from 8 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
34a7b65
Implicit Annotation Expression Values: Delete section
davidfstr Jul 2, 2024
1cb02d0
Subtyping: Replace 'plain type' -> 'unparameterized type'
davidfstr Jul 2, 2024
46842ba
Literal[] TypeExprs: Rephrase. Add new justification. Add clarifying …
davidfstr Jul 2, 2024
7b7e45e
Alter unparameterized TypeExpr to mean TypeExpr[Any]
davidfstr Jul 2, 2024
515ecb1
Optional[T] -> T | None
davidfstr Jul 2, 2024
26204a7
Reference Implemention: typing_extensions implemention now exists
davidfstr Jul 2, 2024
71b5c1a
How to Teach This: Contrast TypeExpr with TypeAlias
davidfstr Jul 2, 2024
c20c9b1
Fix rules related to the value expression (T1 | T2) and UnionType
davidfstr Jul 4, 2024
463a133
Be explicit about what kind of TypeExpr a Literal[...] is
davidfstr Jul 5, 2024
7863e2f
Relationship with UnionType: Retain only the rule required for backwa…
davidfstr Jul 8, 2024
af8ea3d
Grammar fix
davidfstr Jul 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
204 changes: 129 additions & 75 deletions peps/pep-0747.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -254,9 +254,8 @@ A ``TypeExpr`` value represents a :ref:`type expression <typing:type-expression>
such as ``str | None``, ``dict[str, int]``, or ``MyTypedDict``.
A ``TypeExpr`` type is written as
``TypeExpr[T]`` where ``T`` is a type or a type variable. It can also be
written without brackets as just ``TypeExpr``, in which case a type
checker should apply its usual type inference mechanisms to determine
the type of its argument, possibly ``Any``.
written without brackets as just ``TypeExpr``, which is treated the same as
to ``TypeExpr[Any]``.


Using TypeExprs
Expand All @@ -278,7 +277,6 @@ or a variable type:
::

STR_TYPE: TypeExpr = str # variable type
assert_type(STR_TYPE, TypeExpr[str])

Note however that an *unannotated* variable assigned a type expression literal
will not be inferred to be of ``TypeExpr`` type by type checkers because PEP
Expand Down Expand Up @@ -352,7 +350,7 @@ not spell a type are not ``TypeExpr`` values.
::

OPTIONAL_INT_TYPE: TypeExpr = TypeExpr[int | None] # OK
assert isassignable(Optional[int], OPTIONAL_INT_TYPE)
assert isassignable(int | None, OPTIONAL_INT_TYPE)

.. _non_universal_typeexpr:

Expand Down Expand Up @@ -442,14 +440,29 @@ so must be disambiguated based on its argument type:
- As a value expression, ``Annotated[x, ...]`` has type ``object``
if ``x`` has a type that is not ``type[C]`` or ``TypeExpr[T]``.

**Union**: The type expression ``T1 | T2`` is ambiguous with the value ``int1 | int2``,
so must be disambiguated based on its argument type:
**Union**: The type expression ``T1 | T2`` is ambiguous with
the value ``int1 | int2``, ``set1 | set2``, ``dict1 | dict2``, and more,
so must be disambiguated based on its argument types:

- As a value expression, ``x | y`` has type equal to the return type of ``type(x).__or__``
if ``type(x)`` overrides the ``__or__`` method.

- When ``x`` has type ``builtins.type``, ``types.GenericAlias``, or the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how to interpret this statement. The phrase "has type" isn't clear. Are you talking about type equivalence? Assignability?

Also, are these static types or runtime types? I presume it's static types, but if that's the case, then I don't know why GenericAlias is mentioned because that's not a type a static type checker would ever evaluate. It's a runtime implementation detail.

What if the static type of x is a union, and some of the subtypes have a custom __or__ override and some do not? Presumably, this formulation assumes that an expansion of the types of x and y has already been performed, and x and y are not union types?

What if the __or__ method is present, but evaluating it generates a type error (e.g. because y's type is incompatible with the signature)?

We can try to hammer out all of these details, but this is getting really complex. One option is to say that unions never evaluate to TypeExpr unless you use a TypeExpr constructor (i.e. TypeExpr(x | y)). This would also avoid the issue with UnionType.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying to say x | y is evaluated (still) as a normal value expression. If the type of x is type or GenericAlias then the signature in typeshed will be used, which will say that a TypeExpr[X | Y] is returned, as described in §"Changed signatures".

What if the or method is present, but evaluating it generates a type error (e.g. because y's type is incompatible with the signature)?

Use whatever behavior is used now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying to say x | y is evaluated (still) as a normal value expression.

Evaluated by whom? A static type checker? If so, the expression will never be evaluated as GenericAlias because that's not something a static type checker knows or cares about (nor should it).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on your later comments, I now understand that at least pyright special-cases the | operator rather than using typeshed's definitions (for __or__ and __ror__) and using the regular rules for calling an overloaded method.

If there's a desire to continue special-casing the | operator, I might need some help from you to transcribe all the current rules for the | operator (which I doubt are in any specification) so that I can effectively propose a minimal diff to those rules.

internal type of a typing special form, ``type(x).__or__`` has a return type
in the format ``TypeExpr[T1 | T2]``.

- As a value expression, ``x | y`` has type equal to the return type of ``type(y).__ror__``
if ``type(y)`` overrides the ``__ror__`` method.

- When ``y`` has type ``builtins.type``, ``types.GenericAlias``, or the
internal type of a typing special form, ``type(y).__ror__`` has a return type
in the format ``TypeExpr[T1 | T2]``.

- As a value expression, ``x | y`` has type ``TypeExpr[x | y]``
if ``x`` has type ``TypeExpr[t1]`` (or ``type[t1]``)
and ``y`` has type ``TypeExpr[t2]`` (or ``type[t2]``).
- As a value expression, ``x | y`` has type ``int``
if ``x`` has type ``int`` and ``y`` has type ``int``
- As a value expression, ``x | y`` has type ``UnionType``
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This rule says "in all other situations". What other situations are not covered in the above rules? I think they cover everything, right? Can you give an example of types x and y where UnionType would be evaluated?

Copy link
Contributor Author

@davidfstr davidfstr Jul 25, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This rule says "in all other situations". What other situations are not covered in the above rules?

There are no other situations that I am aware of. Perhaps I should I'll just delete the entire bullet.

in all other situations.

- This rule is intended to be consistent with the preexisting fallback rule
used by static type checkers.

The **stringified type expression** ``"T"`` is ambiguous with both
the stringified annotation expression ``"T"``
Expand All @@ -466,71 +479,24 @@ New kinds of type expressions that are introduced should define how they
will be recognized in a value expression context.


Implicit Annotation Expression Values
'''''''''''''''''''''''''''''''''''''

Although this PEP is mostly concerned with *type expressions* rather than
*annotation expressions*, it is straightforward to extend the rules for
:ref:`recognizing type expressions <implicit_typeexpr_values>`
to similar rules for recognizing annotation expressions,
so this PEP takes the opportunity to define those rules as well:

The following **unparameterized annotation expressions** can be recognized unambiguously:

- As a value expression, ``X`` has type ``object``,
for each of the following values of X:

- ``<TypeAlias>``

The following **parameterized annotation expressions** can be recognized unambiguously:

- As a value expression, ``X`` has type ``object``,
for each of the following values of X:

- ``<Required> '[' ... ']'``
- ``<NotRequired> '[' ... ']'``
- ``<ReadOnly> '[' ... ']'``
- ``<ClassVar> '[' ... ']'``
- ``<Final> '[' ... ']'``
- ``<InitVar> '[' ... ']'``
- ``<Unpack> '[' ... ']'``

**Annotated**: The annotation expression ``Annotated[...]`` is ambiguous with
the type expression ``Annotated[...]``,
so must be :ref:`disambiguated based on its argument type <recognizing_annotated>`.

The following **syntactic annotation expressions**
cannot be recognized in a value expression context at all:

- ``'*' unpackable``
- ``name '.' 'args'`` (where ``name`` must be an in-scope ParamSpec)
- ``name '.' 'kwargs'`` (where ``name`` must be an in-scope ParamSpec)

The **stringified annotation expression** ``"T"`` is ambiguous with both
the stringified type expression ``"T"``
and the string literal ``"T"``, and
cannot be recognized in a value expression context at all:

- As a value expression, ``"T"`` continues to have type ``Literal["T"]``.

No other kinds of annotation expressions currently exist.

New kinds of annotation expressions that are introduced should define how they
will (or will not) be recognized in a value expression context.


Literal[] TypeExprs
'''''''''''''''''''

To simplify static type checking, a ``Literal[...]`` value is *not*
considered assignable to a ``TypeExpr`` variable even if all of its members
spell valid types:
A value of ``Literal[...]`` type is *not* considered assignable to
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way this is phrased, it still sounds like you're talking about the expression Literal[...] (where ... is some legal literal value like 1 or "hi"). I think what you mean here is "a value expression whose evaluated type is a literal string expression". If I'm interpreting this correctly, then I agree with the rule, but I think it needs to be reworded because that's not what it currently says.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hopefully the example following the paragraph helps clarify the meaning.

a ``TypeExpr`` variable even if all of its members spell valid types because
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is a "TypeExpr variable"? Is it a variable whose type is declared to be TypeExpr[T] (or a union that includes such a subtype)? If so, what does it mean for a variable to have "members"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again: Hopefully the example following the paragraph helps clarify the meaning.

dynamic values are not allowed in type expressions:

::

STRS_TYPE_NAME: Literal['str', 'list[str]'] = 'str'
STRS_TYPE: TypeExpr = STRS_TYPE_NAME # ERROR: Literal[] value is not a TypeExpr

However ``Literal[...]`` itself is still a ``TypeExpr``:

::

DIRECTION_TYPE: TypeExpr = Literal['left', 'right'] # OK
davidfstr marked this conversation as resolved.
Show resolved Hide resolved


Static vs. Runtime Representations of TypeExprs
'''''''''''''''''''''''''''''''''''''''''''''''
Expand Down Expand Up @@ -569,19 +535,38 @@ Subtyping
Whether a ``TypeExpr`` value can be assigned from one variable to another is
determined by the following rules:

Relationship with type
''''''''''''''''''''''

``TypeExpr[]`` is covariant in its argument type, just like ``type[]``:

- ``TypeExpr[T1]`` is a subtype of ``TypeExpr[T2]`` iff ``T1`` is a
subtype of ``T2``.
- ``type[C1]`` is a subtype of ``TypeExpr[C2]`` iff ``C1`` is a subtype
of ``C2``.

A plain ``type`` can be assigned to a plain ``TypeExpr`` but not the
other way around:
An unparameterized ``type`` can be assigned to an unparameterized ``TypeExpr``
but not the other way around:

- ``type[Any]`` is assignable to ``TypeExpr[Any]``. (But not the
other way around.)

Relationship with UnionType
'''''''''''''''''''''''''''

``TypeExpr[U]`` is a subtype of ``UnionType`` iff ``U`` is a non-empty union type:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this need to be specified? It feels like asking type checkers to understand details of runtime implementation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From §"Backward Compatibility":

As a value expression, X | Y previously had type UnionType (via :pep:604)
but this PEP gives it the more-precise static type TypeExpr[X | Y]
(a subtype of UnionType) while continuing to return a UnionType instance at runtime.
Preserving compability with UnionType is important because UnionType
supports isinstance checks, unlike TypeExpr, and existing code relies
on being able to perform those checks.

Rephrasing:

  • type.__or__ (and other methods) now have return type TypeExpr[X | Y] rather than UnionType
  • Static type checkers need to treat TypeExpr[X | Y] as assignable to UnionType so that existing methods like isinstance which expect a UnionType continue to pass type checking when given a X | Y expression.
    • For example isinstance('words', int | str) needs to still pass type checking even though int | str is now a TypeExpr[int | str] and isinstance expects a UnionType as its second argument.


- ``TypeExpr[X | Y | ...]`` is a subtype of ``UnionType``.
- ``TypeExpr[Union[X, Y, ...]]`` is a subtype of ``UnionType``.
- ``TypeExpr[Optional[X]]`` is a subtype of ``UnionType``.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It isn't at runtime (currently)

Copy link
Contributor Author

@davidfstr davidfstr Jul 5, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed technically I don't think it is, but I don't think the lack of a runtime subtype relationship is observable. §"Interactions with isinstance() and issubclass()" says:

The TypeExpr special form cannot be used as any argument to
issubclass:

So I'd expect the following behavior:

issubclass(TypeExpr[int | str], UnionType)
TypeError: issubclass() arg 1 must be a class

Edit: And indeed I see that behavior with the current implementation of TypeExpr in typing_extensions.

Are there other ways you can think of that a lack of a runtime subtype relationship could be observable?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

from types import UnionType
from typing_extensions import Optional, TypeExpr

def f(x: UnionType):
    assert isinstance(x, UnionType)

def g(x: TypeExpr[Optional[int]]):
    f(x)

g(Optional[int])  # boom

I'm not sure there is much reason for TypeExpr to ever be a subtype of UnionType; I don't think it will significantly help users of TypeExpr.

Copy link
Contributor Author

@davidfstr davidfstr Jul 6, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clever example. OK it seems that if TypeExpr is to (sometimes) be a subtype of UnionType then there are ways to observe it runtime.

I'm not sure there is much reason for TypeExpr to ever be a subtype of UnionType; I don't think it will significantly help users of TypeExpr.

I also don't think it will help a ton, but I think it's necessary for backward compatibility for existing functions that accept UnionType so long as there's no other way to spell "a TypeExpr that is a non-empty union type". Currently §["Rejected Ideas" > "Support pattern matching on type expressions"] does not provide a spelling that can replace existing usage of UnionType.

Do you have an alternative suggestion in mind that both:

  1. gives a TypeExpr[int | str] result for the value expression int | str and
  2. continues to allow isinstance('words', int | str) to pass a type checker?

I believe it should be possible to make TypeExpr[U] be conditionally considered a subtype of UnionType at runtime (via an isinstance check) by overriding __instancecheck__ on the metaclass of TypeExpr UnionType.

Aside: The current implementation of int | str gives a UnionType, but neither Union[int, str] nor Optional[int] give UnionTypes. If we wanted to more-strictly preserve the existing behavior, I'd be open to narrowing the rules in this section to only make TypeExpr[X | Y | ...] a subtype of UnionType:

  • TypeExpr[X | Y | ...] is a subtype of UnionType.
  • TypeExpr[Union[X, Y, ...]] is not a subtype of UnionType.
  • TypeExpr[Optional[X]] is not a subtype of UnionType.
  • TypeExpr[Never] is not a subtype of UnionType.
  • TypeExpr[NoReturn] is not a subtype of UnionType.

Edit: However I will note that (X | None) == Union[X, None] == Optional[X] at runtime so it could be confusing to users if Union and Optional couldn't be used in the same place as an X | Y expression.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the backwards compatibility problem is an issue of type checker inference that we don't have to specify exactly. It's also a very limited problem, mostly applying to isinstance() which is necessarily special-cased by type checkers anyway.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The following code can be written by users today:

def accept_union(u: UnionType):
    pass

accept_union(int | str)

Are you saying we shouldn’t worry about breaking this code so long as we avoid breaking code related to isinstance?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the way to avoid breaking that code can be up to type checkers. We don't need to prescribe it exactly; different type checkers can use different approaches, and adapt it to changes in how the runtime works. For example, type checkers could store something in their internal representation of a TypeExpr type to indicate the runtime construct that was used to create it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

type checkers could store something in their internal representation of a TypeExpr type to indicate the runtime construct that was used to create it.

Yes, this description is consistent with the implementation approach I had in mypy: A single bit like is_uniontype.

But the overall specification rule being implemented is still:

TypeExpr[X | Y | ...] is a subtype of UnionType

I'll update the diff to include only this rule, and not the extraneous ones for Union and Optional.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With only the remaining rule, the code you mentioned before fails (correctly) at type checking time and at runtime:

from types import UnionType
from typing_extensions import Optional, TypeExpr

def f(x: UnionType):
    assert isinstance(x, UnionType)  # AssertionError

def g(x: TypeExpr[Optional[int]]):
    f(x)  # ERROR: TypeExpr[Optional[int]] is not a UnionType

g(Optional[int])

And similar code involving TypeExpr[X | Y | ...] passes (correctly) both at type checking time and at runtime:

from types import UnionType
from typing_extensions import Optional, TypeExpr

def f(x: UnionType):
    assert isinstance(x, UnionType)  # OK (runtime)

def g(x: TypeExpr[int | None]):
    f(x)  # OK (type checking time)

g(int | None)

- ``TypeExpr[Never]`` is *not* a subtype of ``UnionType``.
- ``TypeExpr[NoReturn]`` is *not* a subtype of ``UnionType``.

``UnionType`` is assignable to ``TypeExpr[Any]``.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, this seems problematic because UnionType is assignable to TypeExpr in some cases and TypeExpr is assignable to UnionType in some cases. This breaks the set-theoretic type definitions that we've carefully crafted in recent updates to the typing spec.


Relationship with object
''''''''''''''''''''''''

``TypeExpr[]`` is a kind of ``object``, just like ``type[]``:

- ``TypeExpr[T]`` for any ``T`` is a subtype of ``object``.
Expand Down Expand Up @@ -623,11 +608,33 @@ Changed signatures
''''''''''''''''''

The following signatures related to type expressions introduce
``TypeExpr`` where previously ``object`` existed:
``TypeExpr`` where previously ``object`` or ``Any`` existed:

- ``typing.cast``
- ``typing.assert_type``

The following signatures transforming union type expressions introduce
``TypeExpr`` where previously ``UnionType`` existed so that a more-precise
``TypeExpr`` type can be inferred:

- ``builtins.type[T].__or__``
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Type checkers already special-case the | operator, so the proposed change here is unnecessary — at least from the perspective of static type checkers. Maybe there's some other reason that I'm not considering that would justify such a change.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was assuming that static type checkers actually read typeshed to figure out what to do with the | operator.

Thus the typeshed modifications I've proposed here are intended to get the correct effect (such that the value expression T1 | T2 is deduced to have type TypeExpr[T1 | T2]) if a typechecker were to actually consult the stubs.


- Old: ``def __or__(self, value: Any, /) -> types.UnionType: ...``
- New: ``def __or__[T2](self, value: TypeExpr[T2], /) -> TypeExpr[T | T2]: ...``

- ``builtins.type[T].__ror__``

- Old: ``def __ror__(self, value: Any, /) -> types.UnionType: ...``
- New: ``def __ror__[T1](self, value: TypeExpr[T1], /) -> TypeExpr[T1 | T]: ...``

- ``types.GenericAlias.{__or__,__ror__}``
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Type checkers never look at GenericAlias, so the proposed change is unnecessary unless there's some other justification for making it.

- «the internal type of a typing special form»``.{__or__,__ror__}``

However the implementations of those methods continue to return ``UnionType``
instances at runtime so that runtime ``isinstance`` checks like
``isinstance('42', int | str)`` and ``isinstance(int | str, UnionType)``
continue to work.


Unchanged signatures
''''''''''''''''''''
Expand Down Expand Up @@ -662,12 +669,32 @@ not propose those changes now:

- Returns annotation expressions

The following signatures accepting union type expressions continue
to use ``UnionType``:

- ``builtins.isinstance``
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Type checkers already need to do significant special-casing for isinstance and issubclass calls. Maybe we just change these to accept TypeExpr rather than (or in addition to) UnionType and leave it up to the custom type-checker logic to detect and report cases that the runtime implementation of isinstance and issubclass are not able to handle. That would eliminate the need for TypeExpr to be a subtype of UnionType (as proposed above, which I think is very problematic).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we just change these to accept TypeExpr rather than (or in addition to) UnionType ...

Sounds good to me. I think Jelle made a similar suggestion elsewhere in the comments for this PR.

- ``builtins.issubclass``
- ``typing.get_origin`` (used in an ``@overload``)

The following signatures transforming union type expressions continue
to use ``UnionType`` because it is not possible to infer a more-precise
``TypeExpr`` type:

- ``types.UnionType.{__or__,__ror__}``


Backwards Compatibility
=======================

Previously the rules for recognizing type expression objects
in a value expression context were not defined, so static type checkers
As a value expression, ``X | Y`` previously had type ``UnionType`` (via :pep:`604`)
but this PEP gives it the more-precise static type ``TypeExpr[X | Y]``
(a subtype of ``UnionType``) while continuing to return a ``UnionType`` instance at runtime.
Preserving compability with ``UnionType`` is important because ``UnionType``
supports ``isinstance`` checks, unlike ``TypeExpr``, and existing code relies
on being able to perform those checks.

The rules for recognizing other kinds of type expression objects
in a value expression context were not previously defined, so static type checkers
`varied in what types were assigned <https://discuss.python.org/t/typeform-spelling-for-a-type-annotation-object-at-runtime/51435/34>`_
to such objects. Existing programs manipulating type expression objects
were already limited in manipulating them as plain ``object`` values,
Expand Down Expand Up @@ -717,6 +744,32 @@ spell simple **class objects** like ``int``, ``str``, ``list``, or ``MyClass``.
including those with brackets (like ``list[int]``) or pipes (like ``int | None``),
and including special types like ``Any``, ``LiteralString``, or ``Never``.

A ``TypeExpr`` variable looks similar to a ``TypeAlias`` definition, but
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is a "TypeExpr variable"? I think what you mean is that a variable (or parameter) can be statically evaluated to have a type of TypeExpr[T]?

By "TypeAlias definition", are you talking about a statement of the form <name>: TypeAlias = <type expression>, as define din PEP 613? I'm not sure how this is related to TypeExpr.

Perhaps you're talking about PEP 484 type aliases that have the syntactic form <name> = <expression> and numerous (undocumented) semantic rules and heuristics that distinguish it from a regular variable assignment? If that's the case, then I agree there's potential overlap with the TypeExpr concept. In particular, I was thinking that we could leverage the definitions in this PEP to (at last!) formalize the rules for PEP 484 type aliases. I'm now less sure of this given some of the other limitations we've needed to add to this PEP, such as the requirement that certain ambiguous forms must use an explicit TypeExpr constructor call.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll alter the original line with inline examples like:

A TypeExpr variable (maybe_float: TypeExpr) looks similar to a TypeAlias definition (MaybeFloat: TypeAlias) ...

Hopefully also the examples following the paragraph help clarify the meaning.

can only be used where a dynamic value is expected.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand what "only be used where a dynamic value is expected"? What is a "dynamic value" in this context?

``TypeAlias`` (and the ``type`` statement) by contrast define a name that can
be used where a fixed type is expected:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The term "fixed type" isn't defined anywhere. I think what you mean is that a type alias can be used in a type expression whereas variables cannot?


- Okay, but discouraged in Python 3.12+:

::

MaybeFloat: TypeAlias = float | None
def sqrt(n: float) -> MaybeFloat: ...

- Yes:

::

type MaybeFloat = float | None
def sqrt(n: float) -> MaybeFloat: ...

- No:

::

maybe_float: TypeExpr = float | None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I think I now understand what you were trying to say above. This is all a (confusing) way to reiterate that a variable cannot be used in a type expression. That rule is already spelled out clearly in the "Type Annotations" section of the spec, so I don't think it needs to be repeated in this PEP.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 non-expert commentators have been confused about when to use TypeAlias vs when to use TypeExpr, since either can be used in certain contexts. This subsection hopes to clarify.

Copy link
Contributor

@erictraut erictraut Jul 25, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the intent, but I don't think it's achieving the clarity you're seeking. I'm finding it to be very confusing. I think it could be made clear if it were reworked, but I'm not convinced this paragraph is needed.

Type alias definitions and the TypeExpr symbol are very different, so there should be no confusion between the two. The current name (TypeExpr) is likely contributing to the confusion. Perhaps once the name is fixed, it will become clearer to non-expert readers.

def sqrt(n: float) -> maybe_float: ... # ERROR: Can't use TypeExpr value in a type annotation

It is uncommon for a programmer to define their *own* function which accepts
a ``TypeExpr`` parameter or returns a ``TypeExpr`` value. Instead it is more common
for a programmer to pass a literal type expression to an *existing* function
Expand Down Expand Up @@ -891,8 +944,9 @@ The following will be true when
`mypy#9773 <https://github.com/python/mypy/issues/9773>`__ is implemented:

The mypy type checker supports ``TypeExpr`` types.
A reference implementation of the runtime component is provided in the
``typing_extensions`` module.

A reference implementation of the runtime component is provided in the
``typing_extensions`` module.


Rejected Ideas
Expand Down
Loading