Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support ParamSpec for TypeAliasType #449

Open
wants to merge 63 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
b6bc323
Support ParamSpec for TypeAliasType
Daraan Aug 21, 2024
79985f3
Removed trailing whitespaces
Daraan Aug 21, 2024
02025d4
Merge branch 'main' into TypeAliasType-extension
Daraan Sep 9, 2024
3199b7b
raise TypeError for invalid type_params like typing.TypeAliasType
Daraan Sep 18, 2024
f7d79d9
Keep dunder attributes like typing.TypeAliasType
Daraan Sep 18, 2024
c4c0e68
Added test to catch invalid cases, fix wrong intendation
Daraan Sep 18, 2024
d6af983
removed pyright pragma
Daraan Sep 18, 2024
0a5039b
fix whitespaces
Daraan Sep 18, 2024
e532429
Avoid incompatible global types
Daraan Sep 19, 2024
9911ac7
Add __name__ for TypeAliasType for <=3.10
Daraan Sep 19, 2024
e1b3095
Added missing support for Unpack to TypeAliasType variant
Daraan Sep 19, 2024
408ae2e
Added more invalid test cases
Daraan Sep 19, 2024
621085f
Merge branch 'main' into TypeAliasType-extension
Daraan Sep 19, 2024
b3e6b7a
Unpack invalid Concatenate correctly
Daraan Sep 19, 2024
c3d98c6
Removed parameter checking from TypeAliasType._check_parameter
Daraan Sep 23, 2024
8255667
extended and reworked tests
Daraan Sep 23, 2024
255de76
Clean duplicated tests
Daraan Sep 23, 2024
3037923
Removed duplicated or valid cases from invalid
Daraan Sep 23, 2024
b8ae82e
Slightly more refined tests covering more cases
Daraan Sep 23, 2024
8d2ec0a
Removed cases that is mentioned elsewhere
Daraan Sep 23, 2024
7029d51
Revert change of global Protocol variables
Daraan Sep 24, 2024
02fd0ba
Raise TypeError on parameterless alias
Daraan Sep 24, 2024
5c0938c
Merge remote-tracking branch 'upstream/main' into TypeAliasType-exten…
Daraan Sep 24, 2024
b6fefb0
Correct subscription handling when type_params are empty
Daraan Sep 24, 2024
af0a133
Restructured test cases
Daraan Sep 24, 2024
f2aa35c
Added general and comprehensive test
Daraan Sep 24, 2024
6b1bafb
Subtests to test parameter amount
Daraan Sep 24, 2024
7c1fea7
unify __getitem__ again
Daraan Sep 24, 2024
efa1214
covered all cases for TypeErrors during subscription
Daraan Sep 25, 2024
66eebb1
Removed code addressing #468
Daraan Sep 25, 2024
df10751
Use _types.GenericAlias for TypeAliasType in 3.10+
Daraan Sep 25, 2024
889e9ae
Assure that args and parameter tests pass
Daraan Sep 25, 2024
b6b5a14
Remove code to fix dunder attributes -> other PR
Daraan Sep 26, 2024
9562635
small reordering
Daraan Sep 26, 2024
0033813
Merge remote-tracking branch 'upstream/main' into TypeAliasType-exten…
Daraan Sep 26, 2024
014109c
Remove invalid case
Daraan Sep 26, 2024
0b3ce7d
Updated to latest changes from main
Daraan Sep 26, 2024
0ae4c63
revert mistakes of wrong merge
Daraan Sep 26, 2024
144c7b8
Merge branch 'main' into TypeAliasType-extension
AlexWaygood Sep 26, 2024
e613294
No need to skip tests anymore
Daraan Sep 26, 2024
e44fdcc
Merge remote-tracking branch 'origin/TypeAliasType-extension' into Ty…
Daraan Sep 26, 2024
6bc1f57
removed tests related to: https://github.com/python/cpython/issues/12…
Daraan Sep 30, 2024
e8bfa30
Removed tests related to #474
Daraan Sep 30, 2024
617656d
Removed invalid tests
Daraan Sep 30, 2024
41a87b8
minor comment update
Daraan Sep 30, 2024
a8c4bda
Merge branch 'main' into TypeAliasType-extension
Daraan Sep 30, 2024
249b869
More refined skip reason
Daraan Sep 30, 2024
e71902e
Remove type check for tuples; handled by #477
Daraan Oct 1, 2024
b8799ce
updated changelog
Daraan Oct 1, 2024
5bc1360
3.8, 3.9 use collected parameters more explicitly
Daraan Oct 1, 2024
3ae9e35
Correct error message for 3.10
Daraan Oct 2, 2024
9878cc0
Merge branch 'main' into TypeAliasType-extension
Daraan Oct 11, 2024
3863c8b
Merge remote-tracking branch 'upstream/main' into TypeAliasType-exten…
Daraan Oct 21, 2024
8c86619
Merge remote-tracking branch 'upstream/main' into TypeAliasType-exten…
Daraan Oct 22, 2024
251d312
Merge remote-tracking branch 'upstream/main' into TypeAliasType-exten…
Daraan Oct 22, 2024
b3aa598
Generator not necessary anymore
Daraan Oct 22, 2024
07ba7b7
Merge remote-tracking branch 'upstream/main' into TypeAliasType-exten…
Daraan Oct 22, 2024
28d1e84
Merge remote-tracking branch 'upstream/main' into TypeAliasType-exten…
Daraan Oct 25, 2024
a365355
Merge and type correction
Daraan Oct 25, 2024
fb55b88
TODO: handle None
Daraan Oct 25, 2024
f2d6890
Merge remote-tracking branch 'upstream/main' into TypeAliasType-exten…
Daraan Oct 25, 2024
054f083
Merge remote-tracking branch 'upstream/main' into TypeAliasType-exten…
Daraan Oct 28, 2024
eb840ef
Handle None case correctly, also check list
Daraan Oct 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
124 changes: 119 additions & 5 deletions src/test_typing_extensions.py
Original file line number Diff line number Diff line change
Expand Up @@ -3572,13 +3572,13 @@ class P(Protocol):
self.assertEqual(Alias, Alias2)

def test_protocols_pickleable(self):
global P, CP # pickle wants to reference the class by name
global GlobalProto, CP # pickle wants to reference the class by name
T = TypeVar('T')

@runtime_checkable
class P(Protocol[T]):
class GlobalProto(Protocol[T]):
x = 1
class CP(P[int]):
class CP(GlobalProto[int]):
pass

c = CP()
Expand All @@ -3591,7 +3591,7 @@ class CP(P[int]):
self.assertEqual(x.bar, 'abc')
self.assertEqual(x.x, 1)
self.assertEqual(x.__dict__, {'foo': 42, 'bar': 'abc'})
s = pickle.dumps(P)
s = pickle.dumps(GlobalProto)
D = pickle.loads(s)
Daraan marked this conversation as resolved.
Show resolved Hide resolved
class E:
x = 1
Expand Down Expand Up @@ -7165,13 +7165,38 @@ def test_attributes(self):
self.assertEqual(ListOrSetT.__type_params__, (T,))
self.assertEqual(ListOrSetT.__parameters__, (T,))

subscripted = ListOrSetT[int]
AlexWaygood marked this conversation as resolved.
Show resolved Hide resolved
self.assertEqual(subscripted.__name__, "ListOrSetT")
self.assertEqual(subscripted.__value__, Union[List[T], Set[T]],)
self.assertEqual(subscripted.__type_params__, (T, ))
self.assertEqual(subscripted.__parameters__, ())

T2 = TypeVar("T2")
subscriptedT = ListOrSetT[T2]
self.assertEqual(subscriptedT.__name__, "ListOrSetT")
self.assertEqual(subscriptedT.__value__, Union[List[T], Set[T]],)
self.assertEqual(subscriptedT.__type_params__, (T, ))
self.assertEqual(subscriptedT.__parameters__, (T2, ))

Ts = TypeVarTuple("Ts")
Variadic = TypeAliasType("Variadic", Tuple[int, Unpack[Ts]], type_params=(Ts,))
self.assertEqual(Variadic.__name__, "Variadic")
self.assertEqual(Variadic.__value__, Tuple[int, Unpack[Ts]])
self.assertEqual(Variadic.__type_params__, (Ts,))
self.assertEqual(Variadic.__parameters__, tuple(iter(Ts)))

subscripted_tuple = Variadic[Unpack[Tuple[int, float]]]
self.assertEqual(subscripted_tuple.__name__, "Variadic")
self.assertEqual(subscripted_tuple.__value__, Tuple[int, Unpack[Ts]])
self.assertEqual(subscripted_tuple.__type_params__, (Ts,))
self.assertEqual(subscripted_tuple.__parameters__, ())

subscripted_tupleT = Variadic[Unpack[Tuple[int, T]]]
self.assertEqual(subscripted_tupleT.__name__, "Variadic")
self.assertEqual(subscripted_tupleT.__value__, Tuple[int, Unpack[Ts]])
self.assertEqual(subscripted_tupleT.__type_params__, (Ts,))
self.assertEqual(subscripted_tupleT.__parameters__, (T, ))

def test_cannot_set_attributes(self):
Simple = TypeAliasType("Simple", int)
with self.assertRaisesRegex(AttributeError, "readonly attribute"):
Expand Down Expand Up @@ -7232,12 +7257,13 @@ def test_or(self):
Alias | "Ref"

def test_getitem(self):
T = TypeVar("T")
ListOrSetT = TypeAliasType("ListOrSetT", Union[List[T], Set[T]], type_params=(T,))
subscripted = ListOrSetT[int]
self.assertEqual(get_args(subscripted), (int,))
self.assertIs(get_origin(subscripted), ListOrSetT)
with self.assertRaises(TypeError):
subscripted[str]
subscripted[int] # TypeError: ListOrSetT[int] is not a generic class

still_generic = ListOrSetT[Iterable[T]]
self.assertEqual(get_args(still_generic), (Iterable[T],))
Expand All @@ -7246,6 +7272,94 @@ def test_getitem(self):
self.assertEqual(get_args(fully_subscripted), (Iterable[float],))
self.assertIs(get_origin(fully_subscripted), ListOrSetT)

# Test ParamSpec and Ellipsis
P = ParamSpec('P')
CallableP = TypeAliasType("CallableP", Callable[P, Any], type_params=(P,))
# () -> Any
callable_no_arg = CallableP[[]]
self.assertEqual(get_args(callable_no_arg), ([],))
# (int) -> Any
callable_arg_raw = CallableP[int]
self.assertEqual(get_args(callable_arg_raw), (int,))
callable_arg = CallableP[[int]]
self.assertEqual(get_args(callable_arg), ([int],))
# (int, int) -> Any
callable_arg2 = CallableP[[int, int]]
self.assertEqual(get_args(callable_arg2), ([int, int],))
# (...) -> Any
callable_ellipsis = CallableP[...]
self.assertEqual(get_args(callable_ellipsis), (...,))
callable_ellipsis2 = CallableP[(...,)]
self.assertEqual(callable_ellipsis, callable_ellipsis2)
# (int, ...) -> Any
callable_arg_more = CallableP[[int, ...]]
self.assertEqual(get_args(callable_arg_more), ([int, ...],))
# (T) -> Any
callable_generic = CallableP[[T]]
self.assertEqual(get_args(callable_generic), ([T],))
callable_generic_raw = CallableP[T]
self.assertEqual(get_args(callable_generic_raw), (T,))

# test invalid usage
if not TYPING_3_11_0:
with self.assertRaises(TypeError):
ListOrSetT[Generic[T]]
with self.assertRaises(TypeError):
ListOrSetT[(Generic[T], )]

def test_invalid_cases(self):
Daraan marked this conversation as resolved.
Show resolved Hide resolved
# If these cases fail the specificiation might have changed
T = TypeVar("T")
T2 = TypeVar("T2")
ListOrSetT = TypeAliasType("ListOrSetT", Union[List[T], Set[T]], type_params=(T,))
too_many = ListOrSetT[int, bool]

self.assertEqual(get_args(too_many), (int, bool))
self.assertEqual(too_many.__parameters__, ())

ListOrSet2T = TypeAliasType("ListOrSet2T", Union[List[T], Set[T2]], type_params=(T, T2))
not_enough = ListOrSet2T[int]
self.assertEqual(get_args(not_enough), (int,))
self.assertEqual(not_enough.__parameters__, ())

not_enough2 = ListOrSet2T[T]
self.assertEqual(get_args(not_enough2), (T,))
self.assertEqual(not_enough2.__parameters__, (T,))
# ParamSpec
P = ParamSpec('P')
CallableP = TypeAliasType("CallableP", Callable[P, T], type_params=(P,))

callable_not_enough = CallableP[int]
self.assertEqual(callable_not_enough.__parameters__, ())
self.assertEqual(get_args(callable_not_enough), (int, ))

callable_too_many = CallableP[str, float, T2, int]
self.assertEqual(callable_too_many.__parameters__, (T2, ))
self.assertEqual(get_args(callable_too_many), (str, float, T2, int, ))

# Test with Concatenate
callable_concat = CallableP[Concatenate[Any, T2, P], Any]
reveal_type(callable_concat)
self.assertEqual(callable_concat.__parameters__, (T2, P))
self.assertEqual(get_args(callable_concat), (Concatenate[Any, T2, P], Any))

# TypeVarTuple
Ts = TypeVarTuple("Ts")
Variadic = TypeAliasType("Variadic", Tuple[int, Unpack[Ts]], type_params=(Ts,))
# No Unpack
invalid_tuple_A = Variadic[Tuple[int, T]]
self.assertEqual(invalid_tuple_A.__parameters__, (T, ))
self.assertEqual(get_args(invalid_tuple_A), (Tuple[int, T], ))

# To type tuple
invalid_tuple_B = Variadic[int, T]
self.assertEqual(invalid_tuple_B.__parameters__, (T, ))

# No Tuple, but list
invalud_tuple_C = Variadic[[int, T]]
self.assertEqual(invalud_tuple_C.__parameters__, ())
self.assertEqual(get_args(invalud_tuple_C), ([int, T],))

def test_pickle(self):
global Alias
Alias = TypeAliasType("Alias", int)
Expand Down
69 changes: 59 additions & 10 deletions src/typing_extensions.py
Original file line number Diff line number Diff line change
Expand Up @@ -3487,6 +3487,8 @@ def __init__(self, name: str, value, *, type_params=()):
self.__type_params__ = type_params

parameters = []
if not isinstance(type_params, tuple):
raise TypeError("type_params must be a tuple")
for type_param in type_params:
if isinstance(type_param, TypeVarTuple):
parameters.extend(type_param)
Expand Down Expand Up @@ -3524,16 +3526,63 @@ def _raise_attribute_error(self, name: str) -> Never:
def __repr__(self) -> str:
return self.__name__

def __getitem__(self, parameters):
if not isinstance(parameters, tuple):
parameters = (parameters,)
parameters = [
typing._type_check(
item, f'Subscripting {self.__name__} requires a type.'
)
for item in parameters
]
return typing._GenericAlias(self, tuple(parameters))
if sys.version_info >= (3, 11):
def __getitem__(self, parameters):
if not isinstance(parameters, tuple):
parameters = (parameters,)
parameters = [
typing._type_check(
item, f'Subscripting {self.__name__} requires a type.'
)
for item in parameters
]
Daraan marked this conversation as resolved.
Show resolved Hide resolved
alias = typing._GenericAlias(self, tuple(parameters))
Daraan marked this conversation as resolved.
Show resolved Hide resolved
alias.__value__ = self.__value__
alias.__type_params__ = self.__type_params__
return alias
else:
def _check_parameter(self, item, typ=_marker):
# Allow [], [int], [int, str], [int, ...], [int, T]
if isinstance(item, _UnpackAlias):
args = (
item.__args__ # e.g. (int, str)
if sys.version_info[:2] >= (3, 9)
else item.__args__ # (typing.Tuple[int, float],)
)
# Unpack
yield from [checked
for arg in args
for checked in self._check_parameter(arg)]
elif item is ...:
yield ...
elif isinstance(item, list) and typ is not _marker:
yield [checked
for arg in item
for checked in self._check_parameter(arg)]
else:
yield typing._type_check(
item, f'Subscripting {self.__name__} requires a type.'
)

def __getitem__(self, parameters):
if not isinstance(parameters, tuple):
parameters = (parameters,)
param_difference = (len(parameters) - len(self.__type_params__))
if param_difference > 0:
# invalid case that does not raise an error, fill with dummys
type_params = [*self.__type_params__, *[Any] * param_difference]
else:
type_params = self.__type_params__
parameters = [
checked
for item, typ in zip(parameters, type_params)
for checked in self._check_parameter(item, typ)
]
alias = typing._GenericAlias(self, tuple(parameters))
alias.__name__ = self.__name__
alias.__value__ = self.__value__
alias.__type_params__ = self.__type_params__
return alias

def __reduce__(self):
return self.__name__
Expand Down