Skip to content

Proposal: Use Parameter[T] type annotation instead of T for better type checker compatibility #3386

@kitagry

Description

@kitagry

Summary

Following the recent type improvements in #3369 and #3370, I'd like to propose updating the recommended type annotation style for Luigi parameters.

Background

In #3370, I added @dataclass_transform support and a .pyi stub file. At that time, I was influenced by the existing mypy plugin (#3315) which uses the s: str = Parameter() style, so I followed the same pattern in the documentation examples.

However, after further investigation, I found that the s: Parameter[str] = Parameter() style works better across all major type checkers.

However, after further investigation, I found that the s: Parameter[str] = Parameter() style works better and doesn't require any of these workarounds - the __get__ and __set__ descriptor protocol handles everything.

Current Style

class MyTask(luigi.Task):
    s: str = luigi.Parameter()
    n: int = luigi.IntParameter()

Proposed Style

class MyTask(luigi.Task):
    s: Parameter[str] = luigi.Parameter()
    n: IntParameter = luigi.IntParameter()

Type Checker Comparison

Assignment Error at Class Definition

Type Checker Style 1 (s: str) Style 2 (s: Parameter[str])
pyright ❌ Error ✅ No error
mypy ❌ Error ✅ No error
pyrefly ❌ Error ✅ No error
ty ❌ Error ✅ No error

Type Inference (Both styles work correctly)

Access Style 1 Style 2
task.s str str
task.n int int
Task.__init__ (s: str, n: int) (s: str, n: int)
Task.s str Parameter[str] (more accurate)

Wrong Type Detection in init (Both styles work correctly)

Both styles correctly detect this error:

task = MyTask(s="hello", n="42") # Error: n should be int!

Type Checker Style 1 Style 2
pyright ✅ Detects error ✅ Detects error
mypy ✅ Detects error ✅ Detects error
pyrefly ✅ Detects error ✅ Detects error
ty ✅ Detects error ✅ Detects error

Why Style 2 is Better

  1. No type errors - Works out of the box with all major type checkers
  2. No __new__ override needed - Style 1 requires __new__ to return T to trick type checkers (feat: Add generic type support to Parameter classes #3369)
  3. No .pyi stub file needed - Style 1 requires parameter.pyi for proper type inference (feat: add parameter.pyi for type hinting parameters #3370)
  4. No mypy plugin needed - Style 1 relies on custom mypy plugin (Implement mypy plugin for luigi.Task #3315)
  5. More accurate class access - Task.s correctly shows Parameter[str]
  6. Same type safety - __init__ type checking works identically

Simplification Opportunity

If we adopt Style 2 as the recommended approach, we could potentially:

Suggested Actions

  1. Update documentation examples to use Parameter[T] style
  2. Consider simplifying the type infrastructure (remove __new__ hacks, etc.)
  3. Mention both styles in docs (with Style 2 as recommended)

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions