Skip to content

Do not narrow Flag enums by member equality - #21942

Open
istoolsfox wants to merge 1 commit into
python:masterfrom
istoolsfox:fix/flag-enum-equality-narrowing
Open

Do not narrow Flag enums by member equality#21942
istoolsfox wants to merge 1 commit into
python:masterfrom
istoolsfox:fix/flag-enum-equality-narrowing

Conversation

@istoolsfox

Copy link
Copy Markdown

Fixes #21937

Description

narrow_type_by_identity_equality (used for ==/!=/is/is not narrowing) expands an enum-typed expression into a union of member literals and, in the negative branch, rules out the compared-away member. For enum.Flag subclasses this is unsound: a Flag value can hold any combination of members, so the declared members are not an exhaustive value set.

The reported repro:

class Programs(Flag):
    NONE = 0
    P1 = auto()
    P2 = auto()
    P3 = auto()
    ALL = P1 | P2 | P3

def f(programs: Programs) -> None:
    if programs == Programs.NONE:
        return
    if programs == Programs.P2:
        return
    if Programs.P2 in programs:  # error: Unsupported operand types for in
        ...

After the two == checks, mypy narrows programs to Literal[Programs.P1, Programs.P3, Programs.ALL] -- a type that a runtime value like Programs.P1 | Programs.P2 cannot inhabit. The subsequent Programs.P2 in programs is then rejected: typeshed defines Flag.__contains__(self, other: Self), and no member of the narrowed union accepts a P2 value, even though Programs.P2 in Programs.P1 | Programs.P2 is True at runtime.

This change skips equality/identity narrowing when either operand is a Flag value (an instance of, or a member of, an enum.Flag/enum.IntFlag subclass): the sound type for such a value is the enum class itself. Narrowing for regular enums is unchanged, since their members do form an exhaustive value set.

Checklist

  • Added tests for all changed behaviour: 3 new cases in check-enum.test; they fail without the fix
  • Full testcheck.py suite passes locally (8207 passed, 15 skipped, 7 xfailed); check-narrowing, check-expressions, check-literal, check-unreachable-code also pass
  • mypy --config-file mypy_self_check.ini passes on the touched modules
  • ruff check / ruff format --check (pinned ruff 0.14.3) pass on the touched modules

narrow_type_by_identity_equality expands enum instances into a union of
member literals and rules out the compared-away member in the negative
branch. For enum.Flag subclasses this is unsound: a Flag value can hold
any combination of members, so the declared members are not an exhaustive
value set. After `programs != Programs.P2`, programs can still be
`Programs.P1 | Programs.P2`, so narrowing away the P2 member rejects
`Programs.P2 in programs` -- valid at runtime -- against typeshed's
`Flag.__contains__(self, other: Self)` (pythongh-21937).

Skip equality/identity narrowing when either operand is a Flag value;
the sound type for such a value is the enum class itself. Regular enum
narrowing is unchanged, and existing Flag bitwise-operator behavior is
untouched.
@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incorrect narrowing of Enum.Flag

1 participant