Skip to content

Fix dead config: scheduler.ignore_first_depends_on_past_by_default#68180

Open
kyungjunleeme wants to merge 4 commits into
apache:mainfrom
kyungjunleeme:fix/ignore-first-depends-on-past-dead-config
Open

Fix dead config: scheduler.ignore_first_depends_on_past_by_default#68180
kyungjunleeme wants to merge 4 commits into
apache:mainfrom
kyungjunleeme:fix/ignore-first-depends-on-past-dead-config

Conversation

@kyungjunleeme

@kyungjunleeme kyungjunleeme commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

[scheduler] ignore_first_depends_on_past_by_default became a dead config in Airflow 3: it is still declared in config.yml with default "True", but no code reads it. The Task SDK hardcoded the default:

DEFAULT_IGNORE_FIRST_DEPENDS_ON_PAST: bool = False

so the regression that #22491 fixed in 2.3.0 came back. Adding a new task to an existing DAG whose default_args set depends_on_past=True leaves the new task in no-status forever (PrevDagrunDep: "previous task instance has not run yet") and the DAG run never leaves the running state. A grep for ignore_first_depends_on_past_by_default across main matches only the config.yml declaration — zero readers. (2.10.5 read it at airflow/models/abstractoperator.py.)

Fix

Wire the default back to the config, the same way 2.10.5 did and the way DEFAULT_RETRIES and friends in the same module still do:

DEFAULT_IGNORE_FIRST_DEPENDS_ON_PAST: bool = conf.getboolean(
    "scheduler", "ignore_first_depends_on_past_by_default"
)

The value flows through OPERATOR_DEFAULTS into the serialization client_defaults, so both regular and mapped operators pick up the configured default on the scheduler side. The server-side schema default stays False, so setting the config to False continues to disable the behavior (no client_defaults entry is emitted in that case).

Tests

  • task-sdk: default is True, explicit override wins, and the default follows the config (reload guard against the dead-config regression).
  • airflow-core PrevDagrunDep: a new task with depends_on_past=True passes its first run using only the config default.
  • Updated expectations that pinned the old hardcoded default: test_dag_serialization (operator defaults and the serialized-DAG client_defaults ground truth, which now carries ignore_first_depends_on_past=True) and test_mappedoperator.
  • OpenLineage: the task facet now reports ignore_first_depends_on_past=True on Airflow 3.2+. test_task_info_af3 derives the expected value from the operator so it stays correct across core versions (the Compat 3.0.x provider jobs run it against older cores where the default is still False); the AF3 expected_events system fixtures are updated to match.

related: #17585, #22491


Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 4.8)

Generated-by: Claude Code (Opus 4.8) following the guidelines

…ignored

The `[scheduler] ignore_first_depends_on_past_by_default` option (added in
2.3.0 via apache#22491) became a dead config in Airflow 3: it is still declared in
config.yml with default "True", but no code reads it. The Task SDK hardcoded
`DEFAULT_IGNORE_FIRST_DEPENDS_ON_PAST = False`, so the regression that apache#22491
fixed came back — adding a new task to an existing DAG whose default_args set
`depends_on_past=True` leaves the task in no-status forever (PrevDagrunDep:
"previous task instance has not run yet") and the DAG run never completes.

Wire the default back to the config, matching how 2.10.5 read it (and how
DEFAULT_RETRIES and friends in the same module still read conf). The value
flows through OPERATOR_DEFAULTS into the serialization client_defaults, so
both regular and mapped operators pick up the configured default on the
scheduler side.
The OpenLineage task facet now reports ignore_first_depends_on_past=True on
Airflow 3 (matching the AF2 expectations), since the scheduler config default
is honored again.
- Add the new ignore_first_depends_on_past=True entry to the serialized DAG
  client_defaults ground truth (it now differs from the schema default).
- Derive the expected ignore_first_depends_on_past in the OpenLineage
  test_task_info_af3 from the operator instead of hardcoding, so it passes
  against both older 3.x cores (default False) and 3.2+ (default True).
@kyungjunleeme kyungjunleeme force-pushed the fix/ignore-first-depends-on-past-dead-config branch from a94f2c1 to 7bc1f96 Compare June 7, 2026 18:20
@potiuk potiuk added the ready for maintainer review Set after triaging when all criteria pass. label Jun 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants