Description
When a DAG is paused while it has an active (running) DagRun — e.g. one started via manual trigger before the pause, or a manual trigger on an already-paused DAG that got an initial burst of tasks dispatched — the scheduler stops scheduling further task instances for that run. The run doesn't fail and logs no error; it simply stalls indefinitely in running state with remaining tasks stuck in scheduled, queued_when=None, never sent to the executor.
This happens because is_paused is checked as a single gate that governs both "should new DagRuns be created" and "should ready task instances in existing DagRuns be scheduled." The two are conflated.
Use case/motivation
Pause is a reasonable way to say "don't start new runs of this DAG." It's not intuitive that it also freezes runs already in flight — especially since:
- No error, warning, or DAG/task state reflects why the run stopped progressing.
- A manually triggered run against a paused DAG appears to start successfully (initial tasks execute), masking the problem until the run has been silently stuck for several minutes.
- The eventual fix (unpause) has a side effect the operator may not want: it also releases scheduling for new runs of that DAG, when the operator's only goal was to unstick the one in-flight run.
Suggested change
Preserve pre-run pause semantics: is_paused should gate DagRun creation, not task-instance scheduling for DagRuns already in running state. A DagRun that starts (however it started) while unpaused, or that gets tasks dispatched before a pause takes effect, should be allowed to run to completion. If the DAG is still paused when the run finishes, subsequent runs are blocked as expected — pause state before and after the run is preserved; only the in-flight run is exempted.
At minimum, surfacing this in the UI/logs (e.g., a task/run banner: "this DagRun's DAG is paused; scheduling is suspended") would prevent the silent-stall confusion, even if the gating behavior itself is kept as-is by design.
Are you willing to submit a PR?
Code of Conduct
Description
When a DAG is paused while it has an active (running) DagRun — e.g. one started via manual trigger before the pause, or a manual trigger on an already-paused DAG that got an initial burst of tasks dispatched — the scheduler stops scheduling further task instances for that run. The run doesn't fail and logs no error; it simply stalls indefinitely in
runningstate with remaining tasks stuck inscheduled,queued_when=None, never sent to the executor.This happens because
is_pausedis checked as a single gate that governs both "should new DagRuns be created" and "should ready task instances in existing DagRuns be scheduled." The two are conflated.Use case/motivation
Pause is a reasonable way to say "don't start new runs of this DAG." It's not intuitive that it also freezes runs already in flight — especially since:
Suggested change
Preserve pre-run pause semantics:
is_pausedshould gate DagRun creation, not task-instance scheduling for DagRuns already inrunningstate. A DagRun that starts (however it started) while unpaused, or that gets tasks dispatched before a pause takes effect, should be allowed to run to completion. If the DAG is still paused when the run finishes, subsequent runs are blocked as expected — pause state before and after the run is preserved; only the in-flight run is exempted.At minimum, surfacing this in the UI/logs (e.g., a task/run banner: "this DagRun's DAG is paused; scheduling is suspended") would prevent the silent-stall confusion, even if the gating behavior itself is kept as-is by design.
Are you willing to submit a PR?
Code of Conduct