[Proposal] Separate DAG discovery path from DAG bundle import root

[samraj2k]

A DAG bundle can contain DAG files together with shared Python modules, configuration files, and other resources.
For example:

repository/
├── project/
│   ├── lib/
│   └── dags/
│       └── example_dag.py
└── README.md

The DAG may import shared code using:

from project.lib import some_function

In this structure:

Airflow should scan only repository/project/dags for DAG files.
Python imports need repository on sys.path.

Current behavior:

BaseDagBundle.path is currently used for both:

• DAG discovery and relative file paths.
• The directory added to sys.path by BundleDagBag.

For example, configuring GitDagBundle with:

subdir="project/dags"

limits DAG discovery to that directory. However, only this subdirectory is added to sys.path, so the following import fails:

from project_package.shared import some_function

Proposed solution

Add an import_root property to BaseDagBundle:

@property
def import_root(self) -> Path:
    """Directory added to the Python import path when loading bundle code."""
    return self.path

The default keeps the existing behaviour.

A bundle with separate discovery and import roots could implement:

@property
def path(self) -> Path:
    return self.repository_path / self.subdir

@property
def import_root(self) -> Path:
    return self.repository_path

Similar discussion

• [Question] Bundle-specific Python Path Isolation in Airflow 3.x Git Bundles #61901

Let me know if this is useful, and the solution is acceptable.
I am willing to contribute to this change if accepted, thanks

[hamedrabah]

The safest current solution is to make shared code an installed Python package. Build project into the Airflow image or install it in the environment, then keep the bundle's subdir focused on DAG discovery. Imports no longer depend on the DAG directory being placed on sys.path.

A second option is to make the repository root the bundle path and use .airflowignore to exclude non-DAG directories from discovery while leaving them importable. That works, but an installed package gives clearer versioning and avoids module-name collisions between bundles.

A separate import_root is technically reasonable, but it expands the executable import boundary beyond the directory Airflow scans and versions as DAG content. The design would need to define serialization, bundle refresh behavior, duplicate package names across bundles, and whether workers receive the same root.

So I would use package installation as the supported deployment pattern today. If this becomes an API proposal, keeping import_root = path by default is compatible, but the security and distribution semantics need to be part of the contract rather than treating it as only a sys.path adjustment.

[samraj2k]

Thanks @hamedrabah for the inputs!

safest current solution is to make shared code an installed Python package

Agree on this, but at times this is not a feasible solution. For a existing repository with a specific DAGs directory, the imports can be anchored from top level. Infact this is quite common for large monorepo. The assumptions currently is that the import root is same as where DAGs need to be processed, and from this discussion I want inputs if this is even a valid assumption.

but an installed package gives clearer versioning and avoids module-name collisions between bundles.

Since the file is parsed in the subprocess, there should be no chance of collision imo. The import_path (as per my approach) would be where sys.path will be passed in the subprocess.

but the security and distribution semantics need to be part of the contract

Definitely I can add that, but since this lies on deployment manager to set the correct import path, securty wise too I think it should be fine

Let me know if I am missing something, I can also start a discuss thread of devlist!

[hamedrabah]

Thanks @samraj2k — I agree that large monorepos are the strongest case for relaxing the current assumption. My remaining concern is scope, not whether the parser subprocess itself is isolated: bundle code is also reloaded for task execution and callbacks, so the import root needs to travel with the same versioned bundle checkout in every execution context.

I would support the proposal with four constraints: keep import_root = path as the default; resolve it within the same initialized and versioned checkout rather than an arbitrary host path; propagate it consistently to parsing, task execution, and callbacks; and add tests for duplicate top-level module names plus cross-bundle leakage. With those semantics explicit, a dev-list thread sounds worthwhile.

[samraj2k]

Thanks @hamedrabah for the valuable suggestion.

keep import_root = path as the default

Yes, would agree on this too

same initialized and versioned checkout

Regarding this, just like kwargs etc which are not versioned, I would for simplicity not introduce versioning. import_path change across a bundle require even a bigger change, and the other way can be re-defining a new bundle. This is my opinion. Let me know regarding your concerns.

Also, would start a discuss thread too, thanks