How do disable the DAG version ?

[trongchata]

In our case, we have airflow running in two different data centres, and dag processors are running in two airflow instances
The DAG version increase rapidly, seems every time when the dag processor load it, will increase one version.

Our DAG are saved in db, and generate dynamtically.

Is there any possible to disable the DAG version ?

Airflow version 3.2.1

Everything works good in our lower environment, they are single cluster. The DAG version increase rapidly only happen in PROD, two data centres.

Thank you

[pedro-cf]

https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#disable-bundle-versioning

[mateuscarestiato]

Hi @trongchata! The rapid version increase in your multi-datacenter setup is almost certainly caused by two DAG processors in different instances processing the same DAGs and producing slightly different serializations — leading to each instance detecting "changes" made by the other.

Root cause in multi-datacenter setups:

When DAG processors run in two separate Airflow instances sharing the same metadata database, each processor independently serializes and stores DAG versions. If there's any non-determinism in serialization (timezone handling, float precision, object ordering), each instance will see the other's write as a "change" and create a new version — causing a rapid version increment loop.

Options to address this:

1. Designate a single DAG processor (recommended for 3.x): In Airflow 3.x, DAG processing is handled by the dag-processor component. Run it in only one datacenter and have the other datacenter's scheduler consume DAG metadata from the shared DB:

yaml

In the second DC's airflow.cfg — disable local DAG processing

[dag_processor]
standalone = false
2. Increase the DAG file processing interval to reduce parse frequency:

ini

airflow.cfg

[scheduler]
dag_dir_list_interval = 300 # default is 300s, increase if lower
min_file_process_interval = 60 # seconds between re-processing same file
3. If using DB-stored DAGs (your case), ensure a single writer: Since your DAGs are saved in DB and generated dynamically, ensure only one instance writes DAG definitions to the database. The second datacenter's DAG processor should be configured as read-only or disabled.

4. Check for non-determinism in your dynamic DAG generation (see related discussion Dag Versioning keeps increasing #66103 for diagnostics).

Note: There is currently no config flag to fully disable DAG versioning in Airflow 3.x — it's a core feature of the new architecture. The correct solution is controlling which instances run the DAG processor.

Hope this helps! Are both instances using the same shared metadata database or separate DBs that are synced?

[trongchata]

Hi @mateuscarestiato
Thank you for your suggestion. We do have some different value in two cluster, need to inject into DAG when parse it.
We have two clusters in different data centers, that is for Hot/Hot for DR design.

Both these two airflow instance are using the same shared metadata database.

Now we have separate dag processor component from scheduler.

For your suggestion,

1. I think we can keep DC A as same as before.
2. For DC B, we need to disable dag processor, just let scheduler consume the dag from Airflow db.

But I still not sure how to disable the dag process. As I understand, I only need to change the follow configuration to be False, right ?

AIRFLOW__SCHEDULER__STANDALONE_DAG_PROCESSOR=False

Thank you