Help Needed: Running Airflow DAG with Dependencies in multiple Conda Environments #42643
Replies: 3 comments 3 replies
-
|
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval. |
Beta Was this translation helpful? Give feedback.
-
|
All possible ways of handling conflicting dependencies are described in https://airflow.apache.org/docs/apache-airflow/stable/best-practices.html#handling-conflicting-complex-python-dependencies |
Beta Was this translation helpful? Give feedback.
-
|
Thank You |
Beta Was this translation helpful? Give feedback.
-
|
And please do not open "issue" for such a general broad topic and request to help. In the future open discussions for such questions. |
Beta Was this translation helpful? Give feedback.
-
|
As a first time user i was so confused |
Beta Was this translation helpful? Give feedback.
-
|
That's why I explained it. |
Beta Was this translation helpful? Give feedback.
-
Apache Airflow version
2.10.2
If "Other Airflow 2 version" selected, which one?
No response
What happened?
Hi, I’m seeking assistance with my Airflow setup. My Airflow instance runs in an environment named
prod_env, but the modules required for a particular DAG only work in a separate Conda environment named (ind_env).I require guidance on managing dependencies across different environments to work within Airflow
What you think should happen instead?
No response
How to reproduce
install airflow in one environment and create some dags,and create another environment and there you have some python modules related to a particular dag ;you have to call them in ur current environment.Remember the python packages of ur cuurent environment not directly support those packages currently setup in another environment.
Operating System
Linux
Versions of Apache Airflow Providers
apache-airflow==2.10.2
apache-airflow-providers-amazon==8.28.0
apache-airflow-providers-celery==3.8.1
apache-airflow-providers-common-compat==1.2.0
apache-airflow-providers-common-io==1.4.0
apache-airflow-providers-common-sql==1.14.2
apache-airflow-providers-elasticsearch==5.5.0
apache-airflow-providers-fab==1.2.2
apache-airflow-providers-ftp==3.11.0
apache-airflow-providers-google==10.21.1
apache-airflow-providers-http==4.12.0
apache-airflow-providers-imap==3.6.1
apache-airflow-providers-smtp==1.7.1
apache-airflow-providers-sqlite==3.8.1
Deployment
Virtualenv installation
Deployment details
No response
Anything else?
No response
Are you willing to submit PR?
Code of Conduct
Beta Was this translation helpful? Give feedback.
All reactions