You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Running ./mwaa-local-env package-requirements will always result in an exit code of 0, even on failure.
Steps to Reproduce:
Note: All steps were performed using v2.8.1, but the behavior should be the same for all versions that include this function (v2.2.2+).
Modify requirements.txt to force a failure. E.g. add apache-airflow-providers-datadog==3.5.0 to the requirements.txt included in the repo.
Run ./mwaa-local-env package-requirements and check exit code.
Click here for shell output
➜ aws-mwaa-local-runner git:(v2.8.1) ✗ ./mwaa-local-env package-requirements
Container amazon/mwaa-local:2_8 exists. Skipping build
Packaging requirements.txt into plugins
Collecting apache-airflow-providers-snowflake==5.2.1
Downloading apache_airflow_providers_snowflake-5.2.1-py3-none-any.whl (37 kB)
Collecting apache-airflow-providers-mysql==5.5.1
Downloading apache_airflow_providers_mysql-5.5.1-py3-none-any.whl (25 kB)
ERROR: Cannot install apache-airflow-providers-datadog==3.5.0 because these package versions have conflicting dependencies.
The conflict is caused by:
The user requested apache-airflow-providers-datadog==3.5.0
The user requested (constraint) apache-airflow-providers-datadog==3.5.1
To fix this you could try to:
1. loosen the range of package versions you've specified4. remove package versions to allow pip attempt to solve the dependency conflictERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts[notice] A new release of pip available: 22.3.1 -> 24.2[notice] To update, run: pip install --upgrade pipupdating: README.md (deflated 37%) ➜ aws-mwaa-local-runner git:(v2.8.1) ✗ echo $?0
Note that ./mwaa-local-env test-requirements does have a proper non-zero exit code on failure. However both commands should have a non-zero exit code on failure.
Click here for shell output
➜ aws-mwaa-local-runner git:(v2.8.1) ✗ ./mwaa-local-env test-requirements
Container amazon/mwaa-local:2_8 exists. Skipping build
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.8.1/constraints-3.11.txt"
Installing requirements.txt
Collecting apache-airflow-providers-snowflake==5.2.1
Downloading apache_airflow_providers_snowflake-5.2.1-py3-none-any.whl (37 kB)
Collecting apache-airflow-providers-mysql==5.5.1
Downloading apache_airflow_providers_mysql-5.5.1-py3-none-any.whl (25 kB)
ERROR: Cannot install apache-airflow-providers-datadog==3.5.0 because these package versions have conflicting dependencies.
The conflict is caused by:
The user requested apache-airflow-providers-datadog==3.5.0
The user requested (constraint) apache-airflow-providers-datadog==3.5.1
To fix this you could try to:
1. loosen the range of package versions you've specified2. remove package versions to allow pip attempt to solve the dependency conflictERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts[notice] A new release of pip available: 22.3.1 -> 24.2[notice] To update, run: pip install --upgrade pip ➜ aws-mwaa-local-runner git:(v2.8.1) ✗ echo $? 1
Cause:
The issue stems from the package_requirements() function in entrypoint.sh. Specifically, there is no check that pip3 download -r ... completed successfully. The script will just attempt to zip whatever was downloaded and continue on it's way.
Fix:
I'm working on a fix for this and will open a PR to v2.9.2 when it is ready. I kindly request the fix be merged to lower branches (either all v2.2.2+ or at least to the branches for versions that are still supported).
Thank you!
The text was updated successfully, but these errors were encountered:
On a related note - my fix will only be for pip3 download failures. It may be worth considering using set -e or similar for this and similar commands, but that is presumably a larger, internal conversation.
Running
./mwaa-local-env package-requirements
will always result in an exit code of0
, even on failure.Steps to Reproduce:
Note: All steps were performed using v2.8.1, but the behavior should be the same for all versions that include this function (v2.2.2+).
Modify
requirements.txt
to force a failure. E.g. addapache-airflow-providers-datadog==3.5.0
to therequirements.txt
included in the repo.Run
./mwaa-local-env package-requirements
and check exit code.Click here for shell output
Note that
./mwaa-local-env test-requirements
does have a proper non-zero exit code on failure. However both commands should have a non-zero exit code on failure.Click here for shell output
Cause:
The issue stems from the
package_requirements()
function inentrypoint.sh
. Specifically, there is no check thatpip3 download -r ...
completed successfully. The script will just attempt to zip whatever was downloaded and continue on it's way.Fix:
I'm working on a fix for this and will open a PR to v2.9.2 when it is ready. I kindly request the fix be merged to lower branches (either all v2.2.2+ or at least to the branches for versions that are still supported).
Thank you!
The text was updated successfully, but these errors were encountered: