cleanup-logs.py - 'dict object' has no attribute 'maxLogAgeInDays'

[HansBambel]

When running with a scheduler this part of the code results in failed tasks for me with the following error:

MAX_LOG_AGE_IN_DAYS="{{dag_run.conf.maxLogAgeInDays}}"
if [ "${MAX_LOG_AGE_IN_DAYS}" == "" ]; then
    echo "maxLogAgeInDays conf variable isn't included. Using Default '""" + str(DEFAULT_MAX_LOG_AGE_IN_DAYS) + """'."
    MAX_LOG_AGE_IN_DAYS='""" + str(DEFAULT_MAX_LOG_AGE_IN_DAYS) + """'
fi

Error message:
'dict object' has no attribute 'maxLogAgeInDays'
It seems that the dag does not find the entry maxLogAgeInDays.

I wanted to ask where this value should be set? Following the readme, I only created two variables called airflow_log_cleanup__max_log_age_in_days and airflow_log_cleanup__enable_delete_child_log which are getting called for setting the variables DEFAULT_MAX_LOG_AGE_IN_DAYS and ENABLE_DELETE_CHILD_LOG.

I managed to get it to work by substituting the whole code above by:

MAX_LOG_AGE_IN_DAYS=""" + str(DEFAULT_MAX_LOG_AGE_IN_DAYS) + """

Since the DEFAULT_MAX_LOG_AGE_IN_DAYS already reads out the variable set in the UI, I do not need dag_run.conf.maxLogAgeInDays, right?

In general, I would suggest to rename DEFAULT_MAX_LOG_AGE_IN_DAYS to MAX_LOG_AGE_IN_DAYS, because the variable is not the default, but the 30 in the .get() is:

DEFAULT_MAX_LOG_AGE_IN_DAYS = Variable.get(
    "airflow_log_cleanup__max_log_age_in_days", 30
)

One more thing:
Following best practices it would be better to combine the two variables to a single Variable in the JSON format to reduce DB requests:

airflow_log_cleanup = Variable.get("airflow_log_cleanup", deserialize_json=True)
DEFAULT_MAX_LOG_AGE_IN_DAYS = airflow_log_cleanup.get("max_log_age_in_days", 30)
ENABLE_DELETE_CHILD_LOG = airflow_log_cleanup.get("enable_delete_child_log", "False")

I am using AirFlow 2.0.1

[zjaxxon]

Got the same issue after upgrading to 2.1+. Seems this project isn't under active development but thanks HansBambel for the solution.

[arturbrandysonwelo]

In AirFlow 2.2.5 the solution of HansBambel also works for me. Thank you.