Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bug] Timetables issue #287

Open
tatiana opened this issue Nov 15, 2024 · 0 comments
Open

[bug] Timetables issue #287

tatiana opened this issue Nov 15, 2024 · 0 comments
Labels
bug Something isn't working triage-needed

Comments

@tatiana
Copy link
Collaborator

tatiana commented Nov 15, 2024

This feedback was reported by Karen Braganza on DAG Factory 0.19.0:

Timetables that do not expect a params argument will result in the following import error.

Broken DAG: [/usr/local/airflow/dags/df.py]
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/dagfactory/dagbuilder.py", line 313, in make_timetable
    schedule: Timetable = timetable_obj(**timetable_params)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: plugins.timetables.workday.UnevenIntervalsTimetable() argument after ** must be a mapping, not NoneType

In traditional Airflow, you can create a custom timetable with or without params like below.

schedule=UnevenIntervalsTimetable()
OR
schedule=CronTriggerTimetable("0 18 * * 5", timezone="UTC")

In DAG factory, only the second type of timetable will work.

timetable:
	callable: plugins.timetables.workday.UnevenIntervalsTimetable

OR

timetable:
	callable: airflow.timetables.trigger.CronTriggerTimetable
	params:	
		cron: "0 18 * * 5"
		timetable: "UTC"

The first timetable will produce the TypeError I mentioned because if the params dictionary is not available, the make_timetable() function in dagbuilder.py will receive a None value as the argument for its timetable_params parameter (source code).

dag_kwargs["timetable"] = DagBuilder.make_timetable(
       timetable_args.get("callable"), timetable_args.get("params"))

Since it is expecting a dictionary, timetable_params is passed as **timetable_params to the timetable object within make_timetable(). If None is passed as **timetable_params instead of a dictionary, a TypeError is produced (source code).

schedule: Timetable = timetable_obj(**timetable_params)

My current workaround is to create a dummy argument in my custom timetable to pass in the params dictionary.
Users should be able to use custom timetables without params or a workaround.

@tatiana tatiana added bug Something isn't working triage-needed labels Nov 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage-needed
Projects
None yet
Development

No branches or pull requests

1 participant