You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The problem:
Currently, our examples are heavily dependent on dag_run.start_date as the metadata used for _ingestion_ts
And this add an issue that we can't have a partial reprocessing as the dag_run.start_date will be the start date of the reprocessing run and the framework wont be able to correctly find the parquet files on the currect directory _logical_ts={{ execution_date }}/_ingestion_ts={{ dag_run.start_date }}
The text was updated successfully, but these errors were encountered:
The problem:
Currently, our examples are heavily dependent on
dag_run.start_date
as the metadata used for_ingestion_ts
And this add an issue that we can't have a partial reprocessing as the
dag_run.start_date
will be the start date of the reprocessing run and the framework wont be able to correctly find the parquet files on the currect directory_logical_ts={{ execution_date }}/_ingestion_ts={{ dag_run.start_date }}
The text was updated successfully, but these errors were encountered: