Versatile Data Kit 0.5

[antoniivanov]

Summary

Major features include:

New managed db_connection_execute_operation hook

The hooks enable users to add behavior to existing SQL queries without modifying the code itself.
It is invoked for reach query before and after enabling to track its full execution. For example

@hookimpl(hookwrapper=True)
db_connection_execute_operation(execution_cursor: ExecutionCursor) -> Optional[int]: 
                start = time.time()
                outcome = yield # we yield the execution so that query is executed 
                end = time.time()
                log.info(f" duration: {end - start}. ")

Airflow Provider VDK release (beta)

Users can integrate with Apache Airflow to orchestrate in a DAG (workflow) Data Jobs.
Check out more at airflow-provider-vdk

What's Changed

• airflow-provider-vdk: Adopt auth plugin by @doks5 in [vdk-plugins] airflow-provider-vdk: Adopt auth plugin #856
• airflow-provider-vdk: Example DAG by @gageorgiev in airflow-provider-vdk: Example DAG #847
• airflow-provider-vdk: Fix VDKSensor templating issue, improve example DAG by @gageorgiev in airflow-provider-vdk: Fix VDKSensor templating issue, improve example DAG #852
• control-service: clear execution fail alert when failing with user error by @mrMoZ1 in control-service: clear execution fail alert when failing with user error  #850
• control-service: fix graphql team filter not retrieving special chars by @mrMoZ1 in control-service: fix graphql team filter not retrieveing special chars #863
• control-service: improve api message on oom job execution errors by @mrMoZ1 in control-service: improve api message on oom job execution errors #861
• documentation improvements by @zverulacis in Person/zverulacis/vdk documentation #853
• vdk-control-cli: Adopt new auth exceptions by @doks5 in vdk-control-cli: Adopt new auth exceptions #846
• vdk-core: Add unit test for destination_table in empty queue by @doks5 in vdk-core: Add unit test for destination_table in empty queue #865
• vdk-core: Fix destination_table referenced early by @doks5 in vdk-core: Fix destination_table referenced early #864
• vdk-core: Split execution summary into chunks by @doks5 in vdk-core: Split execution summary into chunks #867
• vdk-core: add new managed db_connection_execute_operation hook by @tozka in vdk-core: add new managed db_connection_execute_operation hook #805
• vdk-core: fix buggy (false positive) connection unit test by @tozka in vdk-core: fix buggy (false positive) connection unit test #841
• vdk-control-api-auth: New VDK Auth exceptions by @doks5 in New VDK Auth exceptions #845
• vdk-heartbeat: pipelines-control-service-integration-tests image rebuild by @ivakoleva in vdk-heartbeat: pipelines-control-service-integration-tests image rebuild #848
• vdk-plugins: Add Managed Database Connection cycle plugin by @tozka in vdk-plugins: Add Managed Database Connection cycle plugin #859
• vdk-test-utils: enable back tests by @tozka in vdk-test-utils: enable back tests #855

New Contributors

• @zverulacis made their first contribution in Add How To section to the readme #842

Full Changelog: v0.4...v0.5

---

This discussion was created from the release Versatile Data Kit 0.5.