Skip to content
This repository has been archived by the owner on Apr 11, 2024. It is now read-only.

Add append and replace load strategies with example DAG for Table #45

Merged
merged 15 commits into from
Apr 5, 2023

Conversation

sunank200
Copy link
Collaborator

@sunank200 sunank200 commented Mar 29, 2023

Description

What is the current behavior?

Although the logic for the replace and append is implemented this is not working because if_exists is not propagated from TransferParams here: https://github.com/astronomer/apache-airflow-provider-transfers/blob/main/src/universal_transfer_operator/data_providers/database/base.py#L202

closes: #47

What is the new behavior?

  • Add append and replace load strategies with example DAG for Table
  • Add example DAGs
  • Add docs
  • Test is already there to test the behaviour

Does this introduce a breaking change?

No

Checklist

  • Created tests which fail without the change (if possible)
  • Extended the README / documentation, if necessary

@sunank200
Copy link
Collaborator Author

Screenshot 2023-03-30 at 12 44 19 AM

@utkarsharma2
Copy link
Collaborator

Can we add test cases to validate the behavior?

  1. Append to table
  2. Replace to table

docs/universal_transfer_operator/operator.rst Outdated Show resolved Hide resolved
docs/universal_transfer_operator/operator.rst Outdated Show resolved Hide resolved
docs/universal_transfer_operator/operator.rst Outdated Show resolved Hide resolved
@sunank200
Copy link
Collaborator Author

Add tests

Added the tests: 9d9f5e8

@sunank200 sunank200 requested a review from phanikumv March 30, 2023 10:43
@sunank200 sunank200 requested a review from utkarsharma2 March 30, 2023 15:44
@sunank200 sunank200 force-pushed the append-and-replace branch from fea7f59 to 57f3527 Compare April 3, 2023 08:32
@sunank200 sunank200 merged commit 8a32809 into main Apr 5, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

append load strategies with schema change
3 participants