[BUG] Redundant fields required when using merge_builder in DeltaTableWriter

[zarembat]

When merging data into a Delta table using DeltaTableWriter with the merge_builder param one needs to provide the source DataFrame and the target table name as arguments to DeltaTableWriter, even though they are already passed to the merge builder. Otherwise Koheesio throws an error about missing required fields. So effectively we are passing the same values twice.

Describe the bug

Take this example of merging data into a Delta table using merge_builder:

spark = get_active_session()
merge_condition = (
    "target.COLUMN_A = source.COLUMN_A AND target.COLUMN_B = source.COLUMN_B"
)

DeltaTableWriter(
    df=source_df,
    table=target_table_name,
    output_mode=BatchOutputMode.MERGE,
    output_mode_params={
        "merge_builder": (
            DeltaTable.forName(sparkSession=spark, tableOrViewName=target_table_name)
            .alias("target")
            .merge(source=source_df.alias("source"), condition=merge_condition)
            .whenNotMatchedInsertAll()
        )
    },
).write()

Even though in the merge_builder param we already pass source_df and target_table_name, we still need to pass the same values as the df and table arguments respectively to DeltaTableWriter. Otherwise Pydantic will throw a Validation Error.

Steps to Reproduce

Use the above code snippet to merge data into a Delta table using merge builder to trigger the error.

Expected behavior

When using a merge builder with appropriate params, one should not be forced to additionally pass the df and table params to DeltaTableWriter.

Environment

Koheesio 0.9.0