Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The transform of lineage result is null when parsing UDAF #125

Open
jeff-zou opened this issue Apr 19, 2024 · 0 comments
Open

The transform of lineage result is null when parsing UDAF #125

jeff-zou opened this issue Apr 19, 2024 · 0 comments

Comments

@jeff-zou
Copy link

jeff-zou commented Apr 19, 2024

public class TestAggregateFunction extends AggregateFunction<String, TestAggregateFunction.TestAggregateAcc> {

    public void accumulate(TestAggregateAcc acc, String param1, String param2, String param3) {
        acc.test = param1 + param2 + param3;
    }

    @Override
    public String getValue(TestAggregateAcc accumulator) {
        return accumulator.test;
    }

    @Override
    public TestAggregateAcc createAccumulator() {
        return new TestAggregateAcc();
    }

    public static class TestAggregateAcc {
        public String test;
    }
}

create function test_aggregate as 'com.hw.lineage.flink.aggregatefunction.TestAggregateFunction'

INSERT INTO dwd_hudi_users SELECT id, name, test_aggregate(concat_ws('_', name, 'test'), name, 'test'), birthday, ts, DATE_FORMAT(birthday, 'yyyyMMdd') FROM ods_mysql_users group by id, name, birthday, ts

The transform of lineage result is null when parsing UDAF.
eq:
LineageResult(sourceCatalog=hive, sourceDatabase=default, sourceTable=ods_mysql_users, sourceColumn=name, targetCatalog=hive, targetDatabase=default, targetTable=dwd_hudi_users, targetColumn=company_name, transform=null)

jeff-zou added a commit to jeff-zou/flink-sql-lineage that referenced this issue Apr 19, 2024
HamaWhiteGG added a commit that referenced this issue Apr 21, 2024
#125 #126 support parse UDTF which the number of input and argument is equal
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant