How to capture bad records while using from_avro in abris

[Tarannump]

I have use case to capture the bad records and store it in a separate location for future reference. Is it possible to get the records which are not deserialised instead of dropping them?

[cerveada]

You can use exception handlers introduced in this PR: #290

You will have to implement your own handler that will store the records though.

[Tarannump]

Thanks @cerveada for your response. We are using Glue Streaming ETL job which contains spark 3.1.1. Based on the PR looks like exception handlers are added in abris 6.3.0 which has spark 3.2.1. Is it possible to get the same feature in abris 5 as well?

[kevinwallimann]

Hi @Tarannump Unfortunately we don't have the capacity to backport the feature to ABRiS 5. Of course, you are welcome to fork the repo or submit a pull request.