Add DateTimeMetric, Analyzer and Example #568
Conversation
|
Hi @rdsharma26, @mentekid please help review this one |
There was a problem hiding this comment.
In general the changes look good and thank you for adding the example.
Could you add some more testing please?
Would like to see more tests for the analyzer behavior (in https://github.com/awslabs/deequ/tree/master/src/test/scala/com/amazon/deequ/analyzers) and tests to see how it works within the VerificationSuite (in https://github.com/awslabs/deequ/blob/master/src/test/scala/com/amazon/deequ/VerificationSuiteTest.scala)
| ) | ||
| } | ||
|
|
||
| "datetimeDistribution analyzer with VerificationSuite" in withSparkSessionJava8APIEnabled { sparkSession => |
There was a problem hiding this comment.
@eycho-am Hi just to confirm this is the kind of test you was looking for right?
Currently It is only implemented as an Analyzer to run with AnalysisRunner or as RequiredAnalyzer
For more advance use case with VerificationSuite, I can open a PR to implement some check based on it
Description of changes:
Add DateTimeMetric support. This chunk part of #299 to only DateTimeMetric support:
java.time.Instantisntead ofjava.sql.timestampBy submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.