-
Notifications
You must be signed in to change notification settings - Fork 538
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add DateTimeMetric, Analyzer and Example #568
base: master
Are you sure you want to change the base?
Conversation
Hi @rdsharma26, @mentekid please help review this one |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In general the changes look good and thank you for adding the example.
Could you add some more testing please?
Would like to see more tests for the analyzer behavior (in https://github.com/awslabs/deequ/tree/master/src/test/scala/com/amazon/deequ/analyzers) and tests to see how it works within the VerificationSuite (in https://github.com/awslabs/deequ/blob/master/src/test/scala/com/amazon/deequ/VerificationSuiteTest.scala)
) | ||
} | ||
|
||
"datetimeDistribution analyzer with VerificationSuite" in withSparkSessionJava8APIEnabled { sparkSession => |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@eycho-am Hi just to confirm this is the kind of test you was looking for right?
Currently It is only implemented as an Analyzer to run with AnalysisRunner or as RequiredAnalyzer
For more advance use case with VerificationSuite
, I can open a PR to implement some check based on it
Description of changes:
Add DateTimeMetric support. This chunk part of #299 to only DateTimeMetric support:
java.time.Instant
isntead ofjava.sql.timestamp
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.