Stop pipeline when ExampleValidator finds anomalies #182
casassg
started this conversation in
Project Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
In cases where any anomalies that are found by ExampleValidator should stop the pipeline execution, the standard components don't have a way of stopping it. That's because none of the standard components depend on the results from ExampleValidator.
We can write a custom component to take the ExampleAnomalies artifact from ExampleValidator, along with the Examples artifact from ExampleGen, and parse the ExampleAnomalies artifact.
If no anomalies are found, the component passes through the Examples artifact from ExampleGen
If any anomalies are found, the component will fail
Downstream components which depend on the Examples artifact from ExampleGen can instead depend on the Examples artifact from this new custom component. When it fails, the pipeline will stop.
An enhancement of this would be to either include a user code module to check the type of anomaly and decide whether to fail or not, or some sort of configuration parameter.
Beta Was this translation helpful? Give feedback.
All reactions