Model Reliability v.s. Model Robustness #11
Jingkang50
started this conversation in
General
Replies: 1 comment
-
Calibration is also one way to evaluate whether the model is reliable or not. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Some readers asked me about the difference of these two terms.
Here is my thought.
Model Reliability:
Reliability modeling is the process of predicting or understanding the reliability of a component or system prior to its implementation. [ref]
Model Robustness:
A model is robust when the accuracy does not change significantly from the baseline accuracy under various conditions. [ref]
So in my opinion, OOD detection is pursuing model reliability, as the models are required to estimate the probability whether the given sample is ID. OOD detection is all about confidence. A good example of model robustness is OOD generalization, which requires stable performance even with domain shifts. OOD generalization is all about accuracy.
In sum:
OOD detection -> Model Reliability
OOD generalization -> Model Robustness
Beta Was this translation helpful? Give feedback.
All reactions