Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docstring return statements #789

Merged
merged 8 commits into from
May 18, 2023
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .git-blame-ignore-revs
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
894744537dc64f729a44f64fbcb58e6b3b5dc1aa
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we're squashing commits, there should only be one entry per PR. Can you check if it's the case that the hash of the first commit is used after the squash?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was actually wondering about this and meant to check how it works, and it seems the above does not. It seems that we'll have to do two separate PRs one with the changes and one with the relevant commits in .git-blame-ignore-revs.

Copy link
Contributor

@jklaise jklaise May 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or squash locally before merging so the PR only has one commit? Edit: on second though, you'd need to know the commit hash before it's created so probably not...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well I could squash the PR commits to one and then add another commit before merging into master instead of squashing?

ad863fead5925eba5e61c915f9c799f9c5575ae4
eb24014ce713fa14193cd5982d58e20d9e51d349
4 changes: 2 additions & 2 deletions alibi_detect/ad/adversarialae.py
Original file line number Diff line number Diff line change
Expand Up @@ -291,8 +291,8 @@ def predict(self, X: np.ndarray, batch_size: int = int(1e10), return_instance_sc
Returns
-------
Dictionary containing 'meta' and 'data' dictionaries.
'meta' has the model's metadata.
'data' contains the adversarial predictions and instance level adversarial scores.
- 'meta' has the model's metadata.
- 'data' contains the adversarial predictions and instance level adversarial scores.
"""
adv_score = self.score(X, batch_size=batch_size)

Expand Down
4 changes: 2 additions & 2 deletions alibi_detect/ad/model_distillation.py
Original file line number Diff line number Diff line change
Expand Up @@ -208,8 +208,8 @@ def predict(self, X: np.ndarray, batch_size: int = int(1e10), return_instance_sc
Returns
-------
Dictionary containing 'meta' and 'data' dictionaries.
'meta' has the model's metadata.
'data' contains the adversarial predictions and instance level adversarial scores.
- 'meta' has the model's metadata.
- 'data' contains the adversarial predictions and instance level adversarial scores.
"""
score = self.score(X, batch_size=batch_size)

Expand Down
8 changes: 6 additions & 2 deletions alibi_detect/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ def concept_drift_dict():


class BaseDetector(ABC):
""" Base class for outlier, adversarial and drift detection algorithms. """
"""Base class for outlier, adversarial and drift detection algorithms."""

def __init__(self):
self.meta = copy.deepcopy(DEFAULT_META)
Expand Down Expand Up @@ -204,10 +204,12 @@ class Detector(Protocol):

Used for typing legacy save and load functionality in `alibi_detect.saving._tensorflow.saving.py`.

Note:
Note
----
This exists to distinguish between detectors with and without support for config saving and loading. Once all
detector support this then this protocol will be removed.
"""

meta: Dict

def predict(self) -> Any: ...
Expand All @@ -219,6 +221,7 @@ class ConfigurableDetector(Detector, Protocol):

Used for typing save and load functionality in `alibi_detect.saving.saving`.
"""

def get_config(self) -> dict: ...

@classmethod
Expand All @@ -233,6 +236,7 @@ class StatefulDetectorOnline(ConfigurableDetector, Protocol):

Used for typing save and load functionality in `alibi_detect.saving.saving`.
"""

t: int = 0

def save_state(self, filepath: Union[str, os.PathLike]): ...
Expand Down
51 changes: 31 additions & 20 deletions alibi_detect/cd/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -135,10 +135,12 @@ def __init__(
def preprocess(self, x: Union[np.ndarray, list]) -> Tuple[Union[np.ndarray, list], Union[np.ndarray, list]]:
"""
Data preprocessing before computing the drift scores.

Parameters
----------
x
Batch of instances.

Returns
-------
Preprocessed reference data and new instances.
Expand Down Expand Up @@ -174,7 +176,7 @@ def get_splits(

Returns
-------
Combined reference and test instances with labels and optionally a list with tuples of
Combined reference and test instances with labels and optionally a list with tuples of \
train and test indices for optionally different folds.
"""
# create dataset and labels
Expand Down Expand Up @@ -269,11 +271,11 @@ def predict(self, x: Union[np.ndarray, list], return_p_val: bool = True,
Returns
-------
Dictionary containing 'meta' and 'data' dictionaries.
'meta' has the model's metadata.
'data' contains the drift prediction and optionally the p-value, performance of the classifier
relative to its expectation under the no-change null, the out-of-fold classifier model
prediction probabilities on the reference and test data as well as the associated reference
and test instances of the out-of-fold predictions, and the trained model.
- 'meta' has the model's metadata.
- 'data' contains the drift prediction and optionally the p-value, performance of the classifier \
relative to its expectation under the no-change null, the out-of-fold classifier model \
prediction probabilities on the reference and test data as well as the associated reference \
and test instances of the out-of-fold predictions, and the trained model.
"""
# compute drift scores
p_val, dist, probs_ref, probs_test, x_ref_oof, x_test_oof = self.score(x)
Expand Down Expand Up @@ -394,10 +396,12 @@ def __init__(
def preprocess(self, x: Union[np.ndarray, list]) -> Tuple[Union[np.ndarray, list], Union[np.ndarray, list]]:
"""
Data preprocessing before computing the drift scores.

Parameters
----------
x
Batch of instances.

Returns
-------
Preprocessed reference data and new instances.
Expand All @@ -418,17 +422,18 @@ def get_splits(self, x_ref: Union[np.ndarray, list], x: Union[np.ndarray, list])
"""
Split reference and test data into two splits -- one of which to learn test locations
and parameters and one to use for tests.

Parameters
----------
x_ref
Data used as reference distribution.
x
Batch of instances.

Returns
-------
Tuple containing split train data and tuple containing split test data
Tuple containing split train data and tuple containing split test data.
"""

n_ref, n_cur = len(x_ref), len(x)
perm_ref, perm_cur = np.random.permutation(n_ref), np.random.permutation(n_cur)
idx_ref_tr, idx_ref_te = perm_ref[:int(n_ref * self.train_size)], perm_ref[int(n_ref * self.train_size):]
Expand Down Expand Up @@ -469,8 +474,8 @@ def predict(self, x: Union[np.ndarray, list], return_p_val: bool = True,
Returns
-------
Dictionary containing 'meta' and 'data' dictionaries.
'meta' has the detector's metadata.
'data' contains the drift prediction and optionally the p-value, threshold, MMD metric and
- 'meta' has the detector's metadata.
- 'data' contains the drift prediction and optionally the p-value, threshold, MMD metric and \
trained kernel.
"""
# compute drift scores
Expand Down Expand Up @@ -586,10 +591,12 @@ def __init__(
def preprocess(self, x: Union[np.ndarray, list]) -> Tuple[np.ndarray, np.ndarray]:
"""
Data preprocessing before computing the drift scores.

Parameters
----------
x
Batch of instances.

Returns
-------
Preprocessed reference data and new instances.
Expand Down Expand Up @@ -627,8 +634,8 @@ def predict(self, x: Union[np.ndarray, list], return_p_val: bool = True, return_
Returns
-------
Dictionary containing 'meta' and 'data' dictionaries.
'meta' has the model's metadata.
'data' contains the drift prediction and optionally the p-value, threshold and MMD metric.
- 'meta' has the model's metadata.
- 'data' contains the drift prediction and optionally the p-value, threshold and MMD metric.
"""
# compute drift scores
p_val, dist, distance_threshold = self.score(x)
Expand Down Expand Up @@ -748,10 +755,12 @@ def __init__(
def preprocess(self, x: Union[np.ndarray, list]) -> Tuple[np.ndarray, np.ndarray]:
"""
Data preprocessing before computing the drift scores.

Parameters
----------
x
Batch of instances.

Returns
-------
Preprocessed reference data and new instances.
Expand Down Expand Up @@ -787,8 +796,8 @@ def predict(self, x: Union[np.ndarray, list], return_p_val: bool = True, return_
Returns
-------
Dictionary containing 'meta' and 'data' dictionaries.
'meta' has the model's metadata.
'data' contains the drift prediction and optionally the p-value, threshold and LSDD metric.
- 'meta' has the model's metadata.
- 'data' contains the drift prediction and optionally the p-value, threshold and LSDD metric.
"""
# compute drift scores
p_val, dist, distance_threshold = self.score(x)
Expand Down Expand Up @@ -980,9 +989,9 @@ def predict(self, x: Union[np.ndarray, list], drift_type: str = 'batch',
Returns
-------
Dictionary containing 'meta' and 'data' dictionaries.
'meta' has the model's metadata.
'data' contains the drift prediction and optionally the feature level p-values,
threshold after multivariate correction if needed and test statistics.
- 'meta' has the model's metadata.
- 'data' contains the drift prediction and optionally the feature level p-values, threshold after \
multivariate correction if needed and test statistics.
"""
# compute drift scores
p_vals, dist = self.score(x)
Expand Down Expand Up @@ -1136,10 +1145,12 @@ def __init__(
def preprocess(self, x: Union[np.ndarray, list]) -> Tuple[np.ndarray, np.ndarray]:
"""
Data preprocessing before computing the drift scores.

Parameters
----------
x
Batch of instances.

Returns
-------
Preprocessed reference data and new instances.
Expand Down Expand Up @@ -1182,9 +1193,9 @@ def predict(self, # type: ignore[override]
Returns
-------
Dictionary containing 'meta' and 'data' dictionaries.
'meta' has the model's metadata.
'data' contains the drift prediction and optionally the p-value, threshold, conditional MMD test statistic
and coupling matrices.
- 'meta' has the model's metadata.
- 'data' contains the drift prediction and optionally the p-value, threshold, conditional MMD test \
statistic and coupling matrices.
"""
# compute drift scores
p_val, dist, distance_threshold, coupling = self.score(x, c)
Expand Down
8 changes: 4 additions & 4 deletions alibi_detect/cd/base_online.py
Original file line number Diff line number Diff line change
Expand Up @@ -185,8 +185,8 @@ def predict(self, x_t: Union[np.ndarray, Any], return_test_stat: bool = True,
Returns
-------
Dictionary containing 'meta' and 'data' dictionaries.
'meta' has the model's metadata.
'data' contains the drift prediction and optionally the test-statistic and threshold.
- 'meta' has the model's metadata.
- 'data' contains the drift prediction and optionally the test-statistic and threshold.
"""
# Compute test stat and check for drift
test_stat = self.score(x_t)
Expand Down Expand Up @@ -442,8 +442,8 @@ def predict(self, x_t: Union[np.ndarray, Any], return_test_stat: bool = True,
Returns
-------
Dictionary containing 'meta' and 'data' dictionaries.
'meta' has the model's metadata.
'data' contains the drift prediction and optionally the test-statistic and threshold.
- 'meta' has the model's metadata.
- 'data' contains the drift prediction and optionally the test-statistic and threshold.
"""
# Compute test stat and check for drift
test_stats = self.score(x_t)
Expand Down
6 changes: 2 additions & 4 deletions alibi_detect/cd/classifier.py
Original file line number Diff line number Diff line change
Expand Up @@ -207,10 +207,8 @@ def predict(self, x: Union[np.ndarray, list], return_p_val: bool = True,
Returns
-------
Dictionary containing 'meta' and 'data' dictionaries

- 'meta' - has the model's metadata.

- 'data' - contains the drift prediction and optionally the p-value, performance of the classifier \
- 'meta' - has the model's metadata.
- 'data' - contains the drift prediction and optionally the p-value, performance of the classifier \
relative to its expectation under the no-change null, the out-of-fold classifier model \
prediction probabilities on the reference and test data, and the trained model. \
"""
Expand Down
12 changes: 6 additions & 6 deletions alibi_detect/cd/context_aware.py
Original file line number Diff line number Diff line change
Expand Up @@ -142,9 +142,9 @@ def predict(self, x: Union[np.ndarray, list], c: np.ndarray,
Returns
-------
Dictionary containing 'meta' and 'data' dictionaries.
'meta' has the model's metadata.
'data' contains the drift prediction and optionally the p-value, threshold, conditional MMD test statistic
and coupling matrices.
- 'meta' has the model's metadata.
- 'data' contains the drift prediction and optionally the p-value, threshold, conditional MMD test \
statistic and coupling matrices.
"""
return self._detector.predict(x, c, return_p_val, return_distance, return_coupling)

Expand All @@ -162,8 +162,8 @@ def score(self, x: Union[np.ndarray, list], c: np.ndarray) -> Tuple[float, float

Returns
-------
p-value obtained from the conditional permutation test, the conditional MMD test statistic, the test
statistic threshold above which drift is flagged, and a tuple containing the coupling matrices
(W_{ref,ref}, W_{test,test}, W_{ref,test}).
p-value obtained from the conditional permutation test, the conditional MMD test statistic, the test \
statistic threshold above which drift is flagged, and a tuple containing the coupling matrices \
:math:`(W_{ref,ref}, W_{test,test}, W_{ref,test})`.
"""
return self._detector.score(x, c)
2 changes: 1 addition & 1 deletion alibi_detect/cd/keops/learned_kernel.py
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,7 @@ def score(self, x: Union[np.ndarray, list]) -> Tuple[float, float, float]:

Returns
-------
p-value obtained from the permutation test, the MMD^2 between the reference and test set,
p-value obtained from the permutation test, the MMD^2 between the reference and test set, \
and the MMD^2 threshold above which drift is flagged.
"""
x_ref, x_cur = self.preprocess(x)
Expand Down
2 changes: 1 addition & 1 deletion alibi_detect/cd/keops/mmd.py
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ def score(self, x: Union[np.ndarray, list]) -> Tuple[float, float, float]:

Returns
-------
p-value obtained from the permutation test, the MMD^2 between the reference and test set,
p-value obtained from the permutation test, the MMD^2 between the reference and test set, \
and the MMD^2 threshold above which drift is flagged.
"""
x_ref, x = self.preprocess(x)
Expand Down
4 changes: 2 additions & 2 deletions alibi_detect/cd/learned_kernel.py
Original file line number Diff line number Diff line change
Expand Up @@ -189,8 +189,8 @@ def predict(self, x: Union[np.ndarray, list], return_p_val: bool = True,
Returns
-------
Dictionary containing 'meta' and 'data' dictionaries.
'meta' has the detector's metadata.
'data' contains the drift prediction and optionally the p-value, threshold, MMD metric and
- 'meta' has the detector's metadata.
- data' contains the drift prediction and optionally the p-value, threshold, MMD metric and \
trained kernel.
"""
return self._detector.predict(x, return_p_val, return_distance, return_kernel)
6 changes: 3 additions & 3 deletions alibi_detect/cd/lsdd.py
Original file line number Diff line number Diff line change
Expand Up @@ -116,8 +116,8 @@ def predict(self, x: Union[np.ndarray, list], return_p_val: bool = True, return_
Returns
-------
Dictionary containing 'meta' and 'data' dictionaries.
'meta' has the model's metadata.
'data' contains the drift prediction and optionally the p-value, threshold and LSDD metric.
- 'meta' has the model's metadata.
- 'data' contains the drift prediction and optionally the p-value, threshold and LSDD metric.
"""
return self._detector.predict(x, return_p_val, return_distance)

Expand All @@ -133,7 +133,7 @@ def score(self, x: Union[np.ndarray, list]) -> Tuple[float, float, float]:

Returns
-------
p-value obtained from the permutation test, the LSDD between the reference and test set,
p-value obtained from the permutation test, the LSDD between the reference and test set, \
and the LSDD threshold above which drift is flagged.
"""
return self._detector.score(x)
Expand Down
4 changes: 2 additions & 2 deletions alibi_detect/cd/lsdd_online.py
Original file line number Diff line number Diff line change
Expand Up @@ -134,8 +134,8 @@ def predict(self, x_t: Union[np.ndarray, Any], return_test_stat: bool = True) \
Returns
-------
Dictionary containing 'meta' and 'data' dictionaries.
'meta' has the model's metadata.
'data' contains the drift prediction and optionally the test-statistic and threshold.
- 'meta' has the model's metadata.
- 'data' contains the drift prediction and optionally the test-statistic and threshold.
"""
return self._detector.predict(x_t, return_test_stat)

Expand Down
6 changes: 3 additions & 3 deletions alibi_detect/cd/mmd.py
Original file line number Diff line number Diff line change
Expand Up @@ -136,8 +136,8 @@ def predict(self, x: Union[np.ndarray, list], return_p_val: bool = True, return_
Returns
-------
Dictionary containing 'meta' and 'data' dictionaries.
'meta' has the model's metadata.
'data' contains the drift prediction and optionally the p-value, threshold and MMD metric.
- 'meta' has the model's metadata.
- 'data' contains the drift prediction and optionally the p-value, threshold and MMD metric.
"""
return self._detector.predict(x, return_p_val, return_distance)

Expand All @@ -153,7 +153,7 @@ def score(self, x: Union[np.ndarray, list]) -> Tuple[float, float, float]:

Returns
-------
p-value obtained from the permutation test, the MMD^2 between the reference and test set,
p-value obtained from the permutation test, the MMD^2 between the reference and test set, \
and the MMD^2 threshold above which drift is flagged.
"""
return self._detector.score(x)
4 changes: 2 additions & 2 deletions alibi_detect/cd/mmd_online.py
Original file line number Diff line number Diff line change
Expand Up @@ -134,8 +134,8 @@ def predict(self, x_t: Union[np.ndarray, Any], return_test_stat: bool = True) \
Returns
-------
Dictionary containing 'meta' and 'data' dictionaries.
'meta' has the model's metadata.
'data' contains the drift prediction and optionally the test-statistic and threshold.
- 'meta' has the model's metadata.
- 'data' contains the drift prediction and optionally the test-statistic and threshold.
"""
return self._detector.predict(x_t, return_test_stat)

Expand Down
Loading