Same, same but different ...
samesame implements classifier two-sample tests (CTSTs) and as a bonus
extension, a noninferiority test (NIT).
These were either missing or implemented with some tradeoffs
(looking at you, sample-splitting) in existing libraries. And so,
samesame fills in the gaps :)
What is samesame good for? It is for data (model) validation, performance
monitoring, drift detection (dataset shift), statistical process control, and
on and on.
Want more?
Here you go.
This motivating example comes from the related R package
dsos.
To install, run the following command:
python -m pip install samesameBelow, you will find an overview of common modules in samesame.
| Function | Module |
|---|---|
| Bayesian inference | samesame.bayes |
| Classifier two-sample tests (CTSTs) | samesame.ctst |
| Noninferiority tests (NITs) | samesame.nit |
When the method is a statistical test, samesame saves (stores) the results of
some potentially computationally intensive results in attributes. These
attributes, when available, can be accessed as follows.
| Attribute | Description |
|---|---|
.statistic |
The test statistic for the hypothesis. |
.null |
The null distribution for the hypothesis. |
.pvalue |
The p-value for the hypothesis. |
.posterior |
The posterior distribution for the hypothesis. |
.bayes_factor |
The bayes factor for the hypothesis. |
To get started, please see the documentation and examples.
samesame has few dependencies beyond the standard library. It will
probably work with some older Python versions. It is, in short, a lightweight
dependency for most machine learning projects.samesame is built on top of,
and is compatible with, scikit-learn and numpy.