Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Temperamental tests #37

Open
htjb opened this issue Aug 21, 2023 · 3 comments
Open

Temperamental tests #37

htjb opened this issue Aug 21, 2023 · 3 comments

Comments

@htjb
Copy link
Owner

htjb commented Aug 21, 2023

Some of the tests occasionally fail because of poor training.

@htjb
Copy link
Owner Author

htjb commented Mar 11, 2024

Okay, in the tests we calculate the accuracy of the MAFs and I think KDEs too by running a series of two sample KS tests between samples drawn from the DE and the original samples used to train. Occasionally one of these KS tests fails because the training is poor (training varies with the random seed since the weights are initialised differently).

A simpler thing to do would be a KS test between the predicted distributions of log-probability vs the log-probability of the original samples. Might be more robust. Also I think the tests currently work with a NS run but should use something from the scipy.stats package.

Also appears there is a bug with tensorflow too after their recent updates (identified in #59 I think).

@htjb
Copy link
Owner Author

htjb commented Apr 2, 2024

Would be good to remove the pytorch dependency from the tests as well. Seems to take a long time to install and not really needed.

@htjb
Copy link
Owner Author

htjb commented Apr 2, 2024

and test a wider range of python versions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant