Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automated Testing #101

Open
mperrin opened this issue Sep 14, 2016 · 5 comments
Open

Automated Testing #101

mperrin opened this issue Sep 14, 2016 · 5 comments
Assignees

Comments

@mperrin
Copy link
Contributor

mperrin commented Sep 14, 2016

The Keck OSIRIS development branch has already worked out how to connect pytest and IDL. Since the OSIRIS pipeline is one of the parents of the GPI data pipeline, it ought to be reasonably straightforward to port this stuff over.

https://github.com/Keck-DataReductionPipelines/OsirisDRP/tree/develop

@mperrin
Copy link
Contributor Author

mperrin commented Sep 14, 2016

Looks like some amount of what's that subtree is equivalent to functionality we already have in gpipy - i.e. tools for parsing XML files etc. So we can use gpipy instead of that stuff.

@mperrin
Copy link
Contributor Author

mperrin commented Sep 14, 2016

Notes - @mperrin will handle setup of infrastructure. @kfollette will make some tests. Let's assume for now the tests will run on a machine that has a local copy of the full Dropbox (e.g. at Berkeley, Cornell, or STScI) and a local working IDL.

Let's not worry about running this in the cloud on travis or anything like that. Once we have the basic test framework functional, we can use Github web hooks to trigger a local test run on one of our computers for every github commit. See https://developer.github.com/webhooks/

@mperrin
Copy link
Contributor Author

mperrin commented Sep 14, 2016

End-to-end functional test suite

  • basic make-a-cube
  • spec mode - beta pic or ASU data - make cube and check sat spots
  • pol mode - 4796A check center and check polarized flux exists in radial pol image
  • calibrations - make some quick wave cal

Implementation

  • Each test will be a subdirectory of tests, which contains at minimum one XML recipe and one python function that implements the test criteria.
  • The input files for the tests will exist in Dropbox\Pipeline Releases\Data_for_DRP_Tests, with a subdirectory for each test. In addition to the input raw files this can optionally contain output files we want to compare to. We can refer to this directory as $GPIESTESTDATA in recipes.
  • We assume the user will already have a working calibration database and don't have to do anything manually about that.

@mperrin mperrin changed the title Automated Unit Testing Automated Testing Sep 14, 2016
@semaphoreP
Copy link
Contributor

Is there anything left to do on this issue? I recall testing the testing code and being happy with the current state. I guess the one thing mentioned that we haven't done is adding a hook so that the tests are run locally on our machines?

@mperrin
Copy link
Contributor Author

mperrin commented Jan 13, 2017

Yeah, the remaining task is setting up some continuous integration setup so these tests actually get executed semi regularly somewhere.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants