NBS Test Cases

[cbrinson-rise8]

Description

NBS provided us a spreadsheet that defines an existing state and then a bunch of tests with expected results. Architect a testing script of sorts, that can parse a similar spreadsheet (can just be a CSV) with seeding data and iterate through the tests with the expectations.

Related Issues

closes #110

Additional Notes

For a full explanation of run instructions, directory structure, and file uses check out the README.md

The key inputs to these tests will be the following:

• a CSV file that can be used to seed the MPI
• a CSV with test cases (basically the same format as the above), but also a column to indicate if it is a match and to whom
• a JSON file with an algorithm config to use
  The setup should make it easy to edit any 3 of those above files, run the test cases, and examine the results.

<--------------------- REMOVE THE LINES BELOW BEFORE MERGING --------------------->

Checklist

Please review and complete the following checklist before submitting your pull request:

• I have ensured that the pull request is of a manageable size, allowing it to be reviewed within a single session.
• I have reviewed my changes to ensure they are clear, concise, and well-documented.
• I have updated the documentation, if applicable.
• I have added or updated test cases to cover my changes, if applicable.
• I have minimized the number of reviewers to include only those essential for the review.

Checklist for Reviewers

Please review and complete the following checklist during the review process:

• The code follows best practices and conventions.
• The changes implement the desired functionality or fix the reported issue.
• The tests cover the new changes and pass successfully.
• Any potential edge cases or error scenarios have been considered.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.15%. Comparing base (3a7e58b) to head (4000764).
Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #123   +/-   ##
=======================================
  Coverage   97.15%   97.15%           
=======================================
  Files          30       30           
  Lines        1301     1301           
=======================================
  Hits         1264     1264           
  Misses         37       37           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ericbuckley]

@cbrinson-rise8 I was going through the README and ran the default tests. I noticed the all of the expected results in the output were either a failure or a match, which is different from the results.csv file you committed. Any ideas why my results would have been different from the ones in the PR?

[cbrinson-rise8]

@cbrinson-rise8 I was going through the README and ran the default tests. I noticed the all of the expected results in the output were either a failure or a match, which is different from the results.csv file you committed. Any ideas why my results would have been different from the ones in the PR?

Hmm not sure but the output file I committed might just be old. I'll run through running the tests and see what my new output file would look like.

[alhayward]

Thanks for this work! Excited that we'll have the functionality to test results, which paves the way for calculating match accuracy performance metrics!

Left several questions and a few suggestions.

[ericbuckley]

@cbrinson-rise8 I think pretty much all of my previous questions have been resolved. I did push a small change to update some language in the README and the sample tests cases, please take a look to make sure you agree with those.

My only remaining thought is I think we should remove tests/algorithm/results/output.csv. That's a generated file and not source, so I think it is best to remove that. Additionally, we may want to add a rule to the .gitignore file, so future devs don't accidentally commit that file. Thoughts?

[ericbuckley]

@cbrinson-rise8 FYI, I also rebased against main and adjusted the algorithm_configuration.json accordingly. 8e1575d