-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test: Local evaluation testing #108
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've reviewed this. It all looks ok to me and seems to check mostly the right things, the main issue (as per my comment) is that we're not checking that the values are the same before and after the migration (which I think we should expect?).
I'd say that we should implement some way to verify this, it could just be a local json file or something, or a pickled python object written to a file maybe?
tests/test_local_eval.py
Outdated
return _get_flag_mock( | ||
value=get_matching_str( | ||
"environment default", | ||
'overridden by "split_segment"', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we can, I'd like to be more specific here. Whether the identity belongs to the percentage split segment shouldn't change before and after migration. I think this is going to mean that we're just checking that it takes one of the 2 possible values.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Switched to static values.
9e1508c
to
e11c727
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approving to confirm that I'm happy with the tests being performed (not as a suggestion we should merge).
DO NOT MERGE. This contains an automated SDK test for Edge V2 migration.
To run, clone, then:
After the migration, run: