-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add function to create expected outputs for deterministic tests #90
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was wondering if we might consider adding a cleanup as an optional feature as well. Since we will create a subfolder for each deterministic category, manually cleaning them up could become messy.
Would it be possible to include something like wasmtest.sh cleanup
or wasmtest.sh cl
, to remove all the /expected
folders and their contents?
Changes from Review comments: Note: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good, agree with the previous comments. @yashaswi2000 We should probably try to cache these expected outputs though since they shouldnt change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good after my suggestions were discussed. Thank you.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
PR for #11
Added a function in wasmtest.sh file for creating expected outputs.
The function can be ran using "wasmtest.sh createexpected" or "wasmtest.sh ce"
All c files under each of the deterministic subfolder are taken and compiled using gcc.
A new folder expected is created in each deterministic subfolder and the output is saved in a <test_case_filename>.output file
If there are any compile or runtime error, they are displayed in the console and alse added to the .output file.
The compiled files are deleted after running
Finally a count of total/success/failure is displayed in the console