Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

experimental: add post-processing logic #440

Merged
merged 2 commits into from
Jul 8, 2024
Merged

Conversation

DavidKorczynski
Copy link
Collaborator

@DavidKorczynski DavidKorczynski commented Jul 6, 2024

This is needed to easily digest the results from a larger scale run

@DavidKorczynski
Copy link
Collaborator Author

Sample run:

$ cat ./targets.txt 
https://github.com/gregjesl/simpleson
https://github.com/memononen/nanosvg
https://github.com/codeplea/tinyexpr
$ python3 ./runner.py -o ../../../oss-fuzz-3/ -i ./targets.txt -t 6 -m openai
....
...
$ python3 ./post-process.py ../../../oss-fuzz-3                                                        
[post-process.py:163]: Analysing: ../../../oss-fuzz-3/build/out/temp-project-0
[post-process.py:174]: Results: https://github.com/gregjesl/simpleson
[post-process.py:175]: - Language: c++
[post-process.py:176]: - Functions from Fuzz Introspector: 108
[post-process.py:178]: - Harnesses generated: 7
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-0/autogen-results/test-fuzz-build-2-fuzzgen-1/fuzz-run.err :: 18 :: 221 :: [asan errors: 1] :: coverage: 28.930818 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-0/autogen-results/test-fuzz-build-1-fuzzgen-8/fuzz-run.err :: 14 :: 216 :: [asan errors: 1] :: coverage: 31.238779 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-0/autogen-results/test-fuzz-build-1-fuzzgen-1/fuzz-run.err :: 18 :: 196 :: [asan errors: 1] :: coverage: 26.145553 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-0/autogen-results/test-fuzz-build-0-fuzzgen-1/fuzz-run.err :: 18 :: 160 :: [asan errors: 1] :: coverage: 23.899371 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-0/autogen-results/test-fuzz-build-0-fuzzgen-8/fuzz-run.err :: 14 :: 159 :: [asan errors: 1] :: coverage: 24.955117 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-0/autogen-results/test-fuzz-build-2-fuzzgen-8/fuzz-run.err :: 14 :: 105 :: [asan errors: 1] :: coverage: 29.174147 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-0/autogen-results/test-fuzz-build-2-fuzzgen-0/fuzz-run.err :: 2 :: 10 :: [asan errors: 0] :: coverage: 2.066487 
[post-process.py:163]: Analysing: ../../../oss-fuzz-3/build/out/temp-project-1
[post-process.py:174]: Results: https://github.com/memononen/nanosvg
[post-process.py:175]: - Language: c
[post-process.py:176]: - Functions from Fuzz Introspector: 173
[post-process.py:178]: - Harnesses generated: 16
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-1/autogen-results/test-fuzz-build-1-fuzzgen-1/fuzz-run.err :: 25 :: 862 :: [asan errors: 0] :: coverage: 55.141727 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-1/autogen-results/test-fuzz-build-1-fuzzgen-7/fuzz-run.err :: 2 :: 818 :: [asan errors: 0] :: coverage: 39.282186 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-1/autogen-results/test-fuzz-build-0-fuzzgen-7/fuzz-run.err :: 2 :: 816 :: [asan errors: 0] :: coverage: 47.836326 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-1/autogen-results/test-fuzz-build-2-fuzzgen-1/fuzz-run.err :: 25 :: 746 :: [asan errors: 0] :: coverage: 38.991430 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-1/autogen-results/test-fuzz-build-2-fuzzgen-13/fuzz-run.err :: 32 :: 735 :: [asan errors: 0] :: coverage: 55.511551 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-1/autogen-results/test-fuzz-build-3-fuzzgen-7/fuzz-run.err :: 2 :: 726 :: [asan errors: 0] :: coverage: 50.576227 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-1/autogen-results/test-fuzz-build-0-fuzzgen-1/fuzz-run.err :: 25 :: 717 :: [asan errors: 0] :: coverage: 36.564526 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-1/autogen-results/test-fuzz-build-0-fuzzgen-13/fuzz-run.err :: 32 :: 703 :: [asan errors: 0] :: coverage: 38.289086 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-1/autogen-results/test-fuzz-build-1-fuzzgen-13/fuzz-run.err :: 32 :: 657 :: [asan errors: 0] :: coverage: 44.092409 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-1/autogen-results/test-fuzz-build-2-fuzzgen-7/fuzz-run.err :: 2 :: 648 :: [asan errors: 0] :: coverage: 41.389529 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-1/autogen-results/test-fuzz-build-3-fuzzgen-13/fuzz-run.err :: 32 :: 628 :: [asan errors: 0] :: coverage: 39.834983 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-1/autogen-results/test-fuzz-build-3-fuzzgen-1/fuzz-run.err :: 25 :: 624 :: [asan errors: 0] :: coverage: 41.957811 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-1/autogen-results/test-fuzz-build-0-fuzzgen-0/fuzz-run.err :: 2 :: 358 :: [asan errors: 0] :: coverage: 26.456739 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-1/autogen-results/test-fuzz-build-3-fuzzgen-0/fuzz-run.err :: 2 :: 328 :: [asan errors: 0] :: coverage: 37.590520 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-1/autogen-results/test-fuzz-build-2-fuzzgen-0/fuzz-run.err :: 2 :: 302 :: [asan errors: 0] :: coverage: 27.649770 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-1/autogen-results/test-fuzz-build-1-fuzzgen-0/fuzz-run.err :: 2 :: 284 :: [asan errors: 0] :: coverage: 31.566820 
[post-process.py:163]: Analysing: ../../../oss-fuzz-3/build/out/temp-project-2
[post-process.py:174]: Results: https://github.com/codeplea/tinyexpr
[post-process.py:175]: - Language: c
[post-process.py:176]: - Functions from Fuzz Introspector: 65
[post-process.py:178]: - Harnesses generated: 14
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-2/autogen-results/test-fuzz-build-2-fuzzgen-15/fuzz-run.err :: 30 :: 180 :: [asan errors: 0] :: coverage: 34.369115 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-2/autogen-results/test-fuzz-build-1-fuzzgen-15/fuzz-run.err :: 30 :: 170 :: [asan errors: 0] :: coverage: 33.081285 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-2/autogen-results/test-fuzz-build-2-fuzzgen-16/fuzz-run.err :: 28 :: 169 :: [asan errors: 0] :: coverage: 33.084809 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-2/autogen-results/test-fuzz-build-1-fuzzgen-21/fuzz-run.err :: 30 :: 168 :: [asan errors: 0] :: coverage: 34.463277 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-2/autogen-results/test-fuzz-build-1-fuzzgen-16/fuzz-run.err :: 28 :: 162 :: [asan errors: 0] :: coverage: 33.208606 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-2/autogen-results/test-fuzz-build-1-fuzzgen-4/fuzz-run.err :: 28 :: 161 :: [asan errors: 0] :: coverage: 32.645403 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-2/autogen-results/test-fuzz-build-2-fuzzgen-4/fuzz-run.err :: 28 :: 161 :: [asan errors: 0] :: coverage: 32.710280 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-2/autogen-results/test-fuzz-build-2-fuzzgen-21/fuzz-run.err :: 30 :: 159 :: [asan errors: 0] :: coverage: 34.146341 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-2/autogen-results/test-fuzz-build-1-fuzzgen-3/fuzz-run.err :: 30 :: 152 :: [asan errors: 0] :: coverage: 34.277620 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-2/autogen-results/test-fuzz-build-2-fuzzgen-9/fuzz-run.err :: 30 :: 152 :: [asan errors: 0] :: coverage: 32.956685 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-2/autogen-results/test-fuzz-build-2-fuzzgen-3/fuzz-run.err :: 30 :: 151 :: [asan errors: 0] :: coverage: 33.019755 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-2/autogen-results/test-fuzz-build-1-fuzzgen-9/fuzz-run.err :: 30 :: 151 :: [asan errors: 0] :: coverage: 34.310019 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-2/autogen-results/test-fuzz-build-1-fuzzgen-22/fuzz-run.err :: 28 :: 50 :: [asan errors: 0] :: coverage: 32.833021 
[post-process.py:180]: ../../../oss-fuzz-3/build/out/temp-project-2/autogen-results/test-fuzz-build-2-fuzzgen-22/fuzz-run.err :: 28 :: 43 :: [asan errors: 0] :: coverage: 32.710280 

@DavidKorczynski
Copy link
Collaborator Author

/gcbrun skip

@DavidKorczynski DavidKorczynski requested a review from DonggeLiu July 6, 2024 13:34
LOG_FMT = '[%(filename)s:%(lineno)d]: %(message)s'


def _get_edge_cov_from_line(line: str) -> Optional[int]:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A regex for the same purpose:

m = LIBFUZZER_COV_REGEX.match(line)

Feel free to use either.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do this in a similar fashion as mentioned in #440 (comment)


def oss_fuzz_out_dir(oss_fuzz_dir: str) -> str:
"""Returns the out folder of a given OSS-Fuzz repository."""
return os.path.join(oss_fuzz_dir, 'build', 'out')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is quite generally useful, how about relocating it to oss_fuzz_checkout.py, where we store all OSS-Fuzz-related functions? (e.g., if so, please also consider removing oss_fuzz in the function name to avoid stuttering, e.g., get_out_dir() or out_dir() as a @property)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I'll leave this for now as this PR is in a bit grander-scheme an effort to prepare for merging the codebases #450 -- A larger PR will come in the near future that does this on a more rigorous level so will do this at that point

Signed-off-by: David Korczynski <[email protected]>
@DavidKorczynski
Copy link
Collaborator Author

/gcbrun skip

@DavidKorczynski DavidKorczynski merged commit f0f75d8 into main Jul 8, 2024
7 checks passed
@DavidKorczynski DavidKorczynski deleted the experimental-fix-300 branch July 8, 2024 09:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants