Benchmark load

[briancoutinho]

What does this PR do?

Adds benchmarks for trace load and parsing #48

Before submitting

• Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
  • N/A
• Did you write any new necessary tests?
  • N/A
• Did you make sure to update the docs?
  • N/A
• Did you update the changelog?
  • N/A
    not sure?

Benchmark results

Run using

python3 benchmarks/trace_load_benchmark.py -p 10 -l 1

Before Anupam's PR #48 optimizing out the get_memory_size we see the load/parsing times were

parse[tests/data/vision_transformer]: Mean +- std dev: 12.4 sec +- 0.5 sec
parse[tests/data/inference_single_rank]: Mean +- std dev: 15.0 sec +- 0.5 sec

After his PR we now see the time reduced for loading

parse[tests/data/vision_transformer]: Mean +- std dev: 6.24 sec +- 0.19 sec
parse[tests/data/inference_single_rank]: Mean +- std dev: 9.25 sec +- 0.19 sec

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.53%. Comparing base (ec3abf1) to head (ee928eb).
Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main      #49   +/-   ##
=======================================
  Coverage   90.53%   90.53%           
=======================================
  Files          43       43           
  Lines        3899     3899           
=======================================
  Hits         3530     3530           
  Misses        369      369           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[louisfeng]

This looks pretty useful to me. Are you ready to land it?

[fengxizhou]

Thanks for adding this benchmark tool. It would help us to optimize the code for higher performance.

[facebook-github-bot]

@briancoutinho has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[facebook-github-bot]

@briancoutinho merged this pull request in 4ac4385.