TEST: Add scripts for performance profiling #129
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Leengit
force-pushed
the
performance_profiling
branch
4 times, most recently
from
1e3c27a to
23366b5
Compare
Leengit
force-pushed
the
performance_profiling
branch
4 times, most recently
from
2afee06 to
2d7b099
Compare
COMP: Update versions of package dependencies
Leengit
force-pushed
the
performance_profiling
branch
2 times, most recently
from
3a8186b to
816b1b0
Compare
ENH: Use time.time consistently rather than time.perf_counter ENH: Use WrappedModel consistently rather than WrapModel
ENH: Add show_structure and predict_and_retrieve
Leengit
force-pushed
the
performance_profiling
branch
from
1ee7887 to
f07bb2a
Compare
ENH: Create create_and_predict so that torch does not crash with a
large number of un-detached tensors
Leengit
force-pushed
the
performance_profiling
branch
from
f07bb2a to
9c29646
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Starts the work towards satisfying #102 by implementing 4 performance scripts. These are a start, but further work will be needed.
Also, towards the performance goals, explores ways to incorporate multiple workers / multiple process into the
torchDatasetandDataLoader. Thepredict_and_detachfunction proved essential to stoptorchfrom crashing with too may un-detached tensors. Other approaches were of less use, were ultimately culled, but are left as intermediate git commits to document what was researched. (For example,DataLoaders stalled inexplicably when iterated within atorch.multiprocessing.Process.) The final commit indicates future directions for this aspect.