Evaluation on docker stats #5
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…utomatically removed 2. Added a shell script collect_stats.sh that will collect docker container stats when you run start_training.sh
…ontinue with implementing comparison between the two implementations
…ced implementation in the same plot for CPU %. Need to fix for memory usage
…now investigate other plot alternatives
… for three workers with graphs showing accumulated values over time
GallonSong
reviewed
Dec 2, 2020
| @@ -0,0 +1,5 @@ | |||
| while true; | |||
There was a problem hiding this comment.
Is this file still needed?
| @@ -23,6 +23,9 @@ fi | |||
|
|
|||
| if [ "$SCENARIO" == "local" ]; | |||
| then | |||
| cd performance | |||
| ./collect_stats.sh & | |||
| cd - | |||
There was a problem hiding this comment.
(1/2) How about replacing these 3 lines with 1 line sh ./performance/collect_stats.sh &, and also make a small change in collect_stats.sh, see (2/2)
performance/collect_stats.sh
Outdated
| @@ -0,0 +1,5 @@ | |||
| while true; | |||
| do | |||
| docker stats --format "table {{.Name}},{{.CPUPerc}},{{.MemUsage}},{{.MemPerc}}" --no-stream >> stats.csv | |||
There was a problem hiding this comment.
(2/2) Follow comment (1/2), change this line to **** >> performance/stats.csv
performance/eval_stats.py
Outdated
| blue = mpatches.Patch(color='blue', label='Original implementation') | ||
| plt.legend(handles=[red, blue]) | ||
| plt.title('Accumulated Sum of Workers Memory Usage') | ||
| plt.show() |
There was a problem hiding this comment.
Nice plots and handling the data with Pandas!
A small suggestion: wrapping repeated code blocks into reusable functions, time-saving and easier for future maintenance.
…e time and remove sleep
…al_stats.py has been updated to handle the new format of the csv file
…the original data
…workers instead of three and three runs for each implementation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
DO NOT MERGE
This branch can be used for collecting stats from our enhanced implementation and make evaluation on collected stats with data using three workers on the original implementation and enhanced implementation.
Collect stats:
start_training.shyou will automatically collect data every 0.5sps aux | grep start_trainingwhich will generate two outputs containing ID in the second column. Runkill <ID>where the ID can be found in the second column in one of the two outputs from previousEvaluate stats:
eval_stats.pyinperformance folderstart_training.shandcollect_stats.shin a folder namnedperformancein Felix's master branch (do not push) in the same way it's implemented in this branchFuture work:
Graphs that I got can be found here