-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add dataloader and Semcova data #10
Conversation
Sample data
Excellent! I have a couple of questions before validating the PR though. Referencing this issue you're saying that each video processing step takes around 3min.. What does that include exactly? Loading all the frames and calculating the ppg for a single video? |
Hi again, I got a little time to look into the dataloader you've implemented. It's really nice, I have never used pytorch for this but it seems useful. Since the function to calculate the PPG is done in numpy and openCV however, I could look into that a little more. One of the reason the method is slow to load (I think) is due to the calculation of the mean and std with an axis argument. When I timed it it was much slower than taking a view of the frame and calculating the mean and std on those views. Here is a version of the class where every functoin is timed and I've implemented a
And the times of the new one:
I've only tried it on one video file so far. Let me think what you think of it and if you feel like it add it to the push-request :)
|
That sounds amazing, thank you Malcolm. I tried it and indeed it speeds up the process considerably. Also by 3 minutes I was talking about the whole while loop for each video, now it took 50 seconds on my machine (I am using the sample_data videos). I won't be working on this for the rest of the day probably, if you can update the PR with the changes that would be great. We can also get rid of the threading version as it won't be of much help. |
Do not calculate the mean and std using axis so that the function goes much faster
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok should be all good
Nice work, guys! 👏 |
Added pytorch dataset and dataloader.
Added Semcova data.