Add optimizations to figures daily job #131
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR is for the optimization of our daily job for updating figures
populate_daily_metrics
It takes around 11 hours to complete on prod.This PR:
figures_enrollmentdata
This reduces queries that were executing on each enrollment separately.update_learners_activity_for_date
from inside a loop over courses to outside of the loop which is over sites. This function executes on site level and has no course specific processing. Executing this function for each course was redundant and was slowing down the figures job.These changes have reduced the jobs time from around 11-13 minutes to 8 to 8.30 minutes. We should see better improvement on prod since it has much more data.
Ticket
https://projects.arbisoft.com/project/edly-product/task/7068