Upsert Application Submissions

[rm03]

Previously, we would create new ApplicationSubmission and ApplicationQuestionResponse objects every time a user submitted their application; this change ensures that each user will have at most one submission per application per committee.

Going forward, we should be able to remove raw SQL queries like this since users won't have multiple submissions associated with an application, but this would be potentially problematic for past applications unless we run a script that drops all submissions that aren't the most recent.

[rohangpta]

Somehow I missed this! Looks great, I'm going to try and run some benchmarks on throughput with this code before merging in.

Curious if moving from create to update_or_create will give us the performance benefits that we hypothesised

[rohangpta]

RE: Raw SQL queries -- it's slightly tricky to figure out how exactly to migrate this logic. An approach we can consider:

• Compute the inverse of the queryset returned under RawSQL (everything but the most recent for each app) and delete them. Do this by hand via SSH (do you have access to this?).
• Migrate logic to omit the RawSQL cruft.
• For sanity, add a unique_together check on "application", "committee" and "user" for the ApplicationSubmission model (and maybe something of the sort to ApplicationQuestionResponse).

If we're doing this, let's also get stats on how much extra data we were storing. I'd imagine we package all these changes + other optimisations into a nice blog post by end of this month.

[rm03]

As discussed:

• We had a total of 40,457 ApplicationSubmission objects stored in the database and 24,112 of them are not most recent submissions.
• Additionally, there were 165,269 ApplicationQuestionResponse objects associated with these submissions (out of a total of 274,445), which means that we've been storing quite a bit of extra data.

I've removed these objects from the database so the last commit shouldn't be problematic.