Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Column Order Bug #5

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Fix Column Order Bug #5

wants to merge 2 commits into from

Conversation

tiger5226
Copy link

Added objectID position tracking. When the template is generated the ordering of conflicting unique columns is alphabetical. This means we cannot assume the objectID can always be appended. It might need to be inserted at a specific position. This adds support for tracking the position and inserting it at the correct position so the delete query correctly deletes the conflicts.

@tiger5226
Copy link
Author

tiger5226 commented Jun 12, 2019

@lyoshenka I was investigating an issue where @tzarebczan got an error on merging two users each with wallet table entries. The wallet table has a Many to 1 relationship with the user table, with uniqueness on user_id and version. I traced the code and apparently the templating used in SQLBoiler writes the conflicting unique columns in alphabetical order. This is problematic because the wallet table is the first table to have unique keys where the user_id is before the "other"(version) columns, and we append the user_id for the delete query which makes the arguments out of order. As a result the delete query does not delete the conflict because there is no match.

What I did to solve this was to keep track of the index of the object id based on the pre-compliment slice, then insert the value in the query argument slice where it is supposed to be rather than appending or prepending it. This should solve all future issues where there are more than 2 columns and the object id column could be anywhere in the array depending on the alphabetical order.

@tzarebczan We do not have to merge this right away as it is only impacts the wallet table rows conflict since this is the only table with the columns out of the assumed order. So we can fix the wallet table issue first, handle these merge errors manually in the meantime and once finished we can merge this to fix the issue. @nikooo777 We just have to be aware of this bug when we add tables with uniqueness columns. We just got lucky that this was the first table where the assumed order did not match the alphabetical order.

@tiger5226
Copy link
Author

BTW, something else to consider here is the possibility of the argument reversal deleting the wrong wallet table entry. In theory this could cause us to lose wallet data. However, I believe this is impossible since these queries are part of a transaction and if it deletes the wrong row, the conflict still exists and the transaction will be rolled back. I don't see a plausible scenario where we delete the wrong row and the merge is still successful.

Copy link
Member

@lyoshenka lyoshenka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm only 90% sure I understand what the issue is but I appreciate your detailed writeup and this sounds like a reasonable solution.

templates/singleton/boil_queries.tpl Show resolved Hide resolved
@lyoshenka
Copy link
Member

I'm guessing this bug snuck in when we added code to deal with multi-column relations, right?

@tiger5226
Copy link
Author

tiger5226 commented Jun 13, 2019

Yes. We always assumed the position of the user_id column in the query argument list and the conflicting column came second. See here. I swapped them around in the query update I did. The real problem came in when I made the assumption on the ordering of the columns automatically generated. The old way it was just hardcoded, user_id first check, first complimented conflict second, so it never depended on the ordering because it was only using the first conflicting one.

We did not see it until the wallet table was added and in use due to the alphabetical ordering.

@nikooo777
Copy link
Collaborator

BTW, something else to consider here is the possibility of the argument reversal deleting the wrong wallet table entry. In theory this could cause us to lose wallet data. However, I believe this is impossible since these queries are part of a transaction and if it deletes the wrong row, the conflict still exists and the transaction will be rolled back. I don't see a plausible scenario where we delete the wrong row and the merge is still successful.

please let's add a test for this

@nikooo777
Copy link
Collaborator

I regret stalling this PR now hahaha.

I have been dealing with this issue for a whole month now where generating the models it just randomly and not always reorders the columns in alphabetical order.
I still don't really understand the problem here and tbh I'm a bit afraid of merging this PR without understanding it.

…ordering of conflicting unique columns is alphabetical. This means we cannot assume the objectID can always be appended. It might need to be inserted at a specific position. This adds support for tracking the position and inserting it at the correct position so the delete query correctly deletes the conflicts.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants