✨ Add WriteLog replay
#2783
✨ Add WriteLog replay
#2783
Conversation
alexras
force-pushed
the
alexras/write-log-replay
branch
from
63e1d66 to
93aa6f6
Compare
alexras
force-pushed
the
alexras/write-log-replay
branch
from
93aa6f6 to
f2e4087
Compare
alexras
force-pushed
the
alexras/write-log-replay
branch
from
f2e4087 to
1511b62
Compare
| @@ -52,8 +52,7 @@ class WriteLog(models.Model): | |||
| created_by_uid = models.CharField(max_length=8, default=DEFAULT_CREATED_BY_UID) | |||
| branch_code = models.IntegerField(default=DEFAULT_BRANCH_CODE) | |||
| run_uid = models.CharField(max_length=16, default=DEFAULT_RUN_UID) | |||
| # Many-to-many tables don't have row UIDs, so this needs to be nullable. | |||
| record_uid = models.JSONField(null=True) | |||
| record_uid = models.JSONField() | |||
There was a problem hiding this comment.
I'm only seeing this now, but I think this should likely be a CharField rather than a JSONField, should it?
There was a problem hiding this comment.
I tried this in the next PR and it seems there is a good reason for that this is a JSON field. Many tests break if turning it into CHARField. I think this is because you're modeling multiple uids as a list within the JSON field.
There was a problem hiding this comment.
The write log record's UID takes one of two forms:
- For tables that are many to many, it's a list of 3-tuples specifying the table's foreign-key relationships by UID
- For all other tables, it's an ordered list of values of the table's UID fields.
Because of that it's easier if those fields are JSONFields rather than CharFields.
| key_constraint=None, | ||
| ) | ||
| ] | ||
| elif table == "lamindb_param": |
There was a problem hiding this comment.
This table does no longer exist. I consciously removed it. So, no need to add it back.
There was a problem hiding this comment.
Can be removed in the next PR.
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2783 +/- ##
==========================================
- Coverage 91.94% 91.80% -0.15%
==========================================
Files 70 74 +4
Lines 10717 9871 -846
==========================================
- Hits 9854 9062 -792
+ Misses 863 809 -54 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Deployment URL: https://b475b581.lamindb.pages.dev |
|
This branch can be deleted as soon as @alexras changes the base of his other open PR. |
This diff implements a "write log replayer" that can apply individual write log records (specified as
WriteLogmodel instances) to the current instance's database. As a side-effect of this change, I had to make more of the system aware of individual columns' types, and I've moved a couple things shared by both the trigger installer and write log replayer into utility functions.