New: Field-level CloudKit record merging

[stephencelis]

Important

The changes released today are breaking and may be incompatible with CloudKit containers created for earlier versions of the alpha. If you experience any issues with the update, try rotating your CloudKit container to a fresh one to see if the issue is resolved.

Today we introduced a new feature to our CloudKit alpha branch: field-level record merging.

Previously, sync events took a last-wins approach, where if device A and device B update a record before they receive each other's updates, the record sent from the last device would write over the changes sent from the first device.

For example:

1. Device A updates a reminder's priority from "low" to "medium"
2. Device B updates the original reminder's due date to today
3. When sync settles, the reminder sent from device B wins, and the change to "medium" reverts to "low"

This is no longer the case. Our CloudKit tools now track when specific fields are updated and moves the last-wins approach to the field level.

In the above case:

3. When sync settles, the final reminder has a priority of "medium" and today as its due date.

We may further support custom conflict resolution strategies in the future, but for now we hope this addresses a scenario that could be common, especially for records that are shared with other participants.

[Narsail]

In your changes you added a bunch of 'ON CONFLICT REPLACE' statements to some of the ReminderList fields. Is that a necessary change for my schemas as well to work with the field level conflict solution?

[stephencelis]

@Narsail Those are unrelated, but general good practices to have when synchronizing old records to devices with newer schemas. Without it the synchronization of those inserts can fail.

[KaiOelfke]

Is there an API to choose between record vs field level resolution? For a reminder the field level makes sense.

But, if I have a record with a start and end time, where I want the invariant of start time < end time it could be an issue.

Start: 10:00
End: 13:00

Device A:
Change start to 11:00

Device B:
Change end to 10:30

After merging:
Start 11:00
End 10:30

Maybe this problem could be avoided with a start time and duration data scheme. But I'm not sure this works for all data schemes where there's some invariants.

I think it should per entity to decide whether it's suitable for field / column or record / row resolution.

[mbrandonw]

Hi @KaiOelfke, there is no API to decide between merging strategies right now, but we are open to adding it in the future.

Maybe this problem could be avoided with a start time and duration data scheme. But I'm not sure this works for all data schemes where there's some invariants.

Combining start and end times into a single field would be one way to fix this. But in general, even with "last edit wins" for the whole record can break invariants in your database because you may have invariants on the associations of the record saved. So there is no perfect answer here.

And really, if you want to enforce invariants in your data, then the best place to do this is at the database level. You can create a trigger that only runs when data is being updated from the server (you can use the SyncEngine.isUpdatingRecord() function to do this), and when it detects and invalid time range is saved it takes some action. For example, it could revert the data to its previous version, or it could set start = end.

[KaiOelfke]

Thank you for the explanation. I am aware of the association invariant problem, but for my cases it would be only for non association columns. I'll have to learn how to set up such triggers then so the new series will be useful for me. This means, I can use it as long as I can restore a valid state based on current local data and incoming server data, but without all transactions from other devices that caused this as there's only going to be one already merged row from the server.

In my case, I would try to take either only the local start end or the server start end depending on what's more recent (LWW). So I guess I could make such a trigger with SyncEngine.isUpdatingRecord() and then evaluate timestamps from the metadata table in the trigger query.