Trigger Based Table Diff Tracking

[stevensJourney]

Overview

We currently expose watched query functionality which allows watching changes of SQLite query's result set. This PR adds methods which can be used to track and process changes to individual SQLite tables.

Temporary SQLite triggers and tables are used to populate a set of table change operations. Records for table row INSERT, UPDATEandDELETE` operations are created as changes occur.

This PR currently adds 2 new SQLite trigger based methods.

Creating a Diff Trigger

createDiffTrigger Configures temporary SQLite triggers which populate a temporary SQLite table with operations as they occur.

const dispose = await database.triggers.createDiffTrigger({
  source: 'lists',
  destination: tempTable,
  columns: ['name'],
  operations: [DiffTriggerOperation.INSERT, DiffTriggerOperation.UPDATE]
});

// logic performs operations on the lists table

// The changes can be fetched in a writeLock

await database.writeLock(async (tx) => {
  const changes = await tx.getAll<TriggerDiffRecord>(/* sql */ `
    SELECT
      *
    FROM
      ${tempTable}
  `);

  // Process changes

  // Clear the temp table after processing
  await tx.execute(/* sql */ ` DELETE FROM ${tempTable}; `);
});

Processing Change Operations

A more convenient trackTableDiff method uses createDiffTrigger to get table changes, but adds additional logic to consume the changes as they occur.

The example below uses triggers to process todo records as they are inserted/created.

const dispose = await database.triggers.trackTableDiff({
  source: 'todos',
  columns: ['list_id'],
  when: { [DiffTriggerOperation.INSERT]: `json_extract(NEW.data, '$.list_id') = '${firstList.id}'` },
  operations: [DiffTriggerOperation.INSERT],
  onChange: async (context) => {
    // Fetches the todo records that were inserted during this diff
    const newTodos = await context.getAll<Database['todos']>(/* sql */ `
      SELECT
        todos.*
      FROM
        DIFF
        JOIN todos ON DIFF.id = todos.id
    `);

    // Process newly added TODOs
  }
});

Docs

For more info see the unit tests and API doc comments.

[changeset-bot]

🦋 Changeset detected

Latest commit: 680d8a2

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 7 packages

Name	Type
@powersync/common	Minor
@powersync/node	Minor
@powersync/react-native	Minor
@powersync/web	Minor
@powersync/op-sqlite	Patch
@powersync/tanstack-react-query	Patch
@powersync/diagnostics-app	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR