Add test with graphql and sqlite storage

[mapineaultvooban]

this is a copy of pull request (#6782). This pull request uses RxDB 16 instead of 15.39. If possible, we would like the fix to be in 15.39.

This PR contains:

A unit test that executes a migration on a database with a sqlite storage. Then a graphql server sends an update with leaked property (assumedMasterState contains the _attachments property). I wrote some comments in the test, because I don't have access to the premium package (RxStorageSQLite) in this repository.

When we use a different storage (e.g in memory or another default storage in the public package), the test passed.

Describe the problem you have without this PR

In our project, we use a RxDatabase with

• a Sqlite storage (RxStorageSQLite from the premium package)
• a graphQL server to push updates to our backend

In one of our schemas, we add a new migration. when executed, it seems that the _attachments property is followed. When a user updates a property in an entity, the graphQL server push the update to our backend, causing an error in the serialization (input -> classes).

Here is a payload of one of our shemas with the problem. In the entity's masterState, we can see that the _attachments property is leaked.

{
  "query": "mutation PushContractAgreements($pushRows: [PushRowContractAgreementInput!]!) {
    pushContractAgreements(contractAgreements: $pushRows) {
        id
        status
        date
        number
        lunchBreaks {
          from
          durationInMinutes
          startsOnNextDay
        }
        signature
        signedAt
        updatedAt
        isDeleted
    }
  }",
  "operationName": "PushContractAgreements",
  "variables": {
    "pushRows": [
      {
        "assumedMasterState": {
          "id": "c3fdbc58-4588-4fe4-a3b4-e0e5467e1d82",
          "status": "created",
          "date": "2025-01-23T12:00:00.000Z",
          "number": null,
          "lunchBreaks": [],
          "signature": null,
          "signedAt": null,
          "updatedAt": 1737640220258,
          "_attachments": {},
          "isDeleted": false
        },
        "newDocumentState": {
          "id": "c3fdbc58-4588-4fe4-a3b4-e0e5467e1d82",
          "status": "ongoing",
          "date": "2025-01-23T12:00:00.000Z",
          "number": null,
          "lunchBreaks": [],
          "signature": null,
          "signedAt": null,
          "updatedAt": 1737647944354,
          "isDeleted": false
        }
      }
    ]
  }
}

Here is our database setup

db = await createRxDatabase<RxGuayCollections>({
     eventReduce: true,
     multiInstance: false,
     name: dbName,
     password: env.DATABASE_PASSWORD,
     storage: getRxStorageSQLite({
       sqliteBasics: getSQLiteBasicsExpoSQLiteAsync(ExpoSqliteStorageNext.openDatabaseAsync),
       // log: console.log.bind(console),
     }),
     ignoreDuplicate: isTest,
     cleanupPolicy: {
       minimumDeletedTime: 1000 * 60 * 60 * 24 * 31, // one month,
       minimumCollectionAge: 1000 * 60, // 60 seconds
       runEach: 1000 * 60 * 5, // 5 minutes
       awaitReplicationsInSync: true,
       waitForLeadership: false,
     },
     allowSlowCount: true,
   });

db.addCollections({
     contract_agreement: {
       autoMigrate: true,
       migrationStrategies: {
         1: (doc: never): never => doc,
       },
       schema: {
         description: 'describes a contract agreement',
         keyCompression: true,
         primaryKey: 'id',
         indexes: ['updatedAt'],
         properties: {
           id: {
             maxLength: 100,
             type: 'string', // <- the primary key must have set maxLength
           },
           isDeleted: {
             type: 'boolean',
           },
           lunchBreaks: {
             items: {
               description: 'describes a time range',
               properties: {
                 from: {
                   type: 'string',
                 },
                 durationInMinutes: {
                   type: 'number',
                 },
                 startsOnNextDay: {
                   type: 'boolean',
                 },
               },
               type: 'object',
             },
             type: 'array',
           },
           number: {
             type: ['null', 'number'],
           },
           signature: {
             type: ['string', 'null'],
           },
           signedAt: {
             type: ['string', 'null'],
           },
           status: {
             type: 'string',
           },
           updatedAt: {
             type: 'number',
             minimum: 0,
             maximum: 99999999999999,
             multipleOf: 1,
           },
         },
         required: ['id', 'status'],
         title: 'Contract agreement schema',
         type: 'object',
         version: 1,
       },
     },
   });

Todos

• Tests
• Documentation
• Typings
• Changelog

[pubkey]

Hi @mapineaultvooban
I think your test is not testing correctly. It fails for memory storage and everything non-dexie. When I add logs, it seems not to fail because of leaking the _attachments:

console.log('in the _attachments assert:');
console.dir(firstRow);

Shouldnt this test be green for the memory storage?

[pubkey]

Ok fixed this by adding await replicationState.awaitInSync(); after the update.

[pubkey]

@mapineaultvooban I ran the exact same test with the SQLite storage and it does not fail.
Not sure how to proceed.

[jwallet]

Hello Daniel,

We are running our react-native app with expo using the newest sdk50 and soon 52 which is using now expo/sqlite/next in 52 sqlite/legacy is removed. Do you have a storage CI test on expo sqlite package? maybe the issue comes from that lib.

EDIT: I will try to debug the sqlite wrapper on my spare time and find where the _attachments is added, cause the issue never happens if we do not run a migration, and it does not happen with a schema with flatten properties (no reference to another schema)

[pubkey]

Does it only fail with Expo SQLite or also with other SQLite libs like in node.js?