Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support yjs in dexie-cloud-addon #2045

Open
wants to merge 78 commits into
base: master
Choose a base branch
from
Open

Commits on Jun 10, 2024

  1. Configuration menu
    Copy the full SHA
    b56cee3 View commit details
    Browse the repository at this point in the history

Commits on Jun 19, 2024

  1. Configuration menu
    Copy the full SHA
    24ac71f View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    4cd9ac1 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    0cce18f View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    b2e86b0 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    ad6c737 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    6742cfa View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    60e83bc View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    1c4b5b0 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    0e069f1 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    22d79b5 View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2024

  1. Comment

    dfahlander committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    7742ebb View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    1200a19 View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2024

  1. Configuration menu
    Copy the full SHA
    60aef01 View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2024

  1. Configuration menu
    Copy the full SHA
    27e13e4 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2024

  1. GC implemented and tested.

    Still need to think it through. Might need to find a better way than iterating all updates in entire DB periodically.
    dfahlander committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    a3ef4b1 View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2024

  1. Finally a working flow

    dfahlander committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    3e32413 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    16a04d0 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    5cb596a View commit details
    Browse the repository at this point in the history

Commits on Jul 28, 2024

  1. Configuration menu
    Copy the full SHA
    8995d85 View commit details
    Browse the repository at this point in the history
  2. Update dts-bundle-generator.

    Could not take latest as it generates double types. Found out we needed version 9.3.1 exactly.
    dfahlander committed Jul 28, 2024
    Configuration menu
    Copy the full SHA
    31ac76e View commit details
    Browse the repository at this point in the history
  3. Refactored Y.js support:

    * Renamed updTable to updatesTable
    * Renamed id / rowId to parentId
    dfahlander committed Jul 28, 2024
    Configuration menu
    Copy the full SHA
    4ceb588 View commit details
    Browse the repository at this point in the history
  4. Updated test

    dfahlander committed Jul 28, 2024
    Configuration menu
    Copy the full SHA
    54ef0e7 View commit details
    Browse the repository at this point in the history
  5. Made it possible to reach the Y.Doc cache from addons or outside dexie:

    * To lookup whether a certain document is open and find it
    dfahlander committed Jul 28, 2024
    Configuration menu
    Copy the full SHA
    c013c51 View commit details
    Browse the repository at this point in the history
  6. Y.js support in dexie-cloud-addon:

    * updates
    * awareness
    * sync
    * websocket
    dfahlander committed Jul 28, 2024
    Configuration menu
    Copy the full SHA
    d9e3f76 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    d2cf903 View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2024

  1. Don't sync Y.Doc on tables not marked for sync.

    Move all yjs related modules to an yjs directory
    dfahlander committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    745c96f View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    659f7a1 View commit details
    Browse the repository at this point in the history
  3. Refactor: use table+prop instead of updatesTable because:

    * updatesTable is not meaningful outside of client
    * server need to know the parent table to verify access
    * parent table + yDoc prop combined forms the same uniqueness as updatesTable.
    dfahlander committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    aa38a76 View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2024

  1. Upgraded pnpm to 9

    David Fahlander committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    41e113a View commit details
    Browse the repository at this point in the history
  2. Upgrade pnpm in workflow

    David Fahlander committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    878be8c View commit details
    Browse the repository at this point in the history
  3. Bump version to alpha

    David Fahlander committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    c501020 View commit details
    Browse the repository at this point in the history

Commits on Aug 26, 2024

  1. Make pnpm not complain when dexie is in alpha or beta

    David Fahlander committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    7ec8ec5 View commit details
    Browse the repository at this point in the history
  2. useYProvider hook

    David Fahlander committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    2852116 View commit details
    Browse the repository at this point in the history
  3. Solving reference error.

    Referring to dexie-cloud-common a better way for pnpm: with workspace:^
    David Fahlander committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    a0299bb View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2024

  1. No need to double-destroy provider

    David Fahlander committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    5482ade View commit details
    Browse the repository at this point in the history
  2. whitespace change only

    David Fahlander committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    b5c6a24 View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2024

  1. Comment spelling

    David Fahlander committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    26bb73d View commit details
    Browse the repository at this point in the history
  2. Wrong call to updateTable.put() - it's inbound

    David Fahlander committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    cc37806 View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2024

  1. Add receivedUntil on YSyncState (formerly YSyncer) and respect it whe…

    …n compressing docs.
    David Fahlander committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    50cd0a2 View commit details
    Browse the repository at this point in the history
  2. Updated Y sync flow:

    * Update receivedUntil after getting server changes over syncWithServer() but not when getting chaotic server changes on a particular document over websocket.
    * Before syncWithServer, compute an Y.js state vector from the server changes that has arrived after receivedFrom so that the server can avoid sending the same changes back.
    * Update YSyncState after sync with the correct unsentFrom and receivedUntil values.
    David Fahlander committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    0591312 View commit details
    Browse the repository at this point in the history
  3. Bump version of dexie-cloud-common

    David Fahlander committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    ca17376 View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2024

  1. Refactor DexieYProvider and renamed

    useYProvider() back to useDocument()
    because the hook not only maintains a provider but also manages the lifetime of the document, keeping it loaded as long as the component is mounted.
    David Fahlander committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    75dae89 View commit details
    Browse the repository at this point in the history
  2. Complete client-side implementation of:

    * Opening a document on the server, providing a state vector of updates that we've got from the server over websocket but not through regular sync.
    * Handling doc-open re-requests in case WebSocket goes down and up again while the document is open in an editor.
    * Don't send the initial awareness state - wait for editor to manage awarenesses - except when the component is unmounted - then we remove our "client" from awareness state and also send a "doc-close" message
    David Fahlander committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    82be5ad View commit details
    Browse the repository at this point in the history
  3. Adjust to newer lib.dom.d.ts: IDBTransactionDurability seem to have b…

    …ecome standard.
    David Fahlander committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    6a83b4e View commit details
    Browse the repository at this point in the history
  4. Put back the YStateVector message (needed in normal sync)

    David Fahlander committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    91b620b View commit details
    Browse the repository at this point in the history
  5. Bump version of dexie-cloud-common

    David Fahlander committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    2324e56 View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2024

  1. Send Yjs updates over binary websocket channel.

    Store serverRevision on sync state for Y-tables in order to include it in the DocOpen request so that the client can get additional updates since last sync over the websocket immediately when the document is opened.
    David Fahlander committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    5c05775 View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2024

  1. Utilities for consuming binary stream from fetch

    David Fahlander committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    02b9c09 View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2024

  1. async binary stream producer

    David Fahlander committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    9f39feb View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2024

  1. Last piece in place for a completed YJS support in dexie-cloud-addon:

    * After login, or when user is accepted to a new realm, all Y docs in the user's realms are downloaded in full over a binary stream from new endpoint in dexie-cloud-server with the ability to continue where it left off in case network would break in the middle.
    * This is nescessary because clients only subscribe to updates on their current realms. When new access is given to user, it cannot just get the latest updates from last revision but need to fully download all the docs on the new realm.
    * With non-YJS changes we are currently baking in the same flow in the ordinary sync, but for YJS we can safely do the downloading outside the normal sync API as the changes are omnidempotent and the CRDT types does not depend on an exact revision being in place.
    David Fahlander committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    d417e2d View commit details
    Browse the repository at this point in the history
  2. Bugfix by review: clearout collectedDocs after bulkAdd()

    David Fahlander committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    b818b3d View commit details
    Browse the repository at this point in the history
  3. Missing break

    David Fahlander committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    6d75a83 View commit details
    Browse the repository at this point in the history
  4. dexie-cloud-addon version bump

    David Fahlander committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    76b4dc9 View commit details
    Browse the repository at this point in the history

Commits on Oct 9, 2024

  1. Bugfixes after testing:

    * Downloading of y documents from dexie-cloud didn't work properly
    * Didn't create dexie-cloud-syncer on updates table until after a long time
    * Garbage collecting yDocs didn't work properly when dexie-cloud addon had been syncing.
    David Fahlander committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    f38451e View commit details
    Browse the repository at this point in the history
  2. Fix nullish check

    David Fahlander committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    01ca6ea View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2024

  1. Configuration menu
    Copy the full SHA
    41332bf View commit details
    Browse the repository at this point in the history
  2. Respect cloud option disableEagerSync in regards to avoid pushing Y m…

    …essages to server
    David Fahlander committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    0a10bd9 View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2024

  1. Version bump on 3 libraries: dexie, dexie-cloud-addon and dexie-react…

    …-hooks
    
    Setting the three of them on the same alpha version
    David Fahlander committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    13116a5 View commit details
    Browse the repository at this point in the history

Commits on Oct 13, 2024

  1. Support providing initial Y.Doc when adding new rows

    + Clone state of provided Y.Doc into updates table
    + strip away physical Y.Doc property before storing to DB
    + throw when trying to replace an Y.Doc property on an existing object.
    + runtime type check of yDoc props.
    + new unit tests of the above
    David Fahlander committed Oct 13, 2024
    Configuration menu
    Copy the full SHA
    92ba86a View commit details
    Browse the repository at this point in the history
  2. Version bump

    David Fahlander committed Oct 13, 2024
    Configuration menu
    Copy the full SHA
    45669c1 View commit details
    Browse the repository at this point in the history
  3. Don't GC every 10 seconds. Instead:

    * Run it 10 seconds after Dexie instance created
    * Then run it every 5 minutes.
    * And don't update the timestamp lastCompressed unless anything actually could be compressed. Because otherwise we might starve the WAL log by writing every 5 minutes despite nothing is happening from the user's perspective.
    David Fahlander committed Oct 13, 2024
    Configuration menu
    Copy the full SHA
    3357ccb View commit details
    Browse the repository at this point in the history
  4. Version bump

    David Fahlander committed Oct 13, 2024
    Configuration menu
    Copy the full SHA
    17e9c1b View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2024

  1. Fix: MissingAPIError("awarenessProtocol was not provided...") when Yj…

    …s integration isn't used.
    David Fahlander committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    3166136 View commit details
    Browse the repository at this point in the history
  2. Make useDocument() not throw with non-dexie Y.Doc instances.

    David Fahlander committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    4dbcc43 View commit details
    Browse the repository at this point in the history
  3. Version bump

    David Fahlander committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    44e392f View commit details
    Browse the repository at this point in the history
  4. Version bump

    David Fahlander committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    5164b0d View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2024

  1. Bugfix for DexieYProvider.release() - it unconditionally destroyed do…

    …cs immediately
    David Fahlander committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    e328321 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    ccd1c2b View commit details
    Browse the repository at this point in the history
  3. Debug printouts for Y messages

    David Fahlander committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    e80b7dd View commit details
    Browse the repository at this point in the history
  4. Bugfix: some YJS updates were never sent to server

    David Fahlander committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    92542c7 View commit details
    Browse the repository at this point in the history
  5. Don't open docs until awarenss is requested.

    David Fahlander committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    addac96 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    db33099 View commit details
    Browse the repository at this point in the history
  7. When rejecting an update, reject all following updates as well from t…

    …he same client, and destroy doc if it is open.
    David Fahlander committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    eefba6f View commit details
    Browse the repository at this point in the history
  8. Feature: defineYDocTrigger(). Should be moved to another addon, but t…

    …rying it out first like this.
    David Fahlander committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    7dd40eb View commit details
    Browse the repository at this point in the history
  9. Version bumps

    David Fahlander committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    d8f9263 View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2024

  1. Dexie Cloud: Support for unsynced properties (local-only properties t…

    …hat are never synced to server)
    David Fahlander committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    da98d8f View commit details
    Browse the repository at this point in the history
  2. Version bump

    David Fahlander committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    582530a View commit details
    Browse the repository at this point in the history