Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP Presence #286

Closed
wants to merge 41 commits into from
Closed

WIP Presence #286

wants to merge 41 commits into from

Conversation

curran
Copy link
Contributor

@curran curran commented Apr 17, 2019

A continuation of #207.

While resolving merge conflicts, I encountered the following:

  • 4024 - OT Type does not support presence - I changed this to 4027, as the code 4024 was introduced to mean something else since this PR was submitted.
  • In Doc.prototype._hardRollback - This PR introduces a solution for not dropping errors, whereas a competing solution already existed (from [Update hard rollback callback logic](Update hard rollback callback logic) by @alecgibson ). I did my best to reconcile the two, favoring the one already presence in master.

gkubisa and others added 20 commits April 26, 2018 12:47
I had to add the --exit flag workaround to mocha.opts to make it exit
when tests are done. A better long-term solution would be to ensure
that nothing keeps node running after all tests are done, see
https://boneskull.com/mocha-v4-nears-release/#mochawontforceexit.
The problem was that unsubscribe re-added the doc to the connection.
Now the doc is removed from the connection after unsubscribe.
Additionally, we're no longer waiting for the unsubscribe response
before executing the callback. It is consistent with Query,
unsubscribe can't fail anyway and the subscribed state is updated
synchronously on the client side.
The issue could not cause problems in practice because
ot-json0 does not support presence.
@coveralls
Copy link

Coverage Status

Coverage decreased (-43.6%) to 52.274% when pulling 33450ae on curran:presence-continuation into abcea41 on share:master.

1 similar comment
@coveralls
Copy link

Coverage Status

Coverage decreased (-43.6%) to 52.274% when pulling 33450ae on curran:presence-continuation into abcea41 on share:master.

@coveralls
Copy link

coveralls commented Apr 17, 2019

Coverage Status

Coverage decreased (-29.9%) to 66.036% when pulling ea0854d on curran:presence-continuation into abcea41 on share:master.

@curran
Copy link
Contributor Author

curran commented Apr 17, 2019

The failing test in CI is expires cached ops, which appears to depend on 1ms time resolution of setTimeout, which makes it flaky. I will attempt to use lolex here.

@curran
Copy link
Contributor Author

curran commented Apr 17, 2019

I don't get the advantage of Object.create(null) over {} for this case, so I'm changing these to simplify. I'm curious @gkubisa , are there any big wins that we get from using Object.create(null) instead of {}? I understand there's a difference in the prototype, but that difference appears inconsequential. Probably there's no performance difference (if anything, I'd guess Object.create(null) would be slower due to the additional property lookup and function call).

@gkubisa
Copy link
Contributor

gkubisa commented Apr 17, 2019

If anything, I would change those to Map instances because that's how they are used. Object.create(null) is the closes thing to Map. The problem with using {} as a map is that some properties exist on the Object prototype, so to be 100% sure that an entry exists in the "map" you'd have to use hasOwnProperty(...).

I'm not sure about the performance of Object.create(null) vs {}, however, that should be inconsequential in this case. An object with a null prototype should be a bit faster when accessing non-existent properties because then the prototype does not need to be checked for that property.

@curran curran changed the title WIP Presence Presence Apr 17, 2019
@curran
Copy link
Contributor Author

curran commented Apr 17, 2019

Thanks @gkubisa for the clarification. Since these objects are often generated externally, and may need to survive JSON.stringify and JSON.parse anyway, it feels better IMO to use {} and live with the Object prototype.

The only further change I'm considering is to make the implementation robust to OT types that do not implement createPresence and comparePresence, leaving more flexibility in the definition of the spec (related issue: ottypes/docs: Presence for OT Types). At that point, the only hard requirement for OT types to support presence would be transformPresence.

The following could serve as defaults if these methods are not provided by OT types (from this json0 fork):

json.createPresence = function(presenceData) {
  return presenceData;
};

json.comparePresence = function(pres1, pres2) {
  return JSON.stringify(pres1) === JSON.stringify(pres2);
};

This could just as well be left as future work too.

Thoughts?

@curran
Copy link
Contributor Author

curran commented Apr 17, 2019

I moved presence logic into a new module, but this is only really used for Doc methods. It could also be made to house presence-related logic for Agent, Connection and Backend as well. Not sure whether or not this is advisable.

It could look something like this:

var Presence = require('./presence');
// in various different files...
Object.assign(Doc.prototype, Presence.docMethods);
Object.assign(Agent.prototype, Presence.agentMethods);
Object.assign(Connection.prototype, Presence.connectionMethods);
Object.assign(Backend.prototype, Presence.backendMethods);

I think I'll stop for now and await review feedback.

Kindly requesting review by @nateps @ericyhwang @gkubisa . Thanks in advance!

Copy link
Contributor Author

@curran curran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Notes from the PR review call with @nateps @ericyhwang @alecgibson :

  • Name presence.current could be better.
  • Maybe presence should be its own class? Now it's a bunch of functions that refer to this.presence. Maybe it should be more like a Presence class with a Document reference. Decorating with methods is sort of a code smell. The Document could create a Presence instance. It would be a lot more clear. this.doc. ... would be more explicit.
  • Doc is the biggest place to focus. Agent, Connection, Backend are not a big concern.
  • Instead of enablePresence, we could take something else, perhaps code. This could be similar to pubSub, where different implementations could be passed in. The implementations might be radically different. There could be a "dummy" default one that doesn't do anything.
  • Options.presence would be on the backend. That should have some effect on all connections using backend.createConnection. Might also be required to configure the Connection (client) as well.
  • Passing something in that implements an interface could reduce bundle size as well, if it's not included.
  • On the server we could either do what we do with PubSub - default implementation is simple, could pass in an alternative. Default could be "presence is not enabled", so app code could import the basic implementation.
  • That Presence class should be passed to the Connection constructor.
  • A separate Presence class should be passed into the Backend constructor.
  • These could live in a presence directory, sharedb/lib/presence - Server impl of presence. Pass this to Backend.
  • Also a client/presence.js - this can be passed to the Connection constructor.
  • Generally presence should be more loosely coupled.
  • Instance on the connection vs. doc.
  • Would be nice to be able to support the use case of multiple presences (be at multiple places within a document). This is possible now to do outside shareDB, in OT type.
  • Would also be nice to let application code specify the transformPresence function.
  • Could even be so decoupled where you could subscribe only to presence.
  • From Connection, could pass collection and doc ID.
  • Ideally Presence object/interface would not be coupled to a doc at all.

In summary (immediately actionable):

  • presence logic should not decorate Doc with methods, and should have minimal interactions/references with Doc internals.
  • Instead of a boolean flag, the presence class instance should be passed into Connection and Backend constructors.

@curran curran changed the title Presence WIP Presence Apr 18, 2019
@curran
Copy link
Contributor Author

curran commented Apr 18, 2019

Stuck on this failing test:

  1) client presence (wrapped-presence-no-compare)
       waits for pending ops before processing future presence:
     ShareDBError: Cannot submit op. Document has not been created. dogs.fido
      at Doc._submit (lib/client/doc.js:703:17)
      at Doc.submitOp (lib/client/doc.js:814:8)
      at Context.<anonymous> (test/client/presence.js:108:22)
      at exports.callEach (lib/util.js:31:7)
      at StatelessPresence.handlePresence (lib/presence/stateless.js:106:33)
      at Doc._handlePresence (lib/client/doc.js:373:17)
      at Connection.handleMessage (lib/client/connection.js:259:20)
      at StreamSocket.socket.onmessage (lib/client/connection.js:132:18)
      at /home/curran/repos/sharedb/lib/stream-socket.js:60:12
      at processTicksAndRejections (internal/process/next_tick.js:74:9)

Skipping that test, the next failing test may reveal the issue:

    - waits for pending ops before processing future presence
    ✓ handles presence sent for earlier revisions (own ops, presence.index < op.index)
    ✓ handles presence sent for earlier revisions (own ops, presence.index === op.index)
    ✓ handles presence sent for earlier revisions (own ops, presence.index > op.index)
    ✓ handles presence sent for earlier revisions (non-own ops, presence.index < op.index)
    1) handles presence sent for earlier revisions (non-own ops, presence.index < op.index)

  1) client presence (wrapped-presence-no-compare)
       handles presence sent for earlier revisions (non-own ops, presence.index < op.index):
     Uncaught Error: Callback was already called.
      at /home/curran/repos/sharedb/node_modules/async/lib/async.js:43:36
      at /home/curran/repos/sharedb/node_modules/async/lib/async.js:723:17
      at /home/curran/repos/sharedb/node_modules/async/lib/async.js:167:37
      at /home/curran/repos/sharedb/test/util.js:20:14
      at exports.callEach (lib/util.js:31:7)
      at /home/curran/repos/sharedb/lib/client/doc.js:981:33
      at Doc.ingestSnapshot (lib/client/doc.js:226:15)
      at Doc._handleFetch (lib/client/doc.js:278:8)
      at Connection.handleMessage (lib/client/connection.js:242:20)
      at StreamSocket.socket.onmessage (lib/client/connection.js:132:18)
      at /home/curran/repos/sharedb/lib/stream-socket.js:60:12
      at processTicksAndRejections (internal/process/next_tick.js:74:9)

Stopping for now.

@curran curran mentioned this pull request Apr 18, 2019
@curran
Copy link
Contributor Author

curran commented Apr 18, 2019

Closing in favor of #288

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants