Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Prefetching Cross-Store References #28

Open
tmagrino opened this issue Sep 20, 2018 · 1 comment
Open

Support for Prefetching Cross-Store References #28

tmagrino opened this issue Sep 20, 2018 · 1 comment

Comments

@tmagrino
Copy link
Member

In Fabric's store, we use object groups as an attempt to bundle and prefetch related objects in batches for the worker. However, there is currently no support for prefetching objects pointed to on other stores in a similar fashion.

In the inlined-treaties branch, I wrote up a somewhat unsafe feature for supporting this through the notion of "associate" objects, which get prefetched from other stores when adding the referring object to a worker's cache. There's probably a safer general design worth considering for the master branch of Fabric to address this in general.

@tmagrino
Copy link
Member Author

tmagrino commented Sep 20, 2018

From discussion with @andrewcmyers over email and slack:

I see various options:

  1. the object has a special field that indicates other objects to prefetch. This can be set at object creation time and overridden later.
    A bit clunky and comes with significant storage overhead but allows the server to do things without unswizzling objects.
  2. the object has a method that returns objects to prefetch along with it. This gives maximum flexibility but has more cost at the server.
  3. there is a special operation that can be invoked by a transaction to prefetch a set of objects, perhaps incorporating a traversal to some depth as well.
  4. A class can mark some of its fields as things to be prefetched. This is the least flexible but it has no storage overhead.

Strategies # 1 and # 2 above could be special cases of # 4.

Tom: I provide a field that's an inlined set of oids that you can add to

For # 1, the class would specify that you should look at the field that # 1 defines to determine what to prefetch.

The advantage of prefetching strategy # 3 is that you don’t ask for things you don’t need. The disadvantage is that you can’t ask for things
you don’t know about.

I suspect there are really two different mechanisms at play here,
depending on whether you are fetching an object for the first time or fetching an object because you know you have an old version. In the first case, there is not much concern that you are refetching a
referenced object you already have. In the second case, you mostly need to know only about referenced objects that have been updated or objects that are newly referenced by the new version of the fetched object.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant