Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide means for accessing reference count without actually fetching the entities #650

Open
novoj opened this issue Aug 20, 2024 · 1 comment
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@novoj
Copy link
Collaborator

novoj commented Aug 20, 2024

After recent discussion with FE team using evitaDB, there may be use case were we want to fetch entity that has existing referenced entities of some type. On top of that however, we want to fetch count of these referenced entities without actually fetching the entities.
Something like that:

{
  listGroup(
    filterBy: {
      attributeCodeEquals: "news-group"
      referenceProductsHaving: {}
    }
  ) {
    productsCount # generated from reference `products`
  }
}

I think it could be solved also with facet summary, probably like this:

{
  queryGroup(
    filterBy: {
      attributeCodeEquals: "news-group"
      referenceProductsHaving: {}
    }
  ) {
    extraResults {
      facetSummary {
        products {
          count
        }
      }
    }
  }
}

But that's quite cumbersome to use on FE. @novoj do you think it would be valid to support the first approach at the GraphQL API level? We could also reuse the filterBy clause from the reference fields. On the backend it could be translated to basic referenceContent.


I think we could automatically provide the count attribute on EntityDecorator level. evitaDB internally always needs to compute reference primary keys to calculate the count, so we could always provide the array of these primary keys (since they have to be fetched anyway). But on the API level the primary keys can easily be thrown away and only the count could be provided to the external clients, thus saving some transport/networking costs.

@novoj novoj added the enhancement New feature or request label Aug 20, 2024
@novoj novoj added this to the Beta milestone Aug 20, 2024
@novoj novoj self-assigned this Aug 20, 2024
@novoj
Copy link
Collaborator Author

novoj commented Aug 21, 2024

This feature request partially opens up a path to a larger extension of the reference fetching enhancements. It would be also beneficial to be able to fetch only chunks of references. After discussion with @lukashornych we'd like to:

  1. be able to avoid sending all the referenced PKs over the wire in all of the protocols (so it should be somehow reflected in the base query language)
  2. be able to paginate the fetched references
  3. combine both above requirements with ordering and filtering constraints

In order to do so, this language extension makes sense to us:

// returns only counts for product reference (no PKs or bodies are sent)
referenceCount('products')
// returns only counts for all entity references (no PKs or bodies are sent)
referenceCount()
// returns only counts for product reference that match filtering criteria in a custom order (no PKs or bodies are sent)
referenceCount('products', filterBy(...), orderBy(...))
// returns first page of 20 primary keys that match filtering criteria in a custom order
referenceContent('products', filterBy(...), orderBy(...), page(1, 20))
// returns first page of 20 referenced entities that match filtering criteria in a custom order
referenceContent('products', filterBy(...), orderBy(...), entityFetchAll(), page(1, 20))

We still need to analyze how this proposal affects the internal structure of EntityDecorator and the connected APIs. This change would also affect all the APIs at once and would cover all our requirements.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant