Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support long-running queries like pageInsights with subscriptions or @stream #10956

Open
mattkrick opened this issue Mar 5, 2025 · 0 comments
Labels

Comments

@mattkrick
Copy link
Member

pageInsights can take a LONG time, longer than 30 seconds.
we have options:

  • use the openAI streaming API and stream chunks (LLMs use SSE, we could use that or websockets)
  • persist the answer to a DB & have the client re-request it after timeoute
  • send intermittent messages saying it's coming soon & then send the final result via subscription

the best option is probably the first because that's what people are used to.
to do that, we could use graphql subscriptions, or the new @stream directive. The latter is the natural choice, because that means we wouldn't have to change our schema based on the whether or not we want the data to come in chunks. Unfortunately, @stream won't be ready until graphql-js v17. So, if we want to implement this now, we should use subscriptions since we've already got that pattern set up. If we can postpone, we should wait for GraphQL v17.

Sources:

@mattkrick mattkrick added the bug label Mar 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant