Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache Prefilling Support #35

Open
MisterGC opened this issue Sep 27, 2023 · 0 comments
Open

Cache Prefilling Support #35

MisterGC opened this issue Sep 27, 2023 · 0 comments

Comments

@MisterGC
Copy link
Contributor

MisterGC commented Sep 27, 2023

Background:
While the current lazy-loading approach to cache filling is effective in many situations, it falls short when dealing with multiple or slow data sources. Users often know the specific spatial extent they wish to analyze and would benefit from a cache prefilling phase, even if it takes time to complete. This also allows developers to initially focus on optimizing the interaction between mapget and client applications like a map viewer, before turning their attention to optimizing the entire data pipeline, including data source interactions.

Goal:
Implement cache prefilling in mapget to enhance user experience by making data rapidly available post-prefill.

Acceptance Criteria:

  1. Spatial Extend Support: mapget should support 'populate' requests that specify a spatial extent for prefilling.
  2. Cancellation: Users should be able to cancel populate requests if needed.
  3. Configurable In-Memory Cache: The existing in-memory cache should be easily configurable in terms of its capacity, allowing users to tailor it to the volume of data they intend to explore.
  4. Progress Indication: A way to track the prefilling phase's progress is essential, ideally with an estimated time-to-completion indicator.
  5. Future-Proofing: The implementation should be designed to be compatible with other caching mechanisms that may be introduced later.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant