Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spike: consider how to solve caching of the measured DataFrame #128

Open
benedeki opened this issue Nov 20, 2023 · 1 comment
Open

Spike: consider how to solve caching of the measured DataFrame #128

benedeki opened this issue Nov 20, 2023 · 1 comment
Labels
enhancement New feature or request

Comments

@benedeki
Copy link
Contributor

benedeki commented Nov 20, 2023

Background

DataFrame which is measured might contain expensive operations.

Feature

Consider how to solve this. Maybe it' would be as easy as adding a call to .cache but maybe something more elaborate.
Also if it would be (just) .cache consider making it possible to switch off - globally, or for that one particular call.

@benedeki benedeki added the enhancement New feature or request label Nov 20, 2023
@jakipatryk
Copy link

My initial thinking is that it should be the app using Atum doing the cache/persist before saving checkpoint (?) in Atum. But if so it has to be stated explicitly somewhere, maybe even with some warning message if input DF in not persisted

@benedeki benedeki moved this from 🆕 To groom to 📋 Backlog in CPS small repos project Mar 11, 2024
@benedeki benedeki added this to the Atum release 0.2 milestone Mar 11, 2024
@benedeki benedeki removed this from the Atum release 0.2 milestone Oct 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: 📋 Backlog
Development

No branches or pull requests

2 participants