Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redesigning wrapper to correctly run from pipeline files. #16

Closed
jlvdb opened this issue Jul 10, 2024 · 0 comments · Fixed by #18
Closed

Redesigning wrapper to correctly run from pipeline files. #16

jlvdb opened this issue Jul 10, 2024 · 0 comments · Fixed by #18
Labels
bug Something isn't working

Comments

@jlvdb
Copy link
Collaborator

jlvdb commented Jul 10, 2024

In the current implementation, the YAW wrapper cannot be run as a ceci pipeline because the order in which stages are run is not deterministic.

Problems

  1. Ceci may decide to run the YawCacheDrop stage before the correlation measurements have even started.
  2. When a new data set is cached and is supposed to inherit the patch centers from a different data set, the caching of these data sets is not guaranteed to happen in the correct order.

Solutions

  1. Remove the YawCacheDrop stage entirely and leave it to the user to clean up the temporarily cached data, as they may be reused if neither the catalog nor the patch centers change.
  2. Add an additional (optional) link for YawCacheCreate instances to share patch centers and ensure execution in the correct order.
@jlvdb jlvdb added the bug Something isn't working label Jul 10, 2024
@jlvdb jlvdb linked a pull request Jul 10, 2024 that will close this issue
4 tasks
@jlvdb jlvdb mentioned this issue Jul 10, 2024
4 tasks
@jlvdb jlvdb removed a link to a pull request Jul 10, 2024
4 tasks
@jlvdb jlvdb linked a pull request Jul 10, 2024 that will close this issue
4 tasks
@jlvdb jlvdb closed this as completed in #18 Jul 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant