Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve handling of large (such as >1000) file orders #169

Closed
danielfromearth opened this issue Apr 26, 2024 · 3 comments
Closed

Improve handling of large (such as >1000) file orders #169

danielfromearth opened this issue Apr 26, 2024 · 3 comments
Assignees
Labels
enhancement New feature or request

Comments

@danielfromearth
Copy link
Collaborator

danielfromearth commented Apr 26, 2024

Perhaps by closing, or writing to file, the intermediate Dataset objects.

@danielfromearth danielfromearth added the enhancement New feature or request label Apr 26, 2024
@cheyenneland
Copy link

Closed xarray dataset object and netCDF dataset object using try, except and finally statements on stitchee.py and had a successful run on the CERES Ordering Tool (more than 3000 files). Will need to do more tests without the try, except and finally statements for both to see if one of them is not needed.

@cheyenneland
Copy link

Took out the edited out the try, except and finally statements for the ncfile and only left it for the xarray dataset object and had a successful test on the CERES Ordering Tool.

@cheyenneland cheyenneland linked a pull request May 15, 2024 that will close this issue
4 tasks
@danielfromearth danielfromearth linked a pull request May 15, 2024 that will close this issue
4 tasks
@danielfromearth
Copy link
Collaborator Author

Closing this issue, because it appears to no longer be needed for working with CERES data. The overarching bundling process splits jobs into manageable groups of files which can be processed by stitchee without such modifications.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
2 participants