Add determine-chunk-shape notebook #31
Conversation
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
| @@ -0,0 +1,665 @@ | |||
| { | |||
There was a problem hiding this comment.
Line #16. # we can set lon to 2 and lat to 1 since we expect twice as many chunks along the lon dimension as the lat dimension.
Maybe also mention what setting -1 does?
Reply via ReviewNB
| @@ -0,0 +1,665 @@ | |||
| { | |||
There was a problem hiding this comment.
100MB
If direct-to-frontend Zarr visualization ever happens, having such large chunks would be a blocker because the frontend would have to download a minimum of one chunk before anything displays.
Reply via ReviewNB
|
Thanks for reviewing @kylebarron - headsup I am planning to post this for feedback on pangeo discourse before merging. |
|
@abarciauskas-bgse do you still plan to merge this? |
|
@wildintellect I think lets close this without merging - this question is largely answered in #121 although we could break out some of the chunk size and shape guidance to a separate page that applies to all n-dimensional data formats (so Zarr, HDF5, NetCDF) |
This question comes up so often and I wanted to try out the new method in pangeo-forge/pangeo-forge-recipes#546 that I thought we might add some basic information on how to think about the question of chunk shape and size.