Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Releasing plan for azarr #1

Open
oeway opened this issue Nov 17, 2022 · 5 comments
Open

Releasing plan for azarr #1

oeway opened this issue Nov 17, 2022 · 5 comments

Comments

@oeway
Copy link

oeway commented Nov 17, 2022

Hi @martindurant Just discovered this repo and it's looks very interesting for my project, thanks for sharing it! Now I am wondering is there a plan to integrate to zarr or release it as a standalone package so we can use it.

The thing I try to solve is that we have the imjoy-rpc library which uses asyncio to wrap remote functions, so we can obtain a zarr store object implemented remotely, however, since all the member functions (including getItem, __getitem__ etc. ) are async, we need something like azarr to be able to use the async functions. We can potentially use threading to wrap the async store as sync store, however, it won't work in pyodide.

@martindurant
Copy link
Owner

Thanks for the interest, and indeed for reminding me about this. The code here is pretty POC, and I wouldn't recommend it for any serious work. However, it would be nice to see it developed further into something that can be useful, particularly with pyodide/pyscript now a serious concern.

I don't think there's a great chance of zarr-python itself wanting to take control of and maintaining this code. I would be happy to push it out, "release" it, wherever; but I don't really have the time to put in much effort myself. Or, actually, a plan of how to make things more solid (beyond obvious things like writing lots of tests).

@joshmoore
Copy link

I don't think there's a great chance of zarr-python itself wanting to take control of and maintaining this code.

@martindurant: why's that?

@martindurant
Copy link
Owner

It would take some discussion to get this into a shape good enough to include in zarr main - but if you think it's a good idea, then we should make it happen. Having zarr b the first async-ready (and browser-ready) data library would be a thing.

Testing for both asyncio and pyodide would be a little complicated, but I suppose both can be done, in separate runs.

@joshmoore
Copy link

Would be most helpful in driving that discussion? A focused community call with some extra invites? A blog and request for feedback?

cc: @MSanKeys963

@MSanKeys963
Copy link

Hi, @martindurant and @joshmoore.

I think the idea of having async-zarr sounds intriguing to me. Discussing the idea in one of the community calls will be a good idea moving forward.
After the discussion, I'm happy to write a blog post, diving more into the features and codebase.

Let me know what you think.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants