Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TypeError: AioSession.__init__() got an unexpected keyword argument '//' #22

Open
will-moore opened this issue Aug 6, 2024 · 7 comments
Open

TypeError: AioSession.__init__() got an unexpected keyword argument '//' #22

will-moore opened this issue Aug 6, 2024 · 7 comments

Comments

@will-moore
Copy link
Member

On my local Mac, I created a new conda env and installed just ome2024-ngff-challenge, then tried converting the remote sample from https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.4/idr0062A/6001240.zarr as described in the README...

(NB: I know this is working for Dom on our pilot-zarr3-dev server with env from https://github.com/dominikl/nf_ngff_conv/blob/main/bf2raw_env.yml but trying to use the env.yml on my local Mac took too long)

$ conda create -n ngff2024 python=3.12
$ conda activate ngff2024

$ pip install -e .

Successfully installed CacheControl-0.14.0 MarkupSafe-2.1.5 aiobotocore-2.13.1 aiohappyeyeballs-2.3.4 aiohttp-3.10.1 aioitertools-0.11.0 aiosignal-1.3.1 arcp-0.2.1 asciitree-0.3.3 attrs-24.1.0 bioblend-1.3.0 botocore-1.34.131 certifi-2024.7.4 charset-normalizer-3.3.2 click-8.1.7 crc32c-2.4.1 donfig-0.8.1.post1 fasteners-0.19 filelock-3.15.4 frozenlist-1.4.1 fsspec-2024.6.1 galaxy2cwl-0.1.4 gxformat2-0.19.0 idna-3.7 iniconfig-2.0.0 isodate-0.6.1 jinja2-3.1.4 jmespath-1.0.1 mistune-3.0.2 ml-dtypes-0.4.0 msgpack-1.0.8 multidict-6.0.5 mypy-extensions-1.0.0 numcodecs-0.13.0 numpy-2.0.1 ome2024-ngff-challenge-0.0.5.post5.dev0+1674057 packaging-24.1 pluggy-1.5.0 pyparsing-3.1.2 pytest-8.3.2 python-dateutil-2.9.0.post0 pyyaml-6.0.1 rdflib-7.0.0 requests-2.32.3 requests-toolbelt-1.0.0 rocrate-0.10.0 ruamel.yaml-0.18.6 ruamel.yaml.clib-0.2.8 s3fs-2024.6.1 schema-salad-8.7.20240718183047 six-1.16.0 tensorstore-0.1.64 tinydb-4.8.0 tqdm-4.66.5 tuspy-1.0.3 typing-extensions-4.12.2 urllib3-2.2.2 wrapt-1.16.0 yarl-1.9.4 zarr-3.0.0a0 zstandard-0.23.0

$ ome2024-ngff-challenge \
>         --input-bucket=idr \
>         --input-endpoint=https://uk1s3.embassy.ebi.ac.uk \
>         --input-anon \
>         zarr/v0.4/idr0062A/6001240.zarr \
>         /tmp/6001240.zarr
Traceback (most recent call last):
  File "/Users/wmoore/opt/anaconda3/envs/ngff2024/bin/ome2024-ngff-challenge", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/Users/wmoore/Desktop/NGFF/ome2024-ngff-challenge/src/ome2024_ngff_challenge/resave.py", line 724, in cli
    converted = main(ns, rocrate)
                ^^^^^^^^^^^^^^^^^
  File "/Users/wmoore/Desktop/NGFF/ome2024-ngff-challenge/src/ome2024_ngff_challenge/resave.py", line 545, in main
    input_config.open_group()
  File "/Users/wmoore/Desktop/NGFF/ome2024-ngff-challenge/src/ome2024_ngff_challenge/resave.py", line 251, in open_group
    self.zr_group = zarr.open_group(store=self.zr_store, zarr_format=2)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/wmoore/opt/anaconda3/envs/ngff2024/lib/python3.12/site-packages/zarr/api/synchronous.py", line 175, in open_group
    sync(
  File "/Users/wmoore/opt/anaconda3/envs/ngff2024/lib/python3.12/site-packages/zarr/sync.py", line 92, in sync
    raise return_result
  File "/Users/wmoore/opt/anaconda3/envs/ngff2024/lib/python3.12/site-packages/zarr/sync.py", line 51, in _runner
    return await coro
           ^^^^^^^^^^
  File "/Users/wmoore/opt/anaconda3/envs/ngff2024/lib/python3.12/site-packages/zarr/api/asynchronous.py", line 523, in open_group
    return await AsyncGroup.open(store_path, zarr_format=zarr_format)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/wmoore/opt/anaconda3/envs/ngff2024/lib/python3.12/site-packages/zarr/group.py", line 144, in open
    zgroup_bytes, zattrs_bytes = await asyncio.gather(
                                 ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/wmoore/opt/anaconda3/envs/ngff2024/lib/python3.12/site-packages/zarr/store/core.py", line 35, in get
    return await self.store.get(self.path, prototype=prototype, byte_range=byte_range)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/wmoore/opt/anaconda3/envs/ngff2024/lib/python3.12/site-packages/zarr/store/remote.py", line 98, in get
    await (
  File "/Users/wmoore/opt/anaconda3/envs/ngff2024/lib/python3.12/site-packages/s3fs/core.py", line 1128, in _cat_file
    return await _error_wrapper(_call_and_read, retries=self.retries)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/wmoore/opt/anaconda3/envs/ngff2024/lib/python3.12/site-packages/s3fs/core.py", line 145, in _error_wrapper
    raise err
  File "/Users/wmoore/opt/anaconda3/envs/ngff2024/lib/python3.12/site-packages/s3fs/core.py", line 113, in _error_wrapper
    return await func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/wmoore/opt/anaconda3/envs/ngff2024/lib/python3.12/site-packages/s3fs/core.py", line 1115, in _call_and_read
    resp = await self._call_s3(
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/wmoore/opt/anaconda3/envs/ngff2024/lib/python3.12/site-packages/s3fs/core.py", line 358, in _call_s3
    await self.set_session()
  File "/Users/wmoore/opt/anaconda3/envs/ngff2024/lib/python3.12/site-packages/s3fs/core.py", line 519, in set_session
    self.session = aiobotocore.session.AioSession(**self.kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: AioSession.__init__() got an unexpected keyword argument '//'
@joshmoore
Copy link
Member

Hmmm.... I'm not sure @will-moore. With these versions:

aiobotocore==2.13.2
aiohappyeyeballs==2.4.0
aiohttp==3.10.5
aioitertools==0.11.0
aiosignal==1.3.1

things are working for me. (Fresh install just now in a new virtualenv) Even downgrading some of the libraries didn't show the error:

pip install -U aiobotocore==2.13.1 aiohttp==3.10.1

@will-moore
Copy link
Member Author

will-moore commented Aug 20, 2024 •
edited
Loading

On my python 3.10 env...

$ pip freeze | grep aio
aiobotocore==2.13.0
aiohttp==3.9.5
aioitertools==0.11.0
aiosignal==1.3.1

$ pip install -U "aiohttp==3.10.5"
$ pip install -U "aiobotocore==2.13.2"

$ pip freeze | grep aio   (now an exact match)
aiobotocore==2.13.2
aiohappyeyeballs==2.4.0
aiohttp==3.10.5
aioitertools==0.11.0
aiosignal==1.3.1


$ ome2024-ngff-challenge         --input-bucket=idr         --input-endpoint=https://uk1s3.embassy.ebi.ac.uk         --input-anon         zarr/v0.4/idr0062A/6001240.zarr         /tmp/6001240.zarr
Traceback (most recent call last):
  File "/Users/wmoore/opt/anaconda3/envs/zarr_v3/bin/ome2024-ngff-challenge", line 8, in <module>
    sys.exit(cli())
  File "/Users/wmoore/Desktop/NGFF/ome2024-ngff-challenge/src/ome2024_ngff_challenge/resave.py", line 866, in cli
    converted = main(ns, rocrate)
  File "/Users/wmoore/Desktop/NGFF/ome2024-ngff-challenge/src/ome2024_ngff_challenge/resave.py", line 666, in main
    input_config.open_group()
  File "/Users/wmoore/Desktop/NGFF/ome2024-ngff-challenge/src/ome2024_ngff_challenge/resave.py", line 300, in open_group
    self.zr_group = zarr.open_group(store=self.zr_store, zarr_format=2)
  File "/Users/wmoore/opt/anaconda3/envs/zarr_v3/lib/python3.10/site-packages/zarr/api/synchronous.py", line 175, in open_group
    sync(
  File "/Users/wmoore/opt/anaconda3/envs/zarr_v3/lib/python3.10/site-packages/zarr/sync.py", line 92, in sync
    raise return_result
  File "/Users/wmoore/opt/anaconda3/envs/zarr_v3/lib/python3.10/site-packages/zarr/sync.py", line 51, in _runner
    return await coro
  File "/Users/wmoore/opt/anaconda3/envs/zarr_v3/lib/python3.10/site-packages/zarr/api/asynchronous.py", line 523, in open_group
    return await AsyncGroup.open(store_path, zarr_format=zarr_format)
  File "/Users/wmoore/opt/anaconda3/envs/zarr_v3/lib/python3.10/site-packages/zarr/group.py", line 144, in open
    zgroup_bytes, zattrs_bytes = await asyncio.gather(
  File "/Users/wmoore/opt/anaconda3/envs/zarr_v3/lib/python3.10/site-packages/zarr/store/core.py", line 35, in get
    return await self.store.get(self.path, prototype=prototype, byte_range=byte_range)
  File "/Users/wmoore/opt/anaconda3/envs/zarr_v3/lib/python3.10/site-packages/zarr/store/remote.py", line 98, in get
    await (
  File "/Users/wmoore/opt/anaconda3/envs/zarr_v3/lib/python3.10/site-packages/s3fs/core.py", line 1128, in _cat_file
    return await _error_wrapper(_call_and_read, retries=self.retries)
  File "/Users/wmoore/opt/anaconda3/envs/zarr_v3/lib/python3.10/site-packages/s3fs/core.py", line 145, in _error_wrapper
    raise err
  File "/Users/wmoore/opt/anaconda3/envs/zarr_v3/lib/python3.10/site-packages/s3fs/core.py", line 113, in _error_wrapper
    return await func(*args, **kwargs)
  File "/Users/wmoore/opt/anaconda3/envs/zarr_v3/lib/python3.10/site-packages/s3fs/core.py", line 1115, in _call_and_read
    resp = await self._call_s3(
  File "/Users/wmoore/opt/anaconda3/envs/zarr_v3/lib/python3.10/site-packages/s3fs/core.py", line 358, in _call_s3
    await self.set_session()
  File "/Users/wmoore/opt/anaconda3/envs/zarr_v3/lib/python3.10/site-packages/s3fs/core.py", line 519, in set_session
    self.session = aiobotocore.session.AioSession(**self.kwargs)
TypeError: AioSession.__init__() got an unexpected keyword argument '//'

Tried the same on a different python 3.12 env, as above... but got the same fail!

@joshmoore
Copy link
Member

Do you have anything in an ~/.aws directory?

@will-moore
Copy link
Member Author

Hmm yes...

$ cat ~/.aws/config
[default]
region =  

$ cat ~/.aws/credentials 
[default]
aws_access_key_id = ***************
aws_secret_access_key = ****************

@joshmoore
Copy link
Member

I'd suggest moving that out of the way.

@will-moore
Copy link
Member Author

$ mv .aws aws_configs made no difference I'm afraid. But thanks for the suggestion.

@will-moore will-moore mentioned this issue Sep 17, 2024
Merged
6 tasks
@will-moore
Copy link
Member Author

Also seeing this at zarr-developers/zarr-python#2186 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@joshmoore @will-moore