Can not use "conda install" in binder environment #173

nils-braun · 2020-11-03T14:00:55Z

It seems that to whatever reasons, it is not possible to install a package into the binder environment via conda.
I tried via the terminal or directly from within the jupyter notebook. I started binder by clicking one of the links from the dask-example page.

It does not really matter which package (I tried dask-sql, but also e.g. pandas, which is already installed). Both fail with:

Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are missing from the target environment:
  - nodejs==14
  - numpy==1.18

The installation seems to work with mamba, which makes me think it is some problem in the way conda resolves the packages - but I have no idea what is going wild her.

Now comes the "funny" part: If I explicitly install

conda install numpy=1.18 nodejs=14

it actually works, even though conda list already shows me them as installed. It will downgrade some packages:

The following packages will be UPDATED:

  certifi                          2020.6.20-py38h32f6830_0 --> 2020.6.20-py38h924ce5b_2

The following packages will be DOWNGRADED:

  jupyterlab                                     2.1.5-py_0 --> 2.1.0-py_1
  pandas                               1.0.5-py38hcb8c335_0 --> 1.0.0-py38hb3f55d8_0
  scikit-learn                        0.23.2-py38h5d63f67_2 --> 0.23.0-py38h3a94b23_0

After that, I can also install other packages...

The text was updated successfully, but these errors were encountered:

mrocklin · 2020-11-03T15:16:20Z

cc'ing conda/dask experts @jakirkham @mariusvniekerk in case they have any insight

quasiben · 2020-11-05T14:08:40Z

Can you get the conda version being used in the binder example ? If it's less than 4.9 we should probably upgrade it

nils-braun · 2020-11-05T20:35:29Z

Thanks for your comment @quasiben. I just checked: The currently running image includes conda in version 4.8.2. If 4.9 will really help, that could be an issue.
Where is the conda version specified? Isn't this part of the mybinder installation process where we do not have a large number of configuration possibilities?

quasiben · 2020-11-05T21:33:27Z

No that's probably not it then. I launched an instanced on binder and things pleasantly worked with mamba. i would suggest either using mamba or pip, or, why not add directly to the env.yml file ? Any reason not to do this @mrocklin ?

nils-braun · 2020-11-05T22:35:31Z

Yes, image size. The package I would like to add (dask-sql) adds another approx 400 MB, which is quite a lot.

mamba is definitely a possibility, but as it should also showcase on how to install the package and mamba is not as known as conda, I am a bit hesitant. But sure, if it works I am fine with it.

jsignell · 2020-11-06T14:31:46Z

I'm wondering if this is a channel issue. It seems that both nodejs==14 and numpy==1.18 are in the env: https://github.com/dask/dask-examples/blob/master/binder/environment.yml, but they are coming from conda-forge. Does your conda install work if you specify the channel?

quasiben · 2020-11-06T14:50:38Z

@jsignell I tried your suggestion but that also doesn't work. Also it does work with Mamba:

jovyan@jupyter-nils-2dbraun-2ddask-2dexamples-2dajduqqec:~$ mamba install -c conda-forge dask-sql

                  __    __    __    __
                 /  \  /  \  /  \  /  \
                /    \/    \/    \/    \
███████████████/  /██/  /██/  /██/  /████████████████████████
              /  / \   / \   / \   / \  \____
             /  /   \_/   \_/   \_/   \    o \__,
            / _/                       \_____/  `
            |/
        ███╗   ███╗ █████╗ ███╗   ███╗██████╗  █████╗
        ████╗ ████║██╔══██╗████╗ ████║██╔══██╗██╔══██╗
        ██╔████╔██║███████║██╔████╔██║██████╔╝███████║
        ██║╚██╔╝██║██╔══██║██║╚██╔╝██║██╔══██╗██╔══██║
        ██║ ╚═╝ ██║██║  ██║██║ ╚═╝ ██║██████╔╝██║  ██║
        ╚═╝     ╚═╝╚═╝  ╚═╝╚═╝     ╚═╝╚═════╝ ╚═╝  ╚═╝

        mamba (0.6.1) supported by @QuantStack

        GitHub:  https://github.com/mamba-org/mamba
        Twitter: https://twitter.com/QuantStack

█████████████████████████████████████████████████████████████

conda-forge/linux-64     Using cache
conda-forge/noarch       Using cache
pkgs/main/noarch         [====================] (00m:00s) No change
pkgs/r/linux-64          [====================] (00m:00s) No change
pkgs/r/noarch            [====================] (00m:00s) No change
pkgs/main/linux-64       [====================] (00m:00s) No change

Looking for: ['dask-sql']

Transaction

  Prefix: /srv/conda/envs/notebook

  Updating specs:

   - dask-sql


  Package              Version  Build           Channel                    Size
─────────────────────────────────────────────────────────────────────────────────
  Install:
─────────────────────────────────────────────────────────────────────────────────

  aiofiles               0.6.0  pyhd8ed1ab_0    conda-forge/noarch        14 KB
  alsa-lib               1.2.3  h516909a_0      conda-forge/linux-64     560 KB
  aniso8601              7.0.0  py_0            conda-forge/noarch        31 KB
  dask-sql               0.1.2  py_0            conda-forge/noarch        18 MB
  fastapi               0.61.1  py_0            conda-forge/noarch        40 KB
  graphene               2.1.8  py_0            conda-forge/noarch        39 KB
  graphql-core           2.3.2  pyh9f0ad1d_0    conda-forge/noarch        89 KB
  graphql-relay          2.0.1  py_0            conda-forge/noarch        18 KB
  h11                   0.11.0  pyh9f0ad1d_0    conda-forge/noarch        44 KB
  httptools              0.1.1  py38h1e0a361_1  conda-forge/linux-64     102 KB
  itsdangerous           1.1.0  py_0            conda-forge/noarch        16 KB
  jpype1                 1.1.2  py38hbf85e49_0  conda-forge/linux-64     452 KB
  openjdk               11.0.8  hacce0ff_0      conda-forge/linux-64     173 MB
  promise                  2.3  py38h32f6830_2  conda-forge/linux-64      35 KB
  pydantic               1.7.2  py38h25fe258_0  conda-forge/linux-64       2 MB
  python-dotenv         0.15.0  pyhd8ed1ab_0    conda-forge/noarch        20 KB
  python-multipart       0.0.5  py_0            conda-forge/noarch        28 KB
  rx                     1.6.1  py_0            conda-forge/noarch        74 KB
  starlette             0.13.6  py_0            conda-forge/noarch         4 KB
  starlette-base        0.13.6  py_0            conda-forge/noarch        45 KB
  ujson                  4.0.1  py38h950e882_1  conda-forge/linux-64      48 KB
  uvicorn               0.12.2  py38h32f6830_0  conda-forge/linux-64      67 KB
  uvloop                0.14.0  py38h1e0a361_2  conda-forge/linux-64       1 MB
  watchgod                 0.6  py_0            conda-forge/noarch        13 KB
  websockets               8.1  py38h1e0a361_2  conda-forge/linux-64      90 KB
  xorg-fixesproto          5.0  h14c3975_1002   conda-forge/linux-64       8 KB
  xorg-inputproto        2.3.2  h14c3975_1002   conda-forge/linux-64      18 KB
  xorg-libxfixes         5.0.3  h516909a_1004   conda-forge/linux-64      17 KB
  xorg-libxi            1.7.10  h516909a_0      conda-forge/linux-64      45 KB
  xorg-libxtst           1.2.3  h516909a_1002   conda-forge/linux-64      31 KB
  xorg-recordproto      1.14.2  h516909a_1002   conda-forge/linux-64       7 KB

  Change:
─────────────────────────────────────────────────────────────────────────────────

  certifi            2020.6.20  py38h32f6830_0  installed                      
  certifi            2020.6.20  py38h924ce5b_2  conda-forge/linux-64     151 KB

  Upgrade:
─────────────────────────────────────────────────────────────────────────────────

  ca-certificates    2020.6.20  hecda079_0      installed                      
  ca-certificates   2020.10.14  0               pkgs/main/linux-64       Cached

  Summary:

  Install: 31 packages
  Change: 1 packages
  Upgrade: 1 packages

  Total download: 197 MB

─────────────────────────────────────────────────────────────────────────────────

Confirm changes: [Y/n]

I also tried with:

conda install -c conda-forge dask-sql --no-update-deps

And received the same PackagesNotFoundError error

jsignell · 2020-11-06T14:57:39Z

Yeah I tried it too :( I feel like there must be some conda settings somewhere, but I can't find them

jsignell · 2020-11-06T15:03:32Z

I feel like there must be some conda settings somewhere, but I can't find them

Nah that's not it either. I just checked them and conda-forge is in the default channels list, so adding the channel makes no difference.

jacobtomlinson · 2021-10-14T15:02:28Z

It's been a while since anyone was here but I tried this again today and can confirm that things are still the same. Conda fails but mamba succeeds to install.

saulshanabrook · 2022-02-02T20:57:00Z

Same for me... So odd!

saulshanabrook · 2022-02-02T21:10:42Z

Locally it fails for me with:

Collecting package metadata (repodata.json): done
Solving environment: failed

ResolvePackageNotFound: 
  - bokeh=2.1.1
  - nodejs=14

It does seem to be building in the github actions with the environment.yml, but it does fail for me locally on mac with an arm chip.

I believe my issue is that these packages, nodejs=14 and bokeh=2.1.1 have no builds on conda forge for an arm mac.

If I upgrade them both to a version that does have those builds, then it can create it locally. I will create a PR that at least updates these two pins minimally, so I can install it locally.

nils-braun changed the title ~~Can not use "conda install in binder environment"~~ Can not use "conda install" in binder environment Nov 3, 2020

nils-braun mentioned this issue Nov 3, 2020

Added an example notebook on dask-sql. #171

Closed

jacobtomlinson added bug Something isn't working help wanted Extra attention is needed labels Oct 14, 2021

saulshanabrook mentioned this issue Feb 2, 2022

Minimal update of deps to allow installing on ARM Mac #207

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can not use "conda install" in binder environment #173

Can not use "conda install" in binder environment #173

nils-braun commented Nov 3, 2020

mrocklin commented Nov 3, 2020

quasiben commented Nov 5, 2020

nils-braun commented Nov 5, 2020

quasiben commented Nov 5, 2020

nils-braun commented Nov 5, 2020

jsignell commented Nov 6, 2020

quasiben commented Nov 6, 2020

jsignell commented Nov 6, 2020

jsignell commented Nov 6, 2020

jacobtomlinson commented Oct 14, 2021

saulshanabrook commented Feb 2, 2022

saulshanabrook commented Feb 2, 2022

Can not use "conda install" in binder environment #173

Can not use "conda install" in binder environment #173

Comments

nils-braun commented Nov 3, 2020

mrocklin commented Nov 3, 2020

quasiben commented Nov 5, 2020

nils-braun commented Nov 5, 2020

quasiben commented Nov 5, 2020

nils-braun commented Nov 5, 2020

jsignell commented Nov 6, 2020

quasiben commented Nov 6, 2020

jsignell commented Nov 6, 2020

jsignell commented Nov 6, 2020

jacobtomlinson commented Oct 14, 2021

saulshanabrook commented Feb 2, 2022

saulshanabrook commented Feb 2, 2022