Development install fails with package version conflict on shinyswatch

[mccalluc]

I'm trying to follow the developer instructions in the README on a fresh checkout with a clean virtualenv:

If you want to do development on Shiny for Python:

pip install -e ".[dev,test]"

I get the following error:

INFO: pip is looking at multiple versions of shinyswatch to determine which version is compatible with other requirements. This could take a while.
Collecting shinyswatch>=0.7.0 (from shiny==0.1.dev1856)
  Using cached shinyswatch-0.7.0-py3-none-any.whl.metadata (7.5 kB)
ERROR: Cannot install shiny, shiny 0.1.dev1856 (from /..../py-shiny) and shiny[dev,test]==0.1.dev1856 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested shiny 0.1.dev1856 (from /..../py-shiny)
    shiny[dev,test] 0.1.dev1856 depends on shiny 0.1.dev1856 (from /..../py-shiny)
    shinyswatch 0.8.0 depends on shiny>=1.2.0
    The user requested shiny 0.1.dev1856 (from /..../py-shiny)
    shiny[dev,test] 0.1.dev1856 depends on shiny 0.1.dev1856 (from /..../py-shiny)
    shinyswatch 0.7.0 depends on shiny>=1.0.0

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip to attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

My environment:

$ python --version
Python 3.13.1
$ git rev-parse --short HEAD
1e9b8683

I looked at the github workflows, but the install process there is different, so this process probably isn't checked by CI. If I sort this out, I'll file a PR.

[gadenbuie]

ERROR: Cannot install shiny, shiny 0.1.dev1856 (from /..../py-shiny) and shiny[dev,test]==0.1.dev1856

This part seems suspicious. Are you sure you're installing into a fresh virtual environment (it's easy to miss) because shiny 0.1 is quite old and probably not what you're intending.

[mccalluc]

Got it: I had forked the repo, which doesn't carry over tags by default, but tags are used by setuptools_scm.

$ python -m setuptools_scm
0.1.dev1856
$ git remote add upstream https://github.com/posit-dev/py-shiny.git
$ git fetch --tags upstream
...
$ python -m setuptools_scm
1.2.2.dev30

And the build gets farther, and hits a different error

"Preparing metadata (pyproject.toml) did not run successfully."

Collecting pandas (from shiny==1.2.2.dev30)
  Using cached pandas-2.1.4.tar.gz (4.3 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [72 lines of output]
      + meson setup /private/var/folders/jm/40wztt291610f8pk5w0sw20r0000gq/T/pip-install-7ggnktzr/pandas_83271b1eab42444ca974c1a72c84285a /private/var/folders/jm/40wztt291610f8pk5w0sw20r0000gq/T/pip-install-7ggnktzr/pandas_83271b1eab42444ca974c1a72c84285a/.mesonpy-x8j7dy2t/build -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --vsenv --native-file=/private/var/folders/jm/40wztt291610f8pk5w0sw20r0000gq/T/pip-install-7ggnktzr/pandas_83271b1eab42444ca974c1a72c84285a/.mesonpy-x8j7dy2t/build/meson-python-native-file.ini
      The Meson build system
      Version: 1.2.1
      Source dir: /private/var/folders/jm/40wztt291610f8pk5w0sw20r0000gq/T/pip-install-7ggnktzr/pandas_83271b1eab42444ca974c1a72c84285a
      Build dir: /private/var/folders/jm/40wztt291610f8pk5w0sw20r0000gq/T/pip-install-7ggnktzr/pandas_83271b1eab42444ca974c1a72c84285a/.mesonpy-x8j7dy2t/build
      Build type: native build
      Project name: pandas
      Project version: 2.1.4
      C compiler for the host machine: cc (clang 15.0.0 "Apple clang version 15.0.0 (clang-1500.3.9.4)")
      C linker for the host machine: cc ld64 1053.12
      C++ compiler for the host machine: c++ (clang 15.0.0 "Apple clang version 15.0.0 (clang-1500.3.9.4)")
      C++ linker for the host machine: c++ ld64 1053.12
      Cython compiler for the host machine: cython (cython 0.29.37)
      Host machine cpu family: aarch64
      Host machine cpu: aarch64
      Program python found: YES (/Users/chuckmccallum/github/mccalluc/py-shiny/venv/bin/python3.13)
      Did not find pkg-config by name 'pkg-config'
      Found Pkg-config: NO
      Run-time dependency python found: YES 3.13
      Build targets in project: 53

      pandas 2.1.4

        User defined options
          Native files: /private/var/folders/jm/40wztt291610f8pk5w0sw20r0000gq/T/pip-install-7ggnktzr/pandas_83271b1eab42444ca974c1a72c84285a/.mesonpy-x8j7dy2t/build/meson-python-native-file.ini
          buildtype   : release
          vsenv       : True
          b_ndebug    : if-release
          b_vscrt     : md

      Found ninja-1.11.1.git.kitware.jobserver-1 at /private/var/folders/jm/40wztt291610f8pk5w0sw20r0000gq/T/pip-build-env-i5dxkh4z/normal/bin/ninja

      Visual Studio environment is needed to run Ninja. It is recommended to use Meson wrapper:
      /private/var/folders/jm/40wztt291610f8pk5w0sw20r0000gq/T/pip-build-env-i5dxkh4z/overlay/bin/meson compile -C .
      + /private/var/folders/jm/40wztt291610f8pk5w0sw20r0000gq/T/pip-build-env-i5dxkh4z/normal/bin/ninja
      [1/151] Generating pandas/__init__.py with a custom command
      [2/151] Generating pandas/_libs/hashtable_class_helper_pxi with a custom command
      [3/151] Generating pandas/_libs/algos_take_helper_pxi with a custom command
      [4/151] Generating pandas/_libs/algos_common_helper_pxi with a custom command
      [5/151] Generating pandas/_libs/sparse_op_helper_pxi with a custom command
      [6/151] Generating pandas/_libs/intervaltree_helper_pxi with a custom command
      [7/151] Generating pandas/_libs/index_class_helper_pxi with a custom command
      [8/151] Generating pandas/_libs/hashtable_func_helper_pxi with a custom command
      [9/151] Generating pandas/_libs/khash_primitive_helper_pxi with a custom command
      [10/151] Compiling C object pandas/_libs/tslibs/parsing.cpython-313-darwin.so.p/.._src_parser_tokenizer.c.o
      [11/151] Compiling Cython source /private/var/folders/jm/40wztt291610f8pk5w0sw20r0000gq/T/pip-install-7ggnktzr/pandas_83271b1eab42444ca974c1a72c84285a/pandas/_libs/tslibs/base.pyx
      [12/151] Compiling C object pandas/_libs/tslibs/base.cpython-313-darwin.so.p/meson-generated_pandas__libs_tslibs_base.pyx.c.o
      FAILED: pandas/_libs/tslibs/base.cpython-313-darwin.so.p/meson-generated_pandas__libs_tslibs_base.pyx.c.o
      cc -Ipandas/_libs/tslibs/base.cpython-313-darwin.so.p -Ipandas/_libs/tslibs -I../../pandas/_libs/tslibs -I../../../../pip-build-env-i5dxkh4z/overlay/lib/python3.13/site-packages/numpy/core/include -I../../pandas/_libs/include -I/opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.13/include/python3.13 -I/opt/homebrew/Cellar/[email protected]/3.13.1/Frameworks/Python.framework/Versions/3.13/include/python3.13 -fvisibility=hidden -fcolor-diagnostics -DNDEBUG -w -std=c99 -O3 -DNPY_NO_DEPRECATED_API=0 -DNPY_TARGET_VERSION=NPY_1_21_API_VERSION -MD -MQ pandas/_libs/tslibs/base.cpython-313-darwin.so.p/meson-generated_pandas__libs_tslibs_base.pyx.c.o -MF pandas/_libs/tslibs/base.cpython-313-darwin.so.p/meson-generated_pandas__libs_tslibs_base.pyx.c.o.d -o pandas/_libs/tslibs/base.cpython-313-darwin.so.p/meson-generated_pandas__libs_tslibs_base.pyx.c.o -c pandas/_libs/tslibs/base.cpython-313-darwin.so.p/pandas/_libs/tslibs/base.pyx.c
      pandas/_libs/tslibs/base.cpython-313-darwin.so.p/pandas/_libs/tslibs/base.pyx.c:5399:70: error: too few arguments to function call, expected 6, have 5
                                                    is_little, !is_unsigned);
                                                                           ^
      /opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.13/include/python3.13/cpython/longobject.h:111:17: note: '_PyLong_AsByteArray' declared here
      PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
                      ^
      pandas/_libs/tslibs/base.cpython-313-darwin.so.p/pandas/_libs/tslibs/base.pyx.c:5633:70: error: too few arguments to function call, expected 6, have 5
                                                    is_little, !is_unsigned);
                                                                           ^
      /opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.13/include/python3.13/cpython/longobject.h:111:17: note: '_PyLong_AsByteArray' declared here
      PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
                      ^
      2 errors generated.
      [13/151] Compiling Cython source /private/var/folders/jm/40wztt291610f8pk5w0sw20r0000gq/T/pip-install-7ggnktzr/pandas_83271b1eab42444ca974c1a72c84285a/pandas/_libs/tslibs/ccalendar.pyx
      [14/151] Compiling Cython source /private/var/folders/jm/40wztt291610f8pk5w0sw20r0000gq/T/pip-install-7ggnktzr/pandas_83271b1eab42444ca974c1a72c84285a/pandas/_libs/tslibs/dtypes.pyx
      [15/151] Compiling Cython source /private/var/folders/jm/40wztt291610f8pk5w0sw20r0000gq/T/pip-install-7ggnktzr/pandas_83271b1eab42444ca974c1a72c84285a/pandas/_libs/tslibs/np_datetime.pyx
      [16/151] Compiling Cython source /private/var/folders/jm/40wztt291610f8pk5w0sw20r0000gq/T/pip-install-7ggnktzr/pandas_83271b1eab42444ca974c1a72c84285a/pandas/_libs/tslibs/nattype.pyx
      [17/151] Compiling Cython source /private/var/folders/jm/40wztt291610f8pk5w0sw20r0000gq/T/pip-install-7ggnktzr/pandas_83271b1eab42444ca974c1a72c84285a/pandas/_libs/tslibs/conversion.pyx
      [18/151] Compiling Cython source /private/var/folders/jm/40wztt291610f8pk5w0sw20r0000gq/T/pip-install-7ggnktzr/pandas_83271b1eab42444ca974c1a72c84285a/pandas/_libs/tslibs/fields.pyx
      [19/151] Compiling Cython source /private/var/folders/jm/40wztt291610f8pk5w0sw20r0000gq/T/pip-install-7ggnktzr/pandas_83271b1eab42444ca974c1a72c84285a/pandas/_libs/tslibs/parsing.pyx
      [20/151] Compiling Cython source /private/var/folders/jm/40wztt291610f8pk5w0sw20r0000gq/T/pip-install-7ggnktzr/pandas_83271b1eab42444ca974c1a72c84285a/pandas/_libs/tslibs/offsets.pyx
      [21/151] Compiling Cython source /private/var/folders/jm/40wztt291610f8pk5w0sw20r0000gq/T/pip-install-7ggnktzr/pandas_83271b1eab42444ca974c1a72c84285a/pandas/_libs/tslibs/period.pyx
      ninja: build stopped: subcommand failed.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.

[notice] A new release of pip is available: 24.3.1 -> 25.0.1
[notice] To update, run: pip install --upgrade pip
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

For the git tags, I'll make a little PR for the README in case that would help anyone else, and then look at the new error.

[gadenbuie]

For the git tags, I'll make a little PR for the README in case that would help anyone else, and then look at the new error.

That'd be great, thanks for the offer. It should be included under the "Development" heading; it's definitely the kind of subtle detail with the automatic versioning that's easy to miss.

[mccalluc]

See #1858 for proposed addition to README.

Still looking at the rest of the install. Since the problem seems to be with an older pandas version, what happens if I require something more recent?

diff --git a/pyproject.toml b/pyproject.toml
index 8d7fc886..993227a9 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -107,7 +107,7 @@ dev = [
     "pre-commit>=2.15.0",
     "wheel",
     "matplotlib",
-    "pandas",
+    "pandas>=2.2.0",
     "pandas-stubs",
     "polars",
     "numpy",

A different error:

        File "/private/var/folders/jm/40wztt291610f8pk5w0sw20r0000gq/T/pip-install-1idsb5pf/modin_6c255784cb0748fb86f0edef7573cf4d/versioneer.py", line 342, in get_config_from_root
          parser = configparser.SafeConfigParser()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      AttributeError: module 'configparser' has no attribute 'SafeConfigParser'. Did you mean: 'RawConfigParser'?

Seems to be hitting this problem in modin:

• versioneer.py is broken in python 3.12 modin-project/modin#7309

And this in the cpython changelog for 3.12.0:

gh-89336: Removed configparser module APIs: the SafeConfigParser class alias, the ParsingError.filename property and parameter, and the ConfigParser.readfp method, all of which were deprecated since Python 3.2.

I'll try installing on python 3.11. (I'll add notes on this issue as I debug, rather than filing lots of little issues along the way, but feel free to close this when the PR comes in. Your call.)

[gadenbuie]

Thanks @mccalluc. In general, our approach is to be lenient with versioning unless we're aware of a specific incompatibility with shiny that needs to be resolved by requiring a newer version of a dependency. In other words, we'd need to distinguish between a) it being difficult to install an older version of pandas (very likely) or b) a compatibility issue prevents using an older version of pandas with shiny.

Please feel free to track your journey here and if you uncover an incompatibility we'll definitely address it.

[mccalluc]

Downgraded to Python 3.11.7 and pip install -e ".[dev,test] worked, with no dependencies pinned! I'll make another PR to suggest that developers use a version less than Python 3.12 until the modin issue is resolved, but I'm not confident that I'm diagnosing it correctly -- This is just what worked for me.

(At the end of this, I'm not sure what the pandas problem was, or whether it was related to modin.)

Closing.