mopidy/install.sh: broken on Bookworm #98

Gadgetoid · 2023-10-25T12:35:00Z

I don't know what the workaround might be, but the changes to Bookworm that prevent pip from installing packages system-wide mean it's impossible to install Mopidy plugins from pypi.

That means the install script can't add essential plugins for Pirate Audio: Mopidy-PiDi Mopidy-Local pidi-display-pil pidi-display-st7789 mopidy-raspberry-gpio, which happens here: https://github.com/pimoroni/pirate-audio/blob/d9d7cadf1686860dcbfe42b74d34454942cba2d4/mopidy/install.sh#L126C14-L126C14

Right now I don't know what the path to fixing this is, and I'm still working out how we approach virtual environments across our products.

So -

Pirate Audio + Mopidy will currently not work on Pi 5, for this and other reasons
Pirate Audio + Modpify will not work on Bookworm OS in a Pi 4, though just audio output is possible

In the mean time I recommend using Raspberry Pi OS Bullseye, which you can still get from here:

http://downloads.raspberrypi.org/raspios_oldstable_full_armhf/images/raspios_oldstable_full_armhf-2023-10-10/

The text was updated successfully, but these errors were encountered:

L0stCh1ld · 2023-12-20T15:43:22Z

there is a dirty kludge fix for now on bookworm. edit 'install.sh' and locate the lines beginning with ' $PIP_BIN install --upgrade ' edit them like so ' $PIP_BIN install --break-system-packages --upgrade '

amigauser1 · 2024-02-04T11:35:36Z

Another way is to add the following line at the beginning (after #!/bin/bash) of install.sh:

export PIP_BREAK_SYSTEM_PACKAGES=1

This avoids adding the break-system-option to all pip lines.

kingosticks · 2024-03-01T18:06:26Z

@Gadgetoid I'd like to fix this (including playback), am I OK to go with either of the last two options? Or something else?

theNizo · 2024-04-09T09:39:22Z

Tried venvs - mopidy can't find gstreamer
Tried installing everything via pip - doesn't change the fact that the EXTERNALLY-MANAGED already exists in the image you flash. You have to install gstreamer1.0-alsa for it to work additionally, and you don't get a systemd-service automatically
Tried with the flag - seems to be working best. GPIO project doesn't work on the newest version of python because of course it doesn't, so buttons don't work.

kingosticks · 2024-04-09T09:53:43Z

Tried venvs - mopidy can't find gstreamer

You can if you create the venv with --system-site-packages

Tried installing everything via pip

I see you've deleted your original post about this, but yes, this doesn't help. Externally-managed is about not installing non-apt packages into the system Python space, sudo pip does nothing to avoid this.

Ignoring externally-managed is the ~~only~~ easiest solution. There are multiple way to ignore it. The non-permanent ways are best, in my opinion. But until there's some appetite for accepting these changes, I'm not going to spend time on it.

theNizo · 2024-04-09T11:42:47Z

You can if you create the venv with --system-site-packages

Oh, thank you, maybe I'll try that.

I see you've deleted your original post

don't want to post incomplete/wrong information here. Just saying "It doesn't work" without details helps no one. - Yes, I still complained about GPIO, but details would be for a different issue, probably in the according git-project.

Gadgetoid · 2024-04-09T12:19:36Z

Just saying "It doesn't work" helps no one.

I appreciate the solidarity, if nothing else 😭

@kingosticks afaik --system-site-packages or BREAK_SYSTEM_PACKAGES are roughly equivalent since Python packages will "shadow" system ones when you're in a venv context and cause some of the (imagined afaict) problems that this whole debacle aims to avoid.

The benefit of --system-site-packages over BREAK_SYSTEM_PACKAGES (the latter causes pip to just disregard the warnings) is that packages will only appear broken (if they do at all) when in a venv context. It's less breaky, so if you can wrangle running Mopidy in the virtual env then it will be self-contained.

Downside of managing in a venv is that Mopidy seems to prefer apt packaging? The two mix poorly, even with --system-site-packages since a Python project cannot specify an apt dependency- if your apt/pypi stuff is out of sync then deps pulling into the venv will shadow the system packages and cause confusion.

I'm probably preaching to the choir here, though. And I'm not even really sure I'm answering the question you asked, or any question at all.

kingosticks · 2024-04-09T13:14:09Z

@kingosticks afaik --system-site-packages or BREAK_SYSTEM_PACKAGES are roughly equivalent since Python packages will "shadow" system ones when you're in a venv context and cause some of the (imagined afaict) problems that this whole debacle aims to avoid.

I'm not sure that's an entirely fair distinction. The --system-site-packages venv option can never break your system. The --break-system-packages pip option (or the equivalent env variable also mentioned) technically could. I have never experienced it either but I appreciate how it could happen.

To distribute a system service on Debian, apt is the most sensible way. You can't really distribute a Debian system service via PyPi. We also need a way to distribute our non-Python dependency for Mopidy-Spotify, apt is how we do that. And once you've got something on apt, there's a low barrier to putting everything you can on there and enjoying the benefits of system package management. Before the changes, this all used to work pretty well (in my biased opinion). Despite the current Python-world mess, I still think apt is the best way to install as much of Mopidy as possible.

since a Python project cannot specify an apt dependency

Technically it can specify and enforce the dependency. It just can't resolve it properly and yes, you get the shadowing situation. We use this flow for our development environments and it mostly works OK. However, I do agree with you and I'm not sure I would burden regular users with this more fragile/confusing setup.

whimshot · 2024-06-12T13:51:19Z

The notice for this says that the issue has been solved. However when I attempt to run the install.sh script a rather long error. I'm doing this on a Raspberry Pi 400 running the latest Raspi OS 64bit (Bookworm)

$ uname -a
Linux rpi400 6.6.31+rpt-rpi-v8 #1 SMP PREEMPT Debian 1:6.6.31-1+rpt1 (2024-05-29) aarch64 GNU/Linux

Here is the output from install.sh

sudo ./install.sh
Updating apt and installing dependencies
Hit:1 https://apt.mopidy.com bullseye InRelease
Hit:2 http://deb.debian.org/debian bookworm InRelease
Hit:3 http://deb.debian.org/debian-security bookworm-security InRelease
Hit:4 http://deb.debian.org/debian bookworm-updates InRelease
Hit:5 http://archive.raspberrypi.com/debian bookworm InRelease
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
All packages are up to date.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
python3-rpi.gpio is already the newest version (0.7.1~a4-1+b4).
python3-spidev is already the newest version (20200602~200721-1+bookworm).
python3-pip is already the newest version (23.0.1+dfsg-1+rpt1).
python3-pil is already the newest version (9.4.0-1.1+deb12u1).
python3-numpy is already the newest version (1:1.24.2-1).
libopenjp2-7 is already the newest version (2.5.0-2).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Verifying python 3.x version
Found Python 3.11

Backing up mopidy config to: /etc/mopidy/mopidy.conf.backup-2024-06-12-09-43-47

Installing mopidy packages
mopidy was already not on hold.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
mopidy is already the newest version (3.4.2-1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Installing Iris web UI for Mopidy
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    For more information visit http://rptl.io/venv

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 514, in get_distribution
    dist = get_provider(dist)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 386, in get_provider
    return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 956, in require
    needed = self.resolve(parse_requirements(requirements))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 815, in resolve
    dist = self._resolve_dist(
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 856, in _resolve_dist
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'mopidy_iris' distribution was not found and is required by the application
Could not find system.sh path for mopidy_iris using python3
Refusing to edit /etc/sudoers.d/010_mopidy-nopasswd with empty system.sh path!
Installing Pirate Audio plugins...
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    For more information visit http://rptl.io/venv

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

Resetting /etc/mopidy/mopidy.conf to package defaults.
Any custom settings have been backed up to /etc/mopidy/mopidy.conf.backup-2024-06-12-09-43-47

Configuring Mopidy

Enabling and starting Mopidy
Synchronizing state of mopidy.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable mopidy

All done!
Mopidy configuration has changed, see summary below and make sure to update /etc/mopidy/mopidy.conf!
Your previous configuration was backed up to /etc/mopidy/mopidy.conf.backup-2024-06-12-09-43-47
103,150d102
<
< [raspberry-gpio]
< enabled = true
< bcm5 = play_pause,active_low,250
< bcm6 = volume_down,active_low,250
< bcm16 = next,active_low,250
< bcm20 = volume_up,active_low,250
< bcm24 = volume_up,active_low,250
<
< [file]
< enabled = true
< media_dirs = /home/pi/Music
< show_dotfiles = false
< excluded_file_extensions =
<   .directory
<   .html
<   .jpeg
<   .jpg
<   .log
<   .nfo
<   .pdf
<   .png
<   .txt
<   .zip
< follow_symlinks = false
< metadata_timeout = 1000
<
< [pidi]
< enabled = true
< display = st7789
< rotation = 90
<
< [mpd]
< hostname = 0.0.0.0
<
< [http]
< hostname = 0.0.0.0
<
< [audio]
< mixer_volume = 40
< output = alsasink device=hw:sndrpihifiberry
<
< [spotify]
< enabled = false
< username =
< password =
< client_id =
< client_secret =

kingosticks · 2024-06-13T10:45:31Z

The notice for this says that the issue has been solved. However when I attempt to run the install.sh script a rather long error. I'm doing this on a Raspberry Pi 400 running the latest Raspi OS 64bit (Bookworm)

$ uname -a
Linux rpi400 6.6.31+rpt-rpi-v8 #1 SMP PREEMPT Debian 1:6.6.31-1+rpt1 (2024-05-29) aarch64 GNU/Linux

Here is the output from install.sh

sudo ./install.sh
Updating apt and installing dependencies
Hit:1 https://apt.mopidy.com bullseye InRelease
Hit:2 http://deb.debian.org/debian bookworm InRelease
Hit:3 http://deb.debian.org/debian-security bookworm-security InRelease
Hit:4 http://deb.debian.org/debian bookworm-updates InRelease
Hit:5 http://archive.raspberrypi.com/debian bookworm InRelease
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
All packages are up to date.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
python3-rpi.gpio is already the newest version (0.7.1~a4-1+b4).
python3-spidev is already the newest version (20200602~200721-1+bookworm).
python3-pip is already the newest version (23.0.1+dfsg-1+rpt1).
python3-pil is already the newest version (9.4.0-1.1+deb12u1).
python3-numpy is already the newest version (1:1.24.2-1).
libopenjp2-7 is already the newest version (2.5.0-2).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Verifying python 3.x version
Found Python 3.11

Backing up mopidy config to: /etc/mopidy/mopidy.conf.backup-2024-06-12-09-43-47

Installing mopidy packages
mopidy was already not on hold.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
mopidy is already the newest version (3.4.2-1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Installing Iris web UI for Mopidy
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    For more information visit http://rptl.io/venv

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 514, in get_distribution
    dist = get_provider(dist)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 386, in get_provider
    return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 956, in require
    needed = self.resolve(parse_requirements(requirements))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 815, in resolve
    dist = self._resolve_dist(
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 856, in _resolve_dist
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'mopidy_iris' distribution was not found and is required by the application
Could not find system.sh path for mopidy_iris using python3
Refusing to edit /etc/sudoers.d/010_mopidy-nopasswd with empty system.sh path!
Installing Pirate Audio plugins...
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    For more information visit http://rptl.io/venv

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

Resetting /etc/mopidy/mopidy.conf to package defaults.
Any custom settings have been backed up to /etc/mopidy/mopidy.conf.backup-2024-06-12-09-43-47

Configuring Mopidy

Enabling and starting Mopidy
Synchronizing state of mopidy.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable mopidy

All done!
Mopidy configuration has changed, see summary below and make sure to update /etc/mopidy/mopidy.conf!
Your previous configuration was backed up to /etc/mopidy/mopidy.conf.backup-2024-06-12-09-43-47
103,150d102
<
< [raspberry-gpio]
< enabled = true
< bcm5 = play_pause,active_low,250
< bcm6 = volume_down,active_low,250
< bcm16 = next,active_low,250
< bcm20 = volume_up,active_low,250
< bcm24 = volume_up,active_low,250
<
< [file]
< enabled = true
< media_dirs = /home/pi/Music
< show_dotfiles = false
< excluded_file_extensions =
<   .directory
<   .html
<   .jpeg
<   .jpg
<   .log
<   .nfo
<   .pdf
<   .png
<   .txt
<   .zip
< follow_symlinks = false
< metadata_timeout = 1000
<
< [pidi]
< enabled = true
< display = st7789
< rotation = 90
<
< [mpd]
< hostname = 0.0.0.0
<
< [http]
< hostname = 0.0.0.0
<
< [audio]
< mixer_volume = 40
< output = alsasink device=hw:sndrpihifiberry
<
< [spotify]
< enabled = false
< username =
< password =
< client_id =
< client_secret =

Do #98 (comment)

VaporwareII · 2024-08-15T20:33:49Z

Another way is to add the following line at the beginning (after #!/bin/bash) of install.sh:

export PIP_BREAK_SYSTEM_PACKAGES=1

This avoids adding the break-system-option to all pip lines.

I ended up going with this to resolve the installation failures. Ironically I ran the install script 3 -4 times to figure out the problem and the mopidy.conf file ended up appending the whole config at the end every time, so it failed afterwards when I tried to start the service. I had to delete the extra config lines, but it worked just fine after that.

infact the whole section on installation and config was so easy and well explained diving into the mopidy docs was like a cold shower.

as an aside, it might be a good idea to add the default aplay /usr/share/sounds/alsa/Front_Center.wav command to the instructions section after you finish the install script to test the sound (assuming it's hooked up to a speaker) before you dive into settings and config on extensions.

kingosticks · 2024-08-15T20:36:23Z

Fyi, if you've not seen, Spotty recently broke the playback library Mopidy-Spotify is now using. Fixes are coming but we're not there yet.

Gadgetoid · 2024-11-11T16:48:08Z

Day late and a dollar short here, but I've got an installer that seems to work for a virtualenv. It's in #105

I've got no audio at the moment, so that's probably another rabbit hole to delve into.

Gadgetoid added the notice This issue is resolved, but left open for posterity/visibility label Oct 25, 2023

quozl mentioned this issue Apr 25, 2024

RuntimeError: Failed to add edge detection pimoroni/mopidy-raspberry-gpio#9

Open

jkeilson mentioned this issue Jun 6, 2024

Update install.sh to test for bookworm firmware location #100

Open

s-martin mentioned this issue Aug 19, 2024

No audio output with Spotify MiczFlor/RPi-Jukebox-RFID#2418

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mopidy/install.sh: broken on Bookworm #98

mopidy/install.sh: broken on Bookworm #98

Gadgetoid commented Oct 25, 2023

L0stCh1ld commented Dec 20, 2023 •

edited

Loading

amigauser1 commented Feb 4, 2024

kingosticks commented Mar 1, 2024 •

edited

Loading

theNizo commented Apr 9, 2024

kingosticks commented Apr 9, 2024 •

edited

Loading

theNizo commented Apr 9, 2024 •

edited

Loading

Gadgetoid commented Apr 9, 2024

kingosticks commented Apr 9, 2024 •

edited

Loading

whimshot commented Jun 12, 2024

kingosticks commented Jun 13, 2024

VaporwareII commented Aug 15, 2024

kingosticks commented Aug 15, 2024

Gadgetoid commented Nov 11, 2024

mopidy/install.sh: broken on Bookworm #98

mopidy/install.sh: broken on Bookworm #98

Comments

Gadgetoid commented Oct 25, 2023

L0stCh1ld commented Dec 20, 2023 • edited Loading

amigauser1 commented Feb 4, 2024

kingosticks commented Mar 1, 2024 • edited Loading

theNizo commented Apr 9, 2024

kingosticks commented Apr 9, 2024 • edited Loading

theNizo commented Apr 9, 2024 • edited Loading

Gadgetoid commented Apr 9, 2024

kingosticks commented Apr 9, 2024 • edited Loading

whimshot commented Jun 12, 2024

kingosticks commented Jun 13, 2024

VaporwareII commented Aug 15, 2024

kingosticks commented Aug 15, 2024

Gadgetoid commented Nov 11, 2024

L0stCh1ld commented Dec 20, 2023 •

edited

Loading

kingosticks commented Mar 1, 2024 •

edited

Loading

kingosticks commented Apr 9, 2024 •

edited

Loading

theNizo commented Apr 9, 2024 •

edited

Loading

kingosticks commented Apr 9, 2024 •

edited

Loading