Merge pull request #517 from MLB-LED-Scoreboard/dev

Release v8.0.0

.git-config-template

@@ -0,0 +1,4 @@
+[filter "noshebang"]
+    clean = grep -v '.!/.*python3'
+    smudge = cat
+    required

.gitattributes

@@ -0,0 +1 @@
+main.py filter=noshebang

.github/workflows/run_unittest_on_pr_open.yml

@@ -12,11 +12,11 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
+        python-version: ["3.8", "3.9", "3.10", "3.11"]
     steps:
       - uses: actions/checkout@v3
       - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
       - name: Install dependencies

.gitignore

@@ -24,6 +24,7 @@ wheels/
 .installed.cfg
 *.egg
 MANIFEST
+venv/
 
 # PyInstaller
 #  Usually these files are written by a python script from a template

README.md

@@ -1,7 +1,8 @@
 # mlb-led-scoreboard
 
-![Current Version](https://img.shields.io/github/v/release/MLB-LED-Scoreboard/MLB-LED-Scoreboard) [![Join Discord](https://img.shields.io/badge/discord-join-blue.svg)](https://discord.gg/FdD6ec9fdt)
+![Current Version](https://img.shields.io/github/v/release/MLB-LED-Scoreboard/MLB-LED-Scoreboard) ![](https://img.shields.io/badge/python-3.8_%7C_3.9_%7C_3.10_%7C_3.11-blue)
 
+[![Join Discord](https://img.shields.io/badge/discord-join-green.svg)](https://discord.gg/FdD6ec9fdt)
 
 <a href="assets/img/header.jpg">
   <img alt="Project header" width="800px" height="auto" src="assets/img/header.jpg">
@@ -118,6 +119,9 @@ cd mlb-led-scoreboard/
 sudo ./install.sh
 ```
 
+This will create a Python Virtual Environment and install all of the required dependencies. The
+virtual environment will be located at `mlb-led-scoreboard/venv/`.
+
 This will install the rgbmatrix binaries, which we get from [another open source library](https://github.com/hzeller/rpi-rgb-led-matrix/tree/master/bindings/python#building). It controls the actual rendering of the scoreboard onto the LEDs. If you're curious, you can read through their documentation on how all of the lower level stuff works.
 
 It will also install the following python libraries that are required for certain parts of the scoreboard to function.
@@ -144,7 +148,7 @@ Additional flags are available for customizing your install:
 -c, --skip-config  Skips default config overwrite without prompting.
 
 -a, --skip-all     Performs all above skips.
-
+--no-venv          Do not create a virtual environment for the dependencies.
 --emulator-only    Do not install dependencies under sudo. Skips building matrix dependencies.
 
 -h, --help         Displays help
@@ -167,11 +171,14 @@ The latest version of the software is available [here](https://github.com/MLB-LE
 Make sure your Raspberry Pi's timezone is configured to your local time zone. They'll often have London time on them by default. You can change the timezone of your raspberry pi by running `sudo raspi-config`.
 
 ## Usage
-`sudo python3 main.py` Running as root is 100% an absolute must, or the matrix won't render.
+The installation script adds a line to the top of `main.py` to automatically pick up the virtual environment.
+This means re-activating the environment (`source ./venv/bin/activate`) is not a requirement.
+
+`sudo ./main.py` Running as root is 100% an absolute must, or the matrix won't render.
 
 **Adafruit HAT/Bonnet users: You must supply a command line flag:**
 
-`sudo python3 main.py --led-gpio-mapping="adafruit-hat"`
+`sudo ./main.py --led-gpio-mapping="adafruit-hat"`
 
 See the Flags section below for more flags you can optionally provide.
 
@@ -180,13 +187,13 @@ See the Flags section below for more flags you can optionally provide.
 The scoreboard can run on other platforms by means of software emulation via `RGBMatrixEmulator`. When running via the emulator, you do not need to prepend your startup commands with `sudo`:
 
 ```sh
-python3 main.py
+./main.py
 ```
 
 You can also force the scoreboard into emulation mode by using the `--emulated` flag:
 
 ```sh
-python3 main.py --emulated
+./main.py --emulated
 ```
 
 When running in emulation mode, you can continue to use your existing command line flags as normal.
@@ -262,7 +269,7 @@ A default `config.json.example` file is included for reference. Copy this file t
 * Pitch Data - Pitch data can be shown on the game screen, See the [coordinates readme file](/coordinates/README.md) for details. In addition, the `short` and `long` pitch description can be changed in data/pitches.py
 
 * Previous Play Data - Data for the previous play can be shown on the game screen. See the [coordinates readme file](/coordinates/README.md) for details. Long and short play descriptions can be changed in data/plays.py
-  * **NOTE:** Because play result data is ephemeral, not every play result will be displayed. Situations like a mound visit, injury, or other timeout immediately following a play often cause the play result to be immediately replaced on the MLB API. 
+  * **NOTE:** Because play result data is ephemeral, not every play result will be displayed. Situations like a mound visit, injury, or other timeout immediately following a play often cause the play result to be immediately replaced on the MLB API.
 
 ### Flags
 
@@ -288,6 +295,7 @@ You can configure your LED matrix with the same flags used in the [rpi-rgb-led-m
 --led-pwm-dither-bits     Time dithering of lower bits (Default: 0)
 --config                  Specify a configuration file name other, omitting json xtn (Default: config)
 --emulated                Force the scoreboard to run in software emulation mode.
+--drop-privileges         Force the matrix driver to drop root privileges after setup. (Default: true)
 ```
 
 ## Personalization

install.sh

@@ -11,7 +11,7 @@ usage() {
         -p, --skip-python:  Skip Python 3 installation. Requires manual Python 3 setup if not already installed.
 
         -a, --skip-all:     Skip all dependencies and config installation (equivalent to -c -p -m).
-
+        --no-venv           Do not create a virtual environment for the dependencies.
         --emulator-only:    Do not install dependencies under sudo. Skips building matrix dependencies (equivalent to -m)
 
 USAGE
@@ -22,6 +22,7 @@ SKIP_PYTHON=false
 SKIP_CONFIG=false
 SKIP_MATRIX=false
 NO_SUDO=false
+SKIP_VENV=false
 
 for arg in "$@"; do
     case $arg in
@@ -41,13 +42,18 @@ for arg in "$@"; do
         SKIP_CONFIG=true
         SKIP_MATRIX=true
         SKIP_PYTHON=true
+        SKIP_VENV=true
         shift # Remove -a / --skip-all from `$@`
         ;;
     --emulator-only)
         SKIP_MATRIX=true
         NO_SUDO=true
         shift # remove --emulator-only from `$@`
         ;;
+    --no-venv)
+        SKIP_VENV=true
+        shift # remove --no-venv from `$@`
+        ;;
     -h | --help)
         usage # run usage function on help
         ;;
@@ -69,6 +75,7 @@ if [ "$SKIP_PYTHON" = false ]; then
         python3-pip \
         python3-pillow \
         python3-tk \
+        python3-venv \
         libxml2-dev \
         libxslt-dev \
         libsdl2-mixer-2.0-0 \
@@ -84,21 +91,52 @@ echo "  Installing dependencies..."
 echo "------------------------------------"
 echo
 
+if [ "$SKIP_VENV" = false ]; then
+    echo "Creating virtual environment..."
+    if [ "$NO_SUDO" = false ]; then
+        sudo python3 -m venv ./venv
+    else
+        python3 -m venv ./venv
+    fi
+    source ./venv/bin/activate
+
+    if ! grep -q "#\!/" main.py; then
+        if [[ "$OSTYPE" == "darwin"* ]]; then
+            sed -i '' '1i\'$'\n''#!'"$(which python3)"$'\n' main.py
+        elif [ "$NO_SUDO" = false ]; then
+            sed -i "1i #\!/usr/bin/sudo $(which python3)" main.py
+        else
+            sed -i "1i #\!$(which python3)" main.py
+        fi
+        chmod +x main.py
+
+        # Add template to .git/config (if it doesn't already exist), and trigger the filter by adding the file.
+        # After that, the shebang should be ignored.
+        if ! grep -q "noshebang" ./.git/config; then
+            cat .git-config-template >> .git/config
+        fi
+
+        git add main.py
+    fi
+fi
+PYTHON=$(which python3)
+
 if [ "$NO_SUDO" = false ]; then
-    sudo python3 -m pip install -r requirements.txt
+    sudo "$PYTHON" -m pip install -r requirements.txt
 else
-    python3 -m pip install -r requirements.txt
+    "$PYTHON" -m pip install -r requirements.txt
 fi
 
 if [ "$SKIP_MATRIX" = false ]; then
     echo "Running rgbmatrix installation..."
+    sudo apt-get install -y make
     mkdir submodules
     cd submodules
     git clone https://github.com/hzeller/rpi-rgb-led-matrix.git matrix
     cd matrix
     git pull
-    make build-python PYTHON=$(which python3)
-    sudo make install-python PYTHON=$(which python3)
+    make build-python PYTHON="$PYTHON"
+    sudo make install-python PYTHON="$PYTHON"
 
     cd ../..
 fi
@@ -123,7 +161,7 @@ else
     echo "  update them with the latest options at this time."
     echo
     echo "  This operation is automatic and will ensure you have up-to-date configuration."
-    echo 
+    echo
     echo "  This action will NOT override any custom configuration you already have unless"
     echo "  the option has been obsoleted and is no longer in use."
     echo "==================================================================================="

main.py

@@ -21,21 +21,7 @@
 import threading
 import time
 
-# TODO: This code addresses CVE-2023-4863 in Pillow < 10.0.1, which requires Python 3.8+
-#   See requirements.txt for rationale.
-try:
-    from PIL import Image
-
-    pil_version = tuple(map(int, Image.__version__.split(".")))
-    if pil_version < (10, 0, 1):
-        debug.warning(f"Attempted to load an insecure PIL version ({Image.__version__}). We require PIL 10.0.1 or higher.")
-
-        raise ModuleNotFoundError
-
-    PIL_LOADED = True
-except:
-    debug.warning("PIL failed to load -- images will not be displayed.")
-    PIL_LOADED = False
+from PIL import Image
 
 # Important! Import the driver first to initialize it, then import submodules as needed.
 import driver
@@ -74,7 +60,7 @@ def main(matrix, config_base):
 
     # MLB image disabled when using renderer, for now.
     # see: https://github.com/ty-porter/RGBMatrixEmulator/issues/9#issuecomment-922869679
-    if os.path.exists(logo_path) and driver.is_hardware() and PIL_LOADED:
+    if os.path.exists(logo_path) and driver.is_hardware():
         logo = Image.open(logo_path)
         matrix.SetImage(logo.convert("RGB"))
         logo.close()

renderers/offday.py

@@ -2,13 +2,7 @@
 
 import time
 
-try:
-    from PIL import Image
-
-    PIL_LOADED = True
-except:
-
-    PIL_LOADED = False
+from PIL import Image
 
 from data.time_formats import TIME_FORMAT_12H
 from data.config.color import Color
@@ -44,10 +38,9 @@ def __render_clock(canvas, layout, colors, time_format):
 
 def __render_weather(canvas, layout, colors, weather):
     if weather.available():
-        if PIL_LOADED:
-            image_file = weather.icon_filename()
-            weather_icon = Image.open(image_file)
-            __render_weather_icon(canvas, layout, colors, weather_icon)
+        image_file = weather.icon_filename()
+        weather_icon = Image.open(image_file)
+        __render_weather_icon(canvas, layout, colors, weather_icon)
         __render_weather_text(canvas, layout, colors, weather.conditions, "conditions")
         __render_weather_text(canvas, layout, colors, weather.temperature_string(), "temperature")
         __render_weather_text(canvas, layout, colors, weather.wind_speed_string(), "wind_speed")

requirements.txt

@@ -1,17 +1,6 @@
 feedparser==6.0.10
 MLB_StatsAPI>=1.6.1
-# PIL is affected by CVE-2023-4863
-#   https://nvd.nist.gov/vuln/detail/CVE-2023-4863
-# 
-# The vulnerability is patched in Pillow >= 10.0.1. This version does not support Python 3.7 due to this version being end-of-life.
-# Python 3.7.3 is the default Python version for Raspbian / Raspberry Pi OS, and upgrading Python versions is difficult for non-technical users.
-#
-# Therefore, addressing the CVE at this time would be a breaking change for most users without an easy upgrade path to Python 3.8+.
-# 
-# Dependabot PR:
-#   https://github.com/MLB-LED-Scoreboard/mlb-led-scoreboard/pull/502
-#
-# Pillow==9.3.1
+Pillow>=10.0.1
 pyowm==3.3.0
 RGBMatrixEmulator>=0.8.4
 tzlocal==4.2

utils.py

@@ -126,6 +126,9 @@ def args():
         help="Force using emulator mode over default matrix display.",
         const=True
     )
+    parser.add_argument(
+        "--drop-privileges", action="store_true", help="Force the matrix driver to drop root privileges after setup."
+    )
     return parser.parse_args()
 
 
@@ -148,6 +151,7 @@ def led_matrix_options(args):
     options.scan_mode = args.led_scan_mode
     options.pwm_lsb_nanoseconds = args.led_pwm_lsb_nanoseconds
     options.led_rgb_sequence = args.led_rgb_sequence
+    options.drop_privileges = args.drop_privileges
 
     try:
         options.pixel_mapper_config = args.led_pixel_mapper

version.py

@@ -1,5 +1,5 @@
 SCRIPT_NAME = "MLB LED Scoreboard"
-SCRIPT_VERSION = "7.0.1"
+SCRIPT_VERSION = "8.0.0"
 
 
 if __name__ == "__main__":