we need to support the ssh approach one last time while the web serve…

…r is offline

CHANGELOG.md

@@ -3,17 +3,18 @@
 #### Highlights
  - Workers now have a webserver on them. This is one of the largest architectural changes to Pioreactor, and lays the foundation for better plugin and calibration cluster management, plus future features.
    - As an example, in your browser, you can enter the url: http://some_worker.local/unit_api/jobs/running to see a list of jobs running on a worker.
-   - Note: there is no user-interface for workers, just an api
+   - Note: there is no interactive user interface for workers, just an API
    - Previous actions that would involve SSHing from leader to a worker are replaced by web requests.
    - APIs that initiate a background task either return with the result, or return a task id that be be looked up at `/api/task_status/`.
 
 #### Bug fixes
- - fixed a issue where a calibrated OD reading would be mapped to max OD signal if it was too low.
+ - fixed an issue where a calibrated OD reading would be mapped to max OD signal if it was too low.
+ - fixed an issue where the Pioreactor UI would lock up if trying to create a new experiment with an existing name.
 
 #### Breaking changes
- - **Lots and lots of API changes**. You'll want to review them on our docs:
+ - **Lots and lots of API changes**. You'll want to review them on our docs: https://docs.pioreactor.com/developer-guide/web-ui-api
  - We no longer recommend the Raspberry Pi Zero (the original Zero, not the Zero 2.) since supporting a web server + pioreactor functions is too much for a single core.
- - `watchdog` is neutered. It use to try to "wake-up" a job, but this was flakey and causing more problems than it solved.
+ - `watchdog` is neutered. It used to try to "wake-up" a job, but this was flaky and causing more problems than it solved.
  - removed python library dependency `sh`
 
 #### Enhancements

pioreactor/cli/pios.py

@@ -10,6 +10,7 @@
 from __future__ import annotations
 
 from concurrent.futures import ThreadPoolExecutor
+from subprocess import run as run_ssh
 
 import click
 
@@ -296,9 +297,29 @@ def _thread_function(unit: str):
                 )
                 r.raise_for_status()
                 return True
-            except HTTPException as e:
-                logger.error(f"Unable to update {target} on {unit} due to server error: {e}.")
-                return False
+            except HTTPException:
+                # TODO: remove this code after next release
+                logger.debug("Falling back on SSH approach")
+                if source:
+                    result = run_ssh(
+                        ["ssh", resolve_to_address(unit), "pio", "update", target, "--source", source]
+                    )
+                elif version:
+                    result = run_ssh(
+                        ["ssh", resolve_to_address(unit), "pio", "update", target, "--version", version]
+                    )
+                else:
+                    raise ValueError("Must supply version or source")
+
+                if result.returncode != 0:
+                    logger.error(f"Unable to update {target} on {unit} due to server error.")
+                    return False
+                else:
+                    return True
+                #########
+
+                # logger.error(f"Unable to update {target} on {unit} due to server error: {e}.")
+                # return False
 
         with ThreadPoolExecutor(max_workers=len(units)) as executor:
             results = executor.map(_thread_function, units)

update_scripts/upcoming/update.sh

@@ -47,8 +47,8 @@ if [ "$HOSTNAME" != "$LEADER_HOSTNAME" ]; then
     chmod +x $UI_FOLDER/main.fcgi
 
     # install lighttp and set up mods
-    unzip lighttpd_packages.zip -d lighttpd_packages
-    dpkg -i lighttpd_packages/*.deb
+    unzip "$SCRIPT_DIR"/lighttpd_packages.zip -d "$SCRIPT_DIR"/lighttpd_packages
+    dpkg -i "$SCRIPT_DIR"/lighttpd_packages/*.deb
 
     # install our own lighttpd service
     cp -u "$SCRIPT_DIR"/lighttpd.service $SYSTEMD_DIR
@@ -70,12 +70,30 @@ if [ "$HOSTNAME" != "$LEADER_HOSTNAME" ]; then
     cp -u "$SCRIPT_DIR"/huey.service $SYSTEMD_DIR
     cp -u "$SCRIPT_DIR"/create_diskcache.service $SYSTEMD_DIR
 
-    systemctl enable huey.service
+    # test new services
+    huey_consumer -h
+    lighttpd -h
+    flask --help
+
+
+
+    # we need to restart the monitor jobs on the worker so that the new table (pio_job_metadata) and db exist first
+    systemctl restart [email protected]
+    sleep 5
+
     systemctl enable create_diskcache.service
     systemctl enable lighttpd.service
+    systemctl enable huey.service
 
+    systemctl start create_diskcache.service
+    systemctl start lighttpd.service
+    systemctl start huey.service
+
+    sleep 1
+
+    # test:
+    curl -LI localhost/unit_api/jobs/running
 
-    # don't need to start these services, as we do a pio update ui which should handle this.
 else
 
     CONFIG_FILE=/etc/lighttpd/conf-available/50-pioreactorui.conf
@@ -89,10 +107,3 @@ else
     fi
 
 fi
-
-
-
-# test services
-huey_consumer -h
-lighttpd -h
-flask --help