Fix process-compose bug and implement readiness_probes to allow dependencies on plugin services

[jefft]

What happened?

Say I have a myapp service that requires a database. Databases take a few seconds to come up, so to avoid a race condition I must declare a process_healthy dependency on mariadb in process-compose.yml:

  version: "0.5"
  processes:
                                
    myapp:
      depends_on:
        mariadb:
          condition: process_healthy
      command: ./my_database_using_app

The condition possibiities are:

• process_completed - this is the default if unspecified and not what we want - myapp would stay 'Pending' on mariadb's completion forever.
• process_started - this would cause a race condition between myapp and mariadb starting.
• process_healthy - this is what we want.

The actual result is that, with condition: process_healthy, the service hangs, in state Pending:

$ devbox services ls
Services running in process-compose:
PID            NAME                NAMESPACE        STATUS         AGE          HEALTH        RESTARTS        EXIT CODE
1412201        mariadb             default          Running        1m21s        -             0               0
1412258        mariadb_logs        default          Running        1m20s        -             1               1
0              myapp               default          Pending        0s           -             0               0

There are two problems, for which I'll attach two PRs:

1. Devbox needs to implement readiness probes for plugin services in order for process_healthy to work. E.g. mariadb should have:

   readiness_probe:
     exec:
       command: "mariadb-admin ping"

2. The version of process-compose Devbox uses (v1.110.0) lacks a bugfix present in v1.116.0 which flags the lack of readiness_probe as an error. If devbox is recompiled to use 1.116.0 then:
   a) The myapp process state becomes Skipped with exit code 1, rather than Pending:

   $ devbox services ls
   Services running in process-compose:
   PID            NAME                NAMESPACE        STATUS         AGE          HEALTH        RESTARTS        EXIT CODE
   1527617        mariadb             default          Running        1m41s        -             0               0
   1527650        mariadb_logs        default          Running        1m40s        -             1               1
   0              myapp               default          Skipped        0s           -             0               1
   

   b) The process-compose log file (currently /tmp/process-compose-$USER.log, but see fix: redirect process-compose internal log to .devbox/process-compose.log #2914) now flags the error (ERR health dependency defined in 'myapp' but no health check exists in 'mariadb') where previously there was nothing:

   

I suggest both these need fixing: upgrade process-compose to better report this error case, and then implement readiness_probes so users can declare dependencies on plugin services.

Steps to reproduce

First apply #2909 and #2906 to get the mariadb plugin in a usable state.

Then:

cat > /tmp/mariadb_testhealth.sh <<'EOF'
#!/bin/bash -eu

if [[ -d /tmp/mariadb_testhealth ]]; then
        ( cd /tmp/mariadb_testhealth && devbox services down >/dev/null 2>&1 || : )
  rm -r /tmp/mariadb_testhealth
fi
mkdir /tmp/mariadb_testhealth
cd /tmp/mariadb_testhealth
devbox init
devbox add mariadb
echo "port = $(( 10000 + RANDOM % 50000))" >> devbox.d/mariadb/my.cnf
cat > my_database_using_app <<'EOF2'
#!/bin/bash
# Connect to MariaDBD via unix socket as root.
sudo mariadb --show-warnings -u root --socket "$MYSQL_UNIX_PORT" -e "select 1;"
EOF2
chmod +x ./my_database_using_app

cat > process-compose.yml <<'EOF2'
version: "0.5"

processes:

  myapp:
    depends_on:
      mariadb:
        condition: process_healthy
    command: ./my_database_using_app
EOF2
devbox services up -b
sleep 1
devbox services ls
EOF
bash /tmp/mariadb_testhealth.sh

Output for me ends with myapp in Pending:

Services running in process-compose:
PID            NAME                NAMESPACE        STATUS         AGE        HEALTH        RESTARTS        EXIT CODE
1552501        mariadb_logs        default          Running        0s         -             1               1
0              myapp               default          Pending        0s         -             0               0
1552424        mariadb             default          Running        1s         -             0               0

Command

No response

devbox.json

Devbox version

0.17.5 + #2906 + #2909

Nix version

nix (Nix) 2.34.6

What system does this bug occur on?

Linux (x86-64)

Debug logs

No response