update Dockerfile

Author: mythz

DOCKER.md

@@ -1,8 +1,8 @@
 # Running llms-py in Docker
 
 The Docker image bundles llms.py with everything its extensions need — Python,
-`bun`, and the .NET SDK — so nothing has to be installed on the host beyond
-Docker itself.
+`bun`, the .NET SDK and `typst` — so nothing has to be installed on the host
+beyond Docker itself.
 
 - [Quick start](#quick-start) — the one-line installer
 - [What the installer does](#what-the-installer-does)
@@ -290,6 +290,137 @@ It exits non-zero if any check fails, so it works as a release gate. If a
 provider API key happens to be in your environment it also runs a live
 `llms --check` against that provider.
 
+## What's in the image
+
+| Tool | Used for |
+| --- | --- |
+| Python 3.11 + llms-py | the CLI and server |
+| `bun` / `bunx` | JavaScript extensions and tools |
+| .NET SDK 10 | running C# code |
+| `typst` | the PDF Studio extension (`.typ` templates → PDF) |
+| `ffmpeg` | audio conversion for voice input |
+| `git` | installing extensions from a repo |
+
+Each is verified at build time, so a broken image fails to build rather than
+failing on your first request.
+
+## Voice input
+
+The microphone lives in the browser, not the container — the chat UI records
+audio and POSTs it to `/transcribe`, so no device passthrough is needed. The
+container's only job is turning that audio into text.
+
+The voice extension's `api` mode needs nothing installed, so it works in the
+container out of the box. Add a key and restart:
+
+```bash
+echo 'GROQ_API_KEY=gsk_...' >> ~/.llms/.env
+llms restart
+```
+
+Configure the provider and model under `defaults` in `~/.llms/llms.json`:
+
+```json
+{
+  "defaults": {
+    "voice": {
+      "provider": "groq",
+      "model": "whisper-large-v3",
+      "language": "en"
+    }
+  }
+}
+```
+
+Or point it at a local speech-to-text server — no key required. From inside the
+container the host is `host.docker.internal`:
+
+```json
+{
+  "defaults": {
+    "voice": {
+      "url": "http://host.docker.internal:8001/v1/audio/transcriptions",
+      "model": "Systran/faster-whisper-small"
+    }
+  }
+}
+```
+
+This is the same configuration llms.py uses everywhere, not something
+Docker-specific — see [Voice Input](https://llmspy.org/docs/features/voice-input)
+for every setting, the `LLMS_TRANSCRIBE_*` environment overrides, and the other
+modes.
+
+Check which mode was selected:
+
+```bash
+llms restart --verbose && llms logs | grep -i voice
+```
+
+```
+Using api for voice: groq [llms.json] model=whisper-large-v3 [llms.json]
+```
+
+The image also ships `ffmpeg`, which the `voxtype` and `transcribe` modes need
+to convert the browser's webm recording. `voxtype` requires a graphical desktop
+session so it never applies in a container; `transcribe` is available if you
+mount your own script at `/usr/local/bin/transcribe`.
+
+### The microphone button is missing
+
+Browsers only expose `getUserMedia` in a **secure context**: HTTPS, or
+`http://localhost` / `http://127.0.0.1`. The default `llms up` binds to
+`127.0.0.1`, so it works.
+
+If you set `LLMS_BIND=0.0.0.0` and browse to `http://192.168.x.x:8000`, the
+browser silently withholds the microphone API and no button appears — nothing to
+do with Docker or your configuration. Reach it over an SSH tunnel
+(`ssh -L 8000:localhost:8000 host`) or put it behind a TLS-terminating reverse
+proxy.
+
+### Logging
+
+`llms` reads two environment variables, both off by default:
+
+| Variable | Effect |
+| --- | --- |
+| `VERBOSE=1` | request/response logging (same as `--verbose`) |
+| `DEBUG=1` | debug logging |
+
+The `llms` command turns them into container env vars for you:
+
+```bash
+llms up --verbose          # request logging
+llms up --debug            # verbose + debug
+llms up --debug -f         # ...and follow the logs
+llms restart --debug       # turn it on for a running server
+llms --debug ls            # one-shot commands too
+llms logs                  # last 200 lines
+llms logs -f               # follow
+llms status                # shows the active log level
+```
+
+To make it permanent, set `LLMS_VERBOSE=1` or `LLMS_DEBUG=1` in `~/.llms/config`.
+
+Running the image directly, pass them as normal env vars:
+
+```bash
+docker run -e VERBOSE=1 -e DEBUG=1 -p 8000:8000 \
+  -v ~/.llms:/home/llms/.llms ghcr.io/servicestack/llms:latest
+```
+
+### Passing your own environment variables
+
+Everything in `~/.llms/.env` is passed into the container, not just API keys, so
+it doubles as a place for any setting the container should see:
+
+```bash
+echo 'TZ=Australia/Perth' >> ~/.llms/.env
+llms restart
+```
+
+`llms setup` preserves anything there that isn't a provider API key.
+
 ## Troubleshooting
 
 **`docker: command not found` / daemon not running**
@@ -301,6 +432,35 @@ The image runs as UID 1000. If your UID is different, the installer detects it
 and adds `--user $(id -u):$(id -g)` — stored as `LLMS_DOCKER_USER_ARGS` in
 `~/.llms/config`. If you're running Docker by hand, add that flag yourself.
 
+**Voice input isn't working**
+Check which mode the extension picked:
+
+```bash
+llms restart --verbose && llms logs | grep -i voice
+```
+
+`Cannot use api - no voice provider configured` means no key and no `voice`
+section — add one. If no microphone button appears at all, see the secure-context
+note above.
+
+**PDF Studio isn't available**
+It disables itself when `typst` isn't on PATH. Check the image has it:
+
+```bash
+llms shell -c 'typst --version'
+```
+
+If it's missing you're on an image built before typst was added — run
+`llms update`.
+
+**C# code fails with "Couldn't find a valid ICU package"**
+An old image without `libicu`. Run `llms update`. As a stopgap you can disable
+globalization instead:
+
+```bash
+echo 'DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1' >> ~/.llms/.env && llms restart
+```
+
 **A provider is enabled but its models don't appear**
 Check the key is actually reaching the container:
 

Dockerfile

@@ -30,11 +30,29 @@ FROM python:3.11-slim
 # Set working directory
 WORKDIR /app
 
-# Install system dependencies and dotnet-sdk 10.0
+# System dependencies.
+#
+# .NET is installed below with dotnet-install.sh rather than from the Microsoft
+# apt repo, so apt never pulls in its native dependencies — they have to be
+# listed explicitly. Without libicu every .NET process aborts on startup with
+# "Couldn't find a valid ICU package installed on the system".
+#
+# libicu-dev is used instead of a pinned libicuNN so this keeps working when the
+# Python base image moves to a newer Debian (bookworm ships libicu72, trixie
+# libicu76). libssl, libstdc++, zlib and libgcc already come with the base image.
+# ffmpeg is needed by the voice extension, which converts the browser's webm
+# recording to 16 kHz mono WAV before handing it to a speech-to-text tool.
+# See https://learn.microsoft.com/dotnet/core/install/linux-debian
 RUN apt-get update && apt-get install -y --no-install-recommends \
     wget \
     git \
     ca-certificates \
+    xz-utils \
+    ffmpeg \
+    libicu-dev \
+    libgssapi-krb5-2 \
+    tzdata \
+    fonts-liberation \
     && rm -rf /var/lib/apt/lists/*
 
 # Install dotnet-sdk 10.0 using install script to bypass GPG SHA1 issues
@@ -46,11 +64,42 @@ RUN wget https://dot.net/v1/dotnet-install.sh -O dotnet-install.sh \
 # Set dotnet environment variables
 ENV DOTNET_ROOT=/usr/share/dotnet
 ENV PATH=$PATH:$DOTNET_ROOT
+ENV DOTNET_NOLOGO=1
+ENV DOTNET_CLI_TELEMETRY_OPTOUT=1
+
+# Install typst — required by the PDF Studio extension (llms/extensions/pdf),
+# which disables itself when `typst` isn't on PATH.
+#
+# The upstream musl builds are statically linked, so one binary works on any
+# distro. TARGETARCH is supplied automatically by buildx for each platform in
+# the multi-arch build. Bump TYPST_VERSION to update.
+ARG TYPST_VERSION=0.15.1
+ARG TARGETARCH
+RUN set -eux; \
+    case "${TARGETARCH:-amd64}" in \
+        amd64) typst_arch=x86_64 ;; \
+        arm64) typst_arch=aarch64 ;; \
+        *) echo "unsupported TARGETARCH: ${TARGETARCH}" >&2; exit 1 ;; \
+    esac; \
+    tarball="typst-${typst_arch}-unknown-linux-musl"; \
+    wget -qO /tmp/typst.tar.xz \
+        "https://github.com/typst/typst/releases/download/v${TYPST_VERSION}/${tarball}.tar.xz"; \
+    tar -xJf /tmp/typst.tar.xz -C /tmp; \
+    install -m 0755 "/tmp/${tarball}/typst" /usr/local/bin/typst; \
+    rm -rf /tmp/typst.tar.xz "/tmp/${tarball}"
 
 # Install bun
 COPY --from=bun /usr/local/bin/bun /usr/local/bin/bun
 RUN ln -s /usr/local/bin/bun /usr/local/bin/bunx
 
+# Fail the build if any runtime is broken rather than at the first user request.
+# `dotnet --version` starts the CLI, which is itself a .NET app, so it exercises
+# the globalization path that needs libicu.
+RUN dotnet --version \
+    && bun --version \
+    && typst --version \
+    && ffmpeg -version > /dev/null
+
 # Create a non-root user
 RUN useradd -m -u 1000 llms && \
     mkdir -p /home/llms/.llms && \
@@ -69,6 +118,13 @@ USER llms
 # Set home directory
 ENV HOME=/home/llms
 
+# Don't buffer stdout/stderr, so `docker logs` shows output as it happens
+ENV PYTHONUNBUFFERED=1
+
+# Warm the .NET first-run cache as the llms user so the first C# request
+# doesn't pay for it (and doesn't try to write to a read-only home).
+RUN dotnet --version > /dev/null 2>&1 || true
+
 # Expose default port
 EXPOSE 8000
 
@@ -81,5 +137,5 @@ HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
     CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000').read()" || exit 1
 
 # Default command - run server on port 8000
+# Set VERBOSE=1 and/or DEBUG=1 to get request logging (see DOCKER.md)
 CMD ["llms", "--serve", "8000"]
-

scripts/test-docker.sh

@@ -280,6 +280,35 @@ else
     fail "dotnet sdk present" "$OUT"
 fi
 
+# dotnet aborts at startup without libicu, so this catches a missing ICU package
+# even though `dotnet --version` above may have succeeded from a cached response.
+if OUT=$(docker run --rm --entrypoint sh "$IMAGE" -c \
+        'cd /tmp && dotnet --list-sdks' 2>&1 | tr -d '\r'); then
+    case "$OUT" in
+        *ICU*|*icu*) fail "dotnet globalization works (libicu present)" "$OUT" ;;
+        *) pass "dotnet globalization works (libicu present)" ;;
+    esac
+else
+    fail "dotnet globalization works (libicu present)" "$(printf '%s' "$OUT" | tail -2 | tr '\n' ' ')"
+fi
+
+if OUT=$(run_in typst --version 2>&1 | tr -d '\r'); then
+    pass "typst present ($OUT)"
+else
+    fail "typst present" "$OUT — the PDF Studio extension disables itself without it"
+fi
+
+# Compile a real document: proves typst runs and has usable fonts.
+if OUT=$(docker run --rm --entrypoint sh "$IMAGE" -c \
+        'cd /tmp && printf "= Hi\n" > t.typ && typst compile t.typ t.pdf && head -c4 t.pdf' 2>&1 | tr -d '\r'); then
+    case "$OUT" in
+        *%PDF*) pass "typst compiles a document to PDF" ;;
+        *)      fail "typst compiles a document to PDF" "unexpected output: $OUT" ;;
+    esac
+else
+    fail "typst compiles a document to PDF" "$(printf '%s' "$OUT" | tail -2 | tr '\n' ' ')"
+fi
+
 if run_in sh -c 'command -v llms' >/dev/null 2>&1; then
     pass "llms is on PATH"
 else
@@ -292,6 +321,13 @@ else
     fail "git present"
 fi
 
+# ffmpeg is what the voice extension uses to convert the browser's recording.
+if OUT=$(docker run --rm --entrypoint sh "$IMAGE" -c 'ffmpeg -version 2>&1 | head -1' 2>&1 | tr -d '\r'); then
+    pass "ffmpeg present (${OUT%% https*})"
+else
+    fail "ffmpeg present" "voice input needs it for the voxtype/transcribe modes"
+fi
+
 # -------------------------------------------------------------------- CLI ---
 
 section "CLI"
@@ -348,12 +384,44 @@ fi
 
 if OUT=$(docker run --rm -v "$WORKDIR/config:/home/llms/.llms" --entrypoint llms "$IMAGE" ls 2>&1); then
     pass "llms ls"
+    case "$OUT" in
+        *"PDF Studio disabled"*|*"typst not found"*)
+            fail "PDF Studio extension is enabled" "typst missing from the image" ;;
+    esac
     ENABLED=$(printf '%s' "$OUT" | grep -o 'enabled providers:.*' | head -1)
     [ -n "$ENABLED" ] && info "$ENABLED"
 else
     fail "llms ls" "$(printf '%s' "$OUT" | tail -3 | tr '\n' ' ')"
 fi
 
+# ---- voice extension -------------------------------------------------------
+# `api` mode should come up from a provider key alone...
+OUT=$(docker run --rm -v "$WORKDIR/config:/home/llms/.llms" -e GROQ_API_KEY=test-key-not-real \
+      -e VERBOSE=1 --entrypoint llms "$IMAGE" ls 2>&1 | tr -d '\r')
+case "$OUT" in
+    *"Using api for voice"*) pass "voice api mode activates from a provider key" ;;
+    *) fail "voice api mode activates from a provider key" \
+            "$(printf '%s' "$OUT" | grep -i voice | head -2 | tr '\n' ' ')" ;;
+esac
+
+# ...and a voice section in llms.json should choose the endpoint and model.
+mkdir -p "$WORKDIR/voice"
+cp "$WORKDIR/config/llms.json" "$WORKDIR/config/providers.json" "$WORKDIR/voice/" 2>/dev/null
+python3 - "$WORKDIR/voice/llms.json" <<'PYEOF' 2>/dev/null
+import json, sys
+cfg = json.load(open(sys.argv[1]))
+cfg["voice"] = {"url": "http://127.0.0.1:9/v1/audio/transcriptions", "model": "test-model"}
+json.dump(cfg, open(sys.argv[1], "w"), indent=4)
+PYEOF
+chmod -R 777 "$WORKDIR/voice" 2>/dev/null
+OUT=$(docker run --rm -v "$WORKDIR/voice:/home/llms/.llms" -e VERBOSE=1 \
+      --entrypoint llms "$IMAGE" ls 2>&1 | tr -d '\r')
+case "$OUT" in
+    *"model=test-model"*) pass "voice config is read from llms.json" ;;
+    *) fail "voice config is read from llms.json" \
+            "$(printf '%s' "$OUT" | grep -i voice | head -2 | tr '\n' ' ')" ;;
+esac
+
 # Optional live provider check when a key happens to be in the environment.
 LIVE_PROVIDER=""; LIVE_KEY=""
 for pair in "groq:GROQ_API_KEY" "openrouter:OPENROUTER_API_KEY" "google:GOOGLE_API_KEY" \