Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker-firefox needs privileged mode to run #187

Open
louhisuo opened this issue Jun 7, 2023 · 9 comments
Open

docker-firefox needs privileged mode to run #187

louhisuo opened this issue Jun 7, 2023 · 9 comments
Labels
bug Something isn't working

Comments

@louhisuo
Copy link

louhisuo commented Jun 7, 2023

Current Behavior

Running jlesage/docker-firefox image in TrueNAS Scale kubernetes cluster. My previous docker-firefox version (v23.03.1) deployed in that environment was running nicely without 'Privileged' mode permission but now we I tried to deploy the latest docker-firefox version (v23.05.2) it requires 'Privileged' mode to be set to be started and running without restarting continuously. I have seccomp profile defined for the pod in both releases so I suspect issue being elsewhere. I have attached 'kubectl logs' which prints container logs for both cases. Main difference what I can see that xvnc will not run in failed case.

Expected Behavior

docker-firefox pod starts and runs without enabling 'Privileged' mode.

Steps To Reproduce

Deploy docker-firefox container (v23.05.2) in TrueNAS Scale using its custom-app helm chart which automatically generates deployment manifest for the container and also injects configurations like seccomp profile to pod template. Try to run container without 'Privileged' mode, it starts but enters cycling pod restart.

Environment

  • OS: TrueNAS Scale
  • OS version: 22.12.2
  • CPU: AMD Ryzen 7 1700
  • Kubernetes (K3s) version: v1.25.3+k3s-9afcd6b9-dirty (bundled with TrueNAS)

Container creation

See Container inspect which shows Kubernetes deployment manifest generated when creating pod thru GUI.

Container log

**Failed container execution - Container log without 'Privileged' mode**
# kubectl logs -n ix-firefox firefox-custom-app-754f7bbbfb-xdchd
[init        ] container is starting...
[cont-env    ] loading container environment variables...
[cont-env    ] APP_NAME: loading...
[cont-env    ] APP_VERSION: loading...
[cont-env    ] DISPLAY: executing...
[cont-env    ] DISPLAY: terminated successfully.
[cont-env    ] DISPLAY: loading...
[cont-env    ] DOCKER_IMAGE_PLATFORM: loading...
[cont-env    ] DOCKER_IMAGE_VERSION: loading...
[cont-env    ] GTK2_RC_FILES: executing...
[cont-env    ] GTK2_RC_FILES: terminated successfully.
[cont-env    ] GTK2_RC_FILES: loading...
[cont-env    ] GTK_THEME: executing...
[cont-env    ] GTK_THEME: terminated successfully.
[cont-env    ] GTK_THEME: loading...
[cont-env    ] HOME: loading...
[cont-env    ] QT_STYLE_OVERRIDE: executing...
[cont-env    ] QT_STYLE_OVERRIDE: terminated successfully.
[cont-env    ] QT_STYLE_OVERRIDE: loading...
[cont-env    ] TAKE_CONFIG_OWNERSHIP: loading...
[cont-env    ] XDG_CACHE_HOME: loading...
[cont-env    ] XDG_CONFIG_HOME: loading...
[cont-env    ] XDG_DATA_HOME: loading...
[cont-env    ] XDG_RUNTIME_DIR: loading...
[cont-env    ] XDG_STATE_HOME: loading...
[cont-env    ] container environment variables initialized.
[cont-secrets] loading container secrets...
[cont-secrets] container secrets loaded.
[cont-init   ] executing container initialization scripts...
[cont-init   ] 10-certs.sh: executing...
[cont-init   ] 10-certs.sh: terminated successfully.
[cont-init   ] 10-check-app-niceness.sh: executing...
[cont-init   ] 10-check-app-niceness.sh: terminated successfully.
[cont-init   ] 10-cjk-font.sh: executing...
[cont-init   ] 10-cjk-font.sh: installing CJK font...
[cont-init   ] 10-cjk-font.sh: fetch http://dl-cdn.alpinelinux.org/alpine/edge/testing/x86_64/APKINDEX.tar.gz
[cont-init   ] 10-cjk-font.sh: fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz
[cont-init   ] 10-cjk-font.sh: fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/community/x86_64/APKINDEX.tar.gz
[cont-init   ] 10-cjk-font.sh: (1/1) Installing font-wqy-zenhei (0.9.45-r2)
[cont-init   ] 10-cjk-font.sh: Executing fontconfig-2.14.2-r3.trigger
[cont-init   ] 10-cjk-font.sh: Executing mkfontscale-1.2.2-r3.trigger
[cont-init   ] 10-cjk-font.sh: OK: 546 MiB in 146 packages
[cont-init   ] 10-cjk-font.sh: terminated successfully.
[cont-init   ] 10-clean-logmonitor-states.sh: executing...
[cont-init   ] 10-clean-logmonitor-states.sh: terminated successfully.
[cont-init   ] 10-clean-tmp-dir.sh: executing...
[cont-init   ] 10-clean-tmp-dir.sh: terminated successfully.
[cont-init   ] 10-fontconfig-cache-dir.sh: executing...
[cont-init   ] 10-fontconfig-cache-dir.sh: terminated successfully.
[cont-init   ] 10-init-users.sh: executing...
[cont-init   ] 10-init-users.sh: terminated successfully.
[cont-init   ] 10-nginx.sh: executing...
[cont-init   ] 10-nginx.sh: terminated successfully.
[cont-init   ] 10-openbox.sh: executing...
[cont-init   ] 10-openbox.sh: terminated successfully.
[cont-init   ] 10-set-tmp-dir-perms.sh: executing...
[cont-init   ] 10-set-tmp-dir-perms.sh: terminated successfully.
[cont-init   ] 10-vnc-password.sh: executing...
[cont-init   ] 10-vnc-password.sh: terminated successfully.
[cont-init   ] 10-web-data.sh: executing...
[cont-init   ] 10-web-data.sh: terminated successfully.
[cont-init   ] 10-x11-unix.sh: executing...
[cont-init   ] 10-x11-unix.sh: terminated successfully.
[cont-init   ] 10-xdg-runtime-dir.sh: executing...
[cont-init   ] 10-xdg-runtime-dir.sh: terminated successfully.
[cont-init   ] 15-install-pkgs.sh: executing...
[cont-init   ] 15-install-pkgs.sh: terminated successfully.
[cont-init   ] 55-check-snd.sh: executing...
[cont-init   ] 55-check-snd.sh: sound not supported: device /dev/snd not exposed to the container.
[cont-init   ] 55-check-snd.sh: terminated successfully.
[cont-init   ] 55-firefox.sh: executing...
[cont-init   ] 55-firefox.sh: terminated successfully.
[cont-init   ] 56-firefox-set-prefs-from-env.sh: executing...
[cont-init   ] 56-firefox-set-prefs-from-env.sh: terminated successfully.
[cont-init   ] 85-take-config-ownership.sh: executing...
[cont-init   ] 85-take-config-ownership.sh: terminated successfully.
[cont-init   ] 89-info.sh: executing...
    ╭――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――╮
    │                                                                      │
    │ Application:           Firefox                                       │
    │ Application Version:   113.0.2-r1                                    │
    │ Docker Image Version:  23.05.2                                       │
    │ Docker Image Platform: linux/amd64                                   │
    │                                                                      │
    ╰――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――╯
[cont-init   ] 89-info.sh: terminated successfully.
[cont-init   ] all container initialization scripts executed.
[init        ] giving control to process supervisor.
[supervisor  ] loading services...
[supervisor  ] loading service 'default'...
[supervisor  ] loading service 'logmonitor'...
[supervisor  ] service 'logmonitor' is disabled.
[supervisor  ] loading service 'logrotate'...
[supervisor  ] loading service 'app'...
[supervisor  ] loading service 'gui'...
[supervisor  ] loading service 'openbox'...
[supervisor  ] loading service 'xvnc'...
[supervisor  ] loading service 'nginx'...
[supervisor  ] loading service 'certsmonitor'...
[supervisor  ] service 'certsmonitor' is disabled.
[supervisor  ] all services loaded.
[supervisor  ] starting services...
[supervisor  ] starting service 'logrotate'...
[supervisor  ] started service 'logrotate'.
[supervisor  ] service 'logrotate' failed to be started: minimum uptime not met.
[supervisor  ] stopping service 'logrotate'...
[logrotate   ] reading config file /opt/base/etc/logrotate.conf
[logrotate   ] including /etc/cont-logrotate.d
[logrotate   ] reading config file nginx
[logrotate   ] acquired lock on state file /config/xdg/state/logrotate/logrotate.status
[logrotate   ] Reading state from file: /config/xdg/state/logrotate/logrotate.status
[logrotate   ] Allocating hash table for state file, size 64 entries
[logrotate   ] Creating new state
[logrotate   ] Creating new state
[logrotate   ] Handling 1 logs
[logrotate   ] rotating pattern: /config/log/nginx/access.log /config/log/nginx/error.log  monthly (6 rotations)
[logrotate   ] empty log files are not rotated, old logs are removed
[logrotate   ] considering log /config/log/nginx/access.log
[logrotate   ]   log /config/log/nginx/access.log does not exist -- skipping
[logrotate   ] considering log /config/log/nginx/error.log
[logrotate   ]   log /config/log/nginx/error.log does not exist -- skipping
[logrotate   ] not running postrotate script, since no logs were rotated
[supervisor  ] service 'logrotate' exited (with status 0).
[supervisor  ] waiting termination of logger thread of service 'logrotate'...
[supervisor  ] logger thread of service 'logrotate' successfully terminated.
[finish      ] executing container finish scripts...
[finish      ] all container finish scripts executed.



**Succesful container execution - Container log with 'Privileged' mode**
# kubectl logs -n ix-firefox firefox-custom-app-798586cd97-znb4w
[init        ] container is starting...
[cont-env    ] loading container environment variables...
[cont-env    ] APP_NAME: loading...
[cont-env    ] APP_VERSION: loading...
[cont-env    ] DISPLAY: executing...
[cont-env    ] DISPLAY: terminated successfully.
[cont-env    ] DISPLAY: loading...
[cont-env    ] DOCKER_IMAGE_PLATFORM: loading...
[cont-env    ] DOCKER_IMAGE_VERSION: loading...
[cont-env    ] GTK2_RC_FILES: executing...
[cont-env    ] GTK2_RC_FILES: terminated successfully.
[cont-env    ] GTK2_RC_FILES: loading...
[cont-env    ] GTK_THEME: executing...
[cont-env    ] GTK_THEME: terminated successfully.
[cont-env    ] GTK_THEME: loading...
[cont-env    ] HOME: loading...
[cont-env    ] QT_STYLE_OVERRIDE: executing...
[cont-env    ] QT_STYLE_OVERRIDE: terminated successfully.
[cont-env    ] QT_STYLE_OVERRIDE: loading...
[cont-env    ] TAKE_CONFIG_OWNERSHIP: loading...
[cont-env    ] XDG_CACHE_HOME: loading...
[cont-env    ] XDG_CONFIG_HOME: loading...
[cont-env    ] XDG_DATA_HOME: loading...
[cont-env    ] XDG_RUNTIME_DIR: loading...
[cont-env    ] XDG_STATE_HOME: loading...
[cont-env    ] container environment variables initialized.
[cont-secrets] loading container secrets...
[cont-secrets] container secrets loaded.
[cont-init   ] executing container initialization scripts...
[cont-init   ] 10-certs.sh: executing...
[cont-init   ] 10-certs.sh: terminated successfully.
[cont-init   ] 10-check-app-niceness.sh: executing...
[cont-init   ] 10-check-app-niceness.sh: terminated successfully.
[cont-init   ] 10-cjk-font.sh: executing...
[cont-init   ] 10-cjk-font.sh: installing CJK font...
[cont-init   ] 10-cjk-font.sh: fetch http://dl-cdn.alpinelinux.org/alpine/edge/testing/x86_64/APKINDEX.tar.gz
[cont-init   ] 10-cjk-font.sh: fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz
[cont-init   ] 10-cjk-font.sh: fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/community/x86_64/APKINDEX.tar.gz
[cont-init   ] 10-cjk-font.sh: (1/1) Installing font-wqy-zenhei (0.9.45-r2)
[cont-init   ] 10-cjk-font.sh: Executing fontconfig-2.14.2-r3.trigger
[cont-init   ] 10-cjk-font.sh: Executing mkfontscale-1.2.2-r3.trigger
[cont-init   ] 10-cjk-font.sh: OK: 546 MiB in 146 packages
[cont-init   ] 10-cjk-font.sh: terminated successfully.
[cont-init   ] 10-clean-logmonitor-states.sh: executing...
[cont-init   ] 10-clean-logmonitor-states.sh: terminated successfully.
[cont-init   ] 10-clean-tmp-dir.sh: executing...
[cont-init   ] 10-clean-tmp-dir.sh: terminated successfully.
[cont-init   ] 10-fontconfig-cache-dir.sh: executing...
[cont-init   ] 10-fontconfig-cache-dir.sh: terminated successfully.
[cont-init   ] 10-init-users.sh: executing...
[cont-init   ] 10-init-users.sh: terminated successfully.
[cont-init   ] 10-nginx.sh: executing...
[cont-init   ] 10-nginx.sh: terminated successfully.
[cont-init   ] 10-openbox.sh: executing...
[cont-init   ] 10-openbox.sh: terminated successfully.
[cont-init   ] 10-set-tmp-dir-perms.sh: executing...
[cont-init   ] 10-set-tmp-dir-perms.sh: terminated successfully.
[cont-init   ] 10-vnc-password.sh: executing...
[cont-init   ] 10-vnc-password.sh: terminated successfully.
[cont-init   ] 10-web-data.sh: executing...
[cont-init   ] 10-web-data.sh: terminated successfully.
[cont-init   ] 10-x11-unix.sh: executing...
[cont-init   ] 10-x11-unix.sh: terminated successfully.
[cont-init   ] 10-xdg-runtime-dir.sh: executing...
[cont-init   ] 10-xdg-runtime-dir.sh: terminated successfully.
[cont-init   ] 15-install-pkgs.sh: executing...
[cont-init   ] 15-install-pkgs.sh: terminated successfully.
[cont-init   ] 55-check-snd.sh: executing...
[cont-init   ] 55-check-snd.sh: sound device group 29.
[cont-init   ] 55-check-snd.sh: terminated successfully.
[cont-init   ] 55-firefox.sh: executing...
[cont-init   ] 55-firefox.sh: terminated successfully.
[cont-init   ] 56-firefox-set-prefs-from-env.sh: executing...
[cont-init   ] 56-firefox-set-prefs-from-env.sh: terminated successfully.
[cont-init   ] 85-take-config-ownership.sh: executing...
[cont-init   ] 85-take-config-ownership.sh: terminated successfully.
[cont-init   ] 89-info.sh: executing...
    ╭――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――╮
    │                                                                      │
    │ Application:           Firefox                                       │
    │ Application Version:   113.0.2-r1                                    │
    │ Docker Image Version:  23.05.2                                       │
    │ Docker Image Platform: linux/amd64                                   │
    │                                                                      │
    ╰――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――╯
[cont-init   ] 89-info.sh: terminated successfully.
[cont-init   ] all container initialization scripts executed.
[init        ] giving control to process supervisor.
[supervisor  ] loading services...
[supervisor  ] loading service 'default'...
[supervisor  ] loading service 'logmonitor'...
[supervisor  ] service 'logmonitor' is disabled.
[supervisor  ] loading service 'logrotate'...
[supervisor  ] loading service 'app'...
[supervisor  ] loading service 'gui'...
[supervisor  ] loading service 'openbox'...
[supervisor  ] loading service 'xvnc'...
[supervisor  ] loading service 'nginx'...
[supervisor  ] loading service 'certsmonitor'...
[supervisor  ] service 'certsmonitor' is disabled.
[supervisor  ] all services loaded.
[supervisor  ] starting services...
[supervisor  ] starting service 'logrotate'...
[supervisor  ] started service 'logrotate'.
[logrotate   ] reading config file /opt/base/etc/logrotate.conf
[logrotate   ] including /etc/cont-logrotate.d
[logrotate   ] reading config file nginx
[logrotate   ] acquired lock on state file /config/xdg/state/logrotate/logrotate.status
[logrotate   ] Reading state from file: /config/xdg/state/logrotate/logrotate.status
[logrotate   ] Allocating hash table for state file, size 64 entries
[logrotate   ] Creating new state
[logrotate   ] Creating new state
[logrotate   ] Handling 1 logs
[logrotate   ] rotating pattern: /config/log/nginx/access.log /config/log/nginx/error.log  monthly (6 rotations)
[logrotate   ] empty log files are not rotated, old logs are removed
[logrotate   ] considering log /config/log/nginx/access.log
[logrotate   ]   log /config/log/nginx/access.log does not exist -- skipping
[logrotate   ] considering log /config/log/nginx/error.log
[logrotate   ]   log /config/log/nginx/error.log does not exist -- skipping
[logrotate   ] not running postrotate script, since no logs were rotated
[supervisor  ] starting service 'xvnc'...
[supervisor  ] started service 'xvnc'.
[xvnc        ] Wed Jun  7 19:31:28 2023
[xvnc        ]  Config:      set rfbport(Int) to 5900
[xvnc        ]  Config:      set UseIPv6(Bool) to no(0)
[xvnc        ]  Config:      set rfbunixpath(String) to /tmp/vnc.sock
[xvnc        ]  Config:      set rfbunixmode(Int) to 0660
[xvnc        ]  Config:      set SecurityTypes(String) to None
[xvnc        ]  Config:      set desktop(String) to Firefox
[xvnc        ] Xvnc TigerVNC 1.13.1 - built May 16 2023 17:54:26
[xvnc        ] Copyright (C) 1999-2022 TigerVNC Team and many others (see README.rst)
[xvnc        ] See https://www.tigervnc.org for information on TigerVNC.
[xvnc        ] Underlying X server release 12014000
[xvnc        ]  vncext:      VNC extension running!
[xvnc        ]  Config:      set immutable AllowOverride
[xvnc        ]  vncext:      Listening for VNC connections on /tmp/vnc.sock (mode 0660)
[xvnc        ]  vncext:      Listening for VNC connections on all interface(s), port 5900
[xvnc        ]  VNCServerST: creating single-threaded server Firefox
[xvnc        ]  vncext:      created VNC server for screen 0
[supervisor  ] waiting for service 'xvnc' to be ready...
[supervisor  ] starting service 'openbox'...
[supervisor  ] started service 'openbox'.
[xvnc        ]  Selection:   Selection owner change for WM_S0
[supervisor  ] waiting for service 'openbox' to be ready...
[supervisor  ] starting service 'nginx'...
[supervisor  ] started service 'nginx'.
[nginx       ] Listening for HTTP connections on port 5800.
[supervisor  ] waiting for service 'nginx' to be ready...
[supervisor  ] starting service 'app'...
[supervisor  ] started service 'app'.
[app         ] Mozilla Firefox 113.0.2
[supervisor  ] all services started.
[supervisor  ] service 'logrotate' exited (with status 0).
[supervisor  ] waiting termination of logger thread of service 'logrotate'...
[supervisor  ] logger thread of service 'logrotate' successfully terminated.

Container inspect

--- This deployment manifest has 'privileged: true', which is only difference to failed case ---
# kubectl get deploy -n ix-firefox firefox-custom-app -o yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "49"
    meta.helm.sh/release-name: firefox
    meta.helm.sh/release-namespace: ix-firefox
  creationTimestamp: "2023-06-04T17:00:59Z"
  generation: 57
  labels:
    app: custom-app-8.0.17
    app.kubernetes.io/instance: firefox
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: custom-app
    app.kubernetes.io/version: 0.20.4145
    helm-revision: "38"
    helm.sh/chart: custom-app-8.0.17
    release: firefox
  name: firefox-custom-app
  namespace: ix-firefox
  resourceVersion: "27395729"
  uid: 1b057a77-45f5-4a05-b5fb-3f4b3efded33
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 3
  selector:
    matchLabels:
      app.kubernetes.io/instance: firefox
      app.kubernetes.io/name: custom-app
      pod.name: main
  strategy:
    type: Recreate
  template:
    metadata:
      annotations:
        rollme: buVN0
      creationTimestamp: null
      labels:
        app: custom-app-8.0.17
        app.kubernetes.io/instance: firefox
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/name: custom-app
        app.kubernetes.io/version: 0.20.4145
        helm-revision: "38"
        helm.sh/chart: custom-app-8.0.17
        pod.name: main
        release: firefox
    spec:
      automountServiceAccountToken: false
      containers:
      - env:
        - name: TZ
          value: Europe/Stockholm
        - name: UMASK
          value: "0022"
        - name: UMASK_SET
          value: "0022"
        - name: NVIDIA_VISIBLE_DEVICES
          value: void
        - name: PUID
          value: "1012"
        - name: USER_ID
          value: "1012"
        - name: UID
          value: "1012"
        - name: PGID
          value: "1012"
        - name: GROUP_ID
          value: "1012"
        - name: GID
          value: "1012"
        - name: DISPLAY_WIDTH
          value: "1920"
        - name: DISPLAY_HEIGHT
          value: "1080"
        - name: DARK_MODE
          value: "1"
        - name: ENABLE_CJK_FONT
          value: "1"
        - name: CONTAINER_DEBUG
          value: "1"
        image: docker.io/jlesage/firefox:v23.05.2
        imagePullPolicy: IfNotPresent
        name: firefox-custom-app
        ports:
        - containerPort: 5800
          name: main
          protocol: TCP
        resources:
          limits:
            cpu: "1"
            memory: 1Gi
          requests:
            cpu: 10m
            memory: 50Mi
        securityContext:
          allowPrivilegeEscalation: true
          capabilities:
            add:
            - CHOWN
            - SETUID
            - SETGID
            - FOWNER
            - DAC_OVERRIDE
            drop:
            - ALL
          privileged: true
          readOnlyRootFilesystem: false
          runAsGroup: 1012
          runAsNonRoot: false
          runAsUser: 0
          seccompProfile:
            type: RuntimeDefault
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /dev/shm
          name: devshm
        - mountPath: /config
          name: persist-list-0
        - mountPath: /downloads
          name: persist-list-1
        - mountPath: /shared
          name: shared
        - mountPath: /tmp
          name: tmp
        - mountPath: /var/logs
          name: varlogs
        - mountPath: /var/run
          name: varrun
      dnsConfig:
        options:
        - name: ndots
          value: "1"
      dnsPolicy: ClusterFirst
      enableServiceLinks: false
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext:
        fsGroup: 1012
        fsGroupChangePolicy: OnRootMismatch
        supplementalGroups:
        - 568
      serviceAccount: default
      serviceAccountName: default
      shareProcessNamespace: false
      terminationGracePeriodSeconds: 60
      volumes:
      - emptyDir:
          medium: Memory
        name: devshm
      - name: persist-list-0
        persistentVolumeClaim:
          claimName: firefox-custom-app-persist-list-0
      - hostPath:
          path: /mnt/nas-tank/seedbox
          type: ""
        name: persist-list-1
      - emptyDir: {}
        name: shared
      - emptyDir: {}
        name: tmp
      - emptyDir: {}
        name: varlogs
      - emptyDir:
          medium: Memory
        name: varrun
status:
  availableReplicas: 1
  conditions:
  - lastTransitionTime: "2023-06-04T17:00:59Z"
    lastUpdateTime: "2023-06-07T17:31:20Z"
    message: ReplicaSet "firefox-custom-app-798586cd97" has successfully progressed.
    reason: NewReplicaSetAvailable
    status: "True"
    type: Progressing
  - lastTransitionTime: "2023-06-07T17:31:24Z"
    lastUpdateTime: "2023-06-07T17:31:24Z"
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  observedGeneration: 57
  readyReplicas: 1
  replicas: 1
  updatedReplicas: 1

Anything else?

No response

@louhisuo louhisuo added the bug Something isn't working label Jun 7, 2023
@jlesage
Copy link
Owner

jlesage commented Jun 7, 2023

Do you get the same failure every time you (re)start the container ?
Is v23.03.1 the last version that work fine ?

@louhisuo
Copy link
Author

louhisuo commented Jun 7, 2023

Yes, I get same failure always when trying run it with setting 'privileged: false'.
I had been using v23.03.1 until this point and that was working fine and did not need 'Privileged' mode.

@jlesage
Copy link
Owner

jlesage commented Jun 7, 2023

Can you try other versions (https://github.com/jlesage/docker-firefox/releases) to see the latest one that works?
I don't see any change that would explain the sudden change of behaviour.

@louhisuo
Copy link
Author

louhisuo commented Jun 7, 2023

Sure, off course. Will probably do it tomorrow as it is pretty late for my timezone. What comes my mind is that perhaps nothing has really changed from docker-firefox side but perhaps they have tighten up security in TrueNAS / Truecharts side. So what I am wondering if docker-firefox container is e.g. getting sufficient Linux capabilities from Kubernetes.

This is what is currently given for containers

securityContext: allowPrivilegeEscalation: true capabilities: add: - CHOWN - SETUID - SETGID - FOWNER - DAC_OVERRIDE drop: - ALL privileged: true readOnlyRootFilesystem: false runAsGroup: 1012 runAsNonRoot: false runAsUser: 0 seccompProfile: type: RuntimeDefault

@louhisuo
Copy link
Author

louhisuo commented Jun 8, 2023

Made some testing and docker-firefox also failed with older releases, even with v23.03.1 which worked for me previously. However I made finding that if I add all 'Capabilities' listed for Baseline profile then docker-firefox container starts and run successfully. So with more relax capabilities also v23.05.2 starts and run.

These profiles are described in Kubernetes documentations under Pod Security Standards

My guess is that TruenNAS/Truechart folks may have tighten up security (which is good thing so kudos for them) when introducing their new common helm chart + custom-app chart which are used to automatically generate deployment manifest for docker-firefox and this caused issue.

@louhisuo
Copy link
Author

louhisuo commented Jun 9, 2023

Some further testing with Linux capabilities to better understand what exact capability set docker-firefox really needs to launch and run.

Truecharts are assigning following capabilities for every pod and deny rest (I did not try to remove any of capabilities from this list, even that is very easy thing to do).

  • CAP_CHOWN
  • CAP_SETUID
  • CAP_SETGID
  • CAP_FOWNER
  • CAP_DAC_OVERRIDE

However this is not sufficient for docker-firefox which requires two additional Linux capabilities to launch and run

  • CAP_KILL
  • CAP_NET_BIND_SERVICE

So it looks like there is no issue in docker-firefox itself which works like charm on Kubernetes when it gets sufficient Linux capabilities. However I would be grateful if you could consider to document which Linux capabilities docker-firefox really need to launch and run?

@jlesage
Copy link
Owner

jlesage commented Jun 9, 2023

However I would be grateful if you could consider to document which Linux capabilities docker-firefox really need to launch and run?

Documenting the minimal set of capabilities might be difficult (I guess we need to try to see what is needed/missing), especially considering that they might change after updates.
In general, the container is tested to work properly with capabilities provided by Docker.

@louhisuo
Copy link
Author

louhisuo commented Jun 9, 2023

Fully understood that this is probably difficult to document and may change in the future. I however wanted to make you aware that above seven Linux capabilities were needed to able to launch and run docker-firefox (v23.05.2) successfully on Kubernetes (others were dropped).

Kubernetes lists three different security profiles, including Linux capabilities in their documentation under Pod Security Standards (wondering if Docker has similar documentation?) From there I took 'Baseline' profile as input for my test.

When considering documentation, it is perhaps sufficient just to refer 'Pod Security Standards' Baseline profile as it provides superset of capabilities which docker-firefox needs. This is because Kubernetes documentation considers capabilities listed under Baseline profile "safe" i.e. preventing known privilege escalations. What do you think?

I will remove 'Bug' label from this issue now as this is definitely not bug in docker-firefox but merely Kubernetes configuration issue. But I will be more than happy to test things (as I have Kubernetes setup with docker-firefox) and also exchange ideas how to document this if you think it is worth to do :)

@louhisuo
Copy link
Author

louhisuo commented Jun 9, 2023

Oh, seems that I am not able to remove 'Bug' tag from this issue, my apologies. Please feel free to remove 'Bug' tag.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants