Docker GPU install Error

[ediaz-biamp]

Hey I'm having trouble installing the docker GPU image. I tried it yesterday and it was fine, but I had to re-install due to some compatibility errors with Obsidian. Today, it won't install at all... Here's the error info, please let me know how to fix. Thanks!

[+] Building 152.1s (12/12) FINISHED docker:desktop-linux
=> [internal] load build definition from Dockerfile.gpu.server 0.0s
=> => transferring dockerfile: 1.92kB 0.0s
=> [internal] load metadata for docker.io/nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04 0.4s
=> [auth] nvidia/cuda:pull token for registry-1.docker.io 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [1/7] FROM docker.io/nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04@sha256:8f9dd0d09d3ad3900357a1cf7f887888b5b74056636cd6ef03c160c3cd4b1d95 0.0s
=> => resolve docker.io/nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04@sha256:8f9dd0d09d3ad3900357a1cf7f887888b5b74056636cd6ef03c160c3cd4b1d95 0.0s
=> [internal] load build context 1.3s
=> => transferring context: 79.93MB 1.3s
=> CACHED [2/7] RUN apt-get update && apt-get install -y --no-install-recommends wget curl unzip git python3 python3-pip libgl1 libglib2.0-0 0.0s
=> [3/7] RUN pip3 install --no-cache-dir torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 89.6s
=> [4/7] WORKDIR /app 0.1s
=> [5/7] COPY ../ ./ 0.1s
=> [6/7] RUN pip3 install --no-cache-dir -e . 35.0s
=> ERROR [7/7] RUN python -c 'from marker.models import load_all_models; load_all_models()' 25.6s

[7/7] RUN python -c 'from marker.models import load_all_models; load_all_models()':
25.02 Traceback (most recent call last):
25.02 File "", line 1, in
25.02 File "/usr/local/lib/python3.10/dist-packages/marker/models.py", line 74, in load_all_models
25.02 order = setup_order_model(device, dtype)
25.02 File "/usr/local/lib/python3.10/dist-packages/marker/models.py", line 61, in setup_order_model
25.02 model = load_order_model()
25.02 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/model.py", line 27, in load_model
25.02 model = OrderVisionEncoderDecoderModel.from_pretrained(checkpoint, config=config, torch_dtype=dtype)
25.02 File "/usr/local/lib/python3.10/dist-packages/transformers/models/vision_encoder_decoder/modeling_vision_encoder_decoder.py", line 376, in from_pretrained
25.02 return super().from_pretrained(pretrained_model_name_or_path, *model_args, **kwargs)
25.02 File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 4096, in from_pretrained
25.02 model = cls(config, *model_args, **model_kwargs)
25.02 File "/usr/local/lib/python3.10/dist-packages/transformers/models/vision_encoder_decoder/modeling_vision_encoder_decoder.py", line 199, in init
25.02 decoder = AutoModelForCausalLM.from_config(config.decoder)
25.02 File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 440, in from_config
25.02 return model_class._from_config(config, **kwargs)
25.02 File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 1543, in _from_config
25.02 model = cls(config, **kwargs)
25.02 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/decoder.py", line 495, in init
25.02 self.model = MBartOrderDecoderWrapper(config)
25.02 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/decoder.py", line 480, in init
25.02 self.decoder = MBartOrderDecoder(config)
25.02 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/decoder.py", line 294, in init
25.02 self.layers = nn.ModuleList([MBartOrderDecoderLayer(config) for _ in range(config.decoder_layers)])
25.02 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/decoder.py", line 294, in
25.02 self.layers = nn.ModuleList([MBartOrderDecoderLayer(config) for _ in range(config.decoder_layers)])
25.02 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/decoder.py", line 209, in init
25.02 self.self_attn = MBART_ATTENTION_CLASSES[config._attn_implementation](
25.02 KeyError: 'sdpa'
25.02 Loaded detection model vikp/surya_det3 on device cpu with dtype torch.float32
25.02 Loaded detection model vikp/surya_layout3 on device cpu with dtype torch.float32

---

Dockerfile.gpu.server:61

59 | RUN pip3 install --no-cache-dir -e .
60 |
61 | >>> RUN python -c 'from marker.models import load_all_models; load_all_models()'
62 |
63 | EXPOSE 8080

ERROR: failed to solve: process "/bin/sh -c python -c 'from marker.models import load_all_models; load_all_models()'" did not complete successfully: exit code: 1

View build details: docker-desktop://dashboard/build/desktop-linux/desktop-linux/rsc4z5o5u43zcoc69dk21y0b5
Unable to find image 'marker-api-gpu:latest' locally
docker: Error response from daemon: pull access denied for marker-api-gpu, repository does not exist or may require 'docker login'.
See 'docker run --help'.

[cahya-wirawan]

Try to replace "apt install python3-packaging" to "apt-get install -y --no-install-recommends python3-packaging" in Dockerfile.gpu.server.
or use following simpler Dockerfile that I got from https://github.com/adithya-s-k/marker-api/blob/980bc9396e6791c283f5dac24e631bcfb9469d6f/Dockerfile:

FROM pytorch/pytorch:2.3.1-cuda11.8-cudnn8-runtime

# Update package lists and install necessary packages
RUN apt-get update \
    && apt-get install -y ffmpeg libsm6 libxext6

# Assuming you have a setup.py or requirements.txt in your project directory
COPY . /app
WORKDIR /app

# Install your Python package (assuming it has a setup.py)
RUN pip3 install --no-cache-dir -e .

# Download models used by convert
#RUN python -c 'from marker.models import load_all_models; load_all_models()'

# Expose the desired port
EXPOSE 8000

# Run the server
CMD ["uvicorn", "server:app", "--host", "0.0.0.0" ,"--port", "8000", "--workers", "4"]

[ediaz-biamp]

Thanks for the quick reply. So I tried replacing that line in the Dockerfile, and I still obtained an error on install during the load_models part... I will attempt to install with the other Dockerfile now.
+] Building 278.8s (12/12) FINISHED docker:desktop-linux
=> [internal] load build definition from Dockerfile.gpu.server 0.0s
=> => transferring dockerfile: 1.95kB 0.0s
=> [internal] load metadata for docker.io/nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04 1.5s
=> [auth] nvidia/cuda:pull token for registry-1.docker.io 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> CACHED [1/7] FROM docker.io/nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04@sha256:8f9dd0d09d3ad3900357a1cf7f887888b5b74056636cd6ef03c160c3cd4b1d95 0.0s
=> => resolve docker.io/nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04@sha256:8f9dd0d09d3ad3900357a1cf7f887888b5b74056636cd6ef03c160c3cd4b1d95 0.0s
=> [internal] load build context 1.4s
=> => transferring context: 79.93MB 1.4s
=> [2/7] RUN apt-get update && apt-get install -y --no-install-recommends wget curl unzip git python3 python3-pip libgl1 libglib2.0-0 cu 124.8s
=> [3/7] RUN pip3 install --no-cache-dir torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 90.5s
=> [4/7] WORKDIR /app 0.1s
=> [5/7] COPY ../ ./ 0.2s
=> [6/7] RUN pip3 install --no-cache-dir -e . 36.0s
=> ERROR [7/7] RUN python -c 'from marker.models import load_all_models; load_all_models()' 25.7s

[7/7] RUN python -c 'from marker.models import load_all_models; load_all_models()':
24.96 Traceback (most recent call last):
24.96 File "", line 1, in
24.96 File "/usr/local/lib/python3.10/dist-packages/marker/models.py", line 74, in load_all_models
24.97 order = setup_order_model(device, dtype)
24.97 File "/usr/local/lib/python3.10/dist-packages/marker/models.py", line 61, in setup_order_model
24.97 model = load_order_model()
24.97 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/model.py", line 27, in load_model
24.97 model = OrderVisionEncoderDecoderModel.from_pretrained(checkpoint, config=config, torch_dtype=dtype)
24.97 File "/usr/local/lib/python3.10/dist-packages/transformers/models/vision_encoder_decoder/modeling_vision_encoder_decoder.py", line 376, in from_pretrained
24.97 return super().from_pretrained(pretrained_model_name_or_path, *model_args, **kwargs)
24.97 File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 4096, in from_pretrained
24.97 model = cls(config, *model_args, **model_kwargs)
24.97 File "/usr/local/lib/python3.10/dist-packages/transformers/models/vision_encoder_decoder/modeling_vision_encoder_decoder.py", line 199, in init
24.97 decoder = AutoModelForCausalLM.from_config(config.decoder)
24.97 File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 440, in from_config
24.97 return model_class._from_config(config, **kwargs)
24.97 File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 1543, in _from_config
24.97 model = cls(config, **kwargs)
24.97 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/decoder.py", line 495, in init
24.97 self.model = MBartOrderDecoderWrapper(config)
24.97 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/decoder.py", line 480, in init
24.97 self.decoder = MBartOrderDecoder(config)
24.97 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/decoder.py", line 294, in init
24.97 self.layers = nn.ModuleList([MBartOrderDecoderLayer(config) for _ in range(config.decoder_layers)])
24.97 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/decoder.py", line 294, in
24.97 self.layers = nn.ModuleList([MBartOrderDecoderLayer(config) for _ in range(config.decoder_layers)])
24.97 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/decoder.py", line 209, in init
24.97 self.self_attn = MBART_ATTENTION_CLASSES[config._attn_implementation](
24.97 KeyError: 'sdpa'
24.97 Loaded detection model vikp/surya_det3 on device cpu with dtype torch.float32
24.97 Loaded detection model vikp/surya_layout3 on device cpu with dtype torch.float32

---

Dockerfile.gpu.server:61

59 | RUN pip3 install --no-cache-dir -e .
60 |
61 | >>> RUN python -c 'from marker.models import load_all_models; load_all_models()'
62 |
63 | EXPOSE 8080

ERROR: failed to solve: process "/bin/sh -c python -c 'from marker.models import load_all_models; load_all_models()'" did not complete successfully: exit code: 1

View build details: docker-desktop://dashboard/build/desktop-linux/desktop-linux/qvja33iou5cm6lkmu3oyig3iy
Unable to find image 'marker-api-gpu:latest' locally
docker: Error response from daemon: pull access denied for marker-api-gpu, repository does not exist or may require 'docker login'.
See 'docker run --help'.

[cahya-wirawan]

Forget this Dockerfile.gpu.server, it has many bugs. Just use the Dockerfile I mentioned above.

[ediaz-biamp]

So I fixed my computer's permission issues, and I just tried the new Dockerfile. It seems the problem is persisting... Any tips on solving?
ERROR: failed to solve: process "/bin/sh -c python -c 'from marker.models import load_all_models; load_all_models()'" did not complete successfully: exit code: 1

[cahya-wirawan]

Probably you have the issue with transformers version as mentioned in VikParuchuri/marker#318
So, try to add transformers = "4.45.2" in pyproject.toml under [tool.poetry.dependencies] and build again.

[cahya-wirawan]

@ediaz-biamp I prepared a marker-api Dockerfile https://github.com/cahya-wirawan/marker-api/blob/old-docker/docker/Dockerfile.gpu-ubuntu20.server
You can also try its docker image at https://hub.docker.com/repository/docker/wirawan/marker-api.

And I created a simple web user interface for marker-api (which based also on the web ui of marker-api) https://hub.docker.com/repository/docker/wirawan/marker-ui
Feel free to try it.