Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker improvements and arm64 image building. #703

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 0 additions & 39 deletions Dockerfile

This file was deleted.

64 changes: 64 additions & 0 deletions Dockerfile_amd64
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# ==================================================================================================
# === Build duckling library =======================================================================

FROM docker.io/haskell:8 AS builder

RUN apt-get update -qq \
&& apt-get install -qq -y --fix-missing --no-install-recommends \
libpcre3 libpcre3-dev build-essential pkg-config \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

RUN mkdir /log/
WORKDIR /duckling/

ADD . .

ENV LANG=C.UTF-8
RUN stack setup

# NOTE:`stack build` will use as many cores as are available to build
# in parallel. However, this can cause OOM issues as the linking step
# in GHC can be expensive. If the build fails, try specifying the
# '-j1' flag to force the build to run sequentially.
RUN stack install --test --no-run-tests

# Run tests. If tests are not built in above step, the main library would be rebuilt here as well.
RUN stack test --system-ghc

# ==================================================================================================
# === Copy prebuilt files to new image to greatly reduce the image size ============================

FROM docker.io/debian:buster AS base

ENV LANG C.UTF-8

RUN apt-get update -qq \
&& apt-get install -qq -y --no-install-recommends \
libpcre3 libgmp10 \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

COPY --from=builder /root/.local/bin/duckling-example-exe /usr/local/bin/

# Additional dependencies are required for the following test
RUN apt-get update -qq \
&& apt-get install -y --no-install-recommends \
curl \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

# Run simple test to check if requests are answered
RUN /bin/bash -c "duckling-example-exe -p 8008 --no-access-log --no-error-log & \
sleep 3 && \
response=$(curl -XPOST http://0.0.0.0:8008/parse --data 'locale=en_GB&text=42€') && \
label='{\"value\":42,\"type\":\"value\",\"unit\":\"EUR\"}' && \
if [[ '$response' == *'$label'* ]] ; then echo 'ok' ; exit 0 ; else echo 'error' ; exit -1 ; fi;"

# Uninstall curl and it's dependencies again
RUN apt-get remove -y curl && apt-get autoremove -y

# Use port 8008 instead of default port, because port 8000 is already in use by portainer
EXPOSE 8008

CMD ["duckling-example-exe", "-p", "8008", "--no-access-log", "--no-error-log"]
73 changes: 73 additions & 0 deletions Dockerfile_arm64
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# ==================================================================================================
# === Build duckling library =======================================================================

FROM docker.io/haskell:8 AS builder

RUN apt-get update -qq \
&& apt-get install -qq -y --fix-missing --no-install-recommends \
libpcre3 libpcre3-dev build-essential pkg-config wget \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

# https://docs.haskellstack.org/en/stable/maintainers/releases/#setting-up-an-arm-vm-for-releases
# Using apt-get install haskell-stack installs version 1.7.3 which is too old
RUN wget https://github.com/commercialhaskell/stack/releases/download/v2.1.3/stack-2.1.3-linux-aarch64.tar.gz
RUN tar -xf stack-*-linux-aarch64.tar.gz && \
cd stack-*-linux-aarch64/ && \
chmod +x stack && \
mv stack /usr/bin/
RUN stack --version

RUN mkdir /log/
WORKDIR /duckling/

ADD . .

ENV LANG=C.UTF-8
RUN stack setup --compiler ghc-8.10.7 --system-ghc

# NOTE:`stack build` will use as many cores as are available to build
# in parallel. However, this can cause OOM issues as the linking step
# in GHC can be expensive. If the build fails, try specifying the
# '-j1' flag to force the build to run sequentially.
RUN stack install -j`nproc` --compiler ghc-8.10.7 --system-ghc --test --no-run-tests

# Run tests. If tests are not built in above step, the main library would be rebuilt here as well.
RUN stack test --compiler ghc-8.10.7 --system-ghc

# ==================================================================================================
# === Copy prebuilt files to new image to greatly reduce the image size ============================

FROM docker.io/debian:buster AS base

ENV LANG C.UTF-8

RUN apt-get update -qq \
&& apt-get install -qq -y --no-install-recommends \
libpcre3 libgmp10 libnuma1 \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

COPY --from=builder /root/.local/bin/duckling-example-exe /usr/local/bin/

# Additional dependencies are required for the following test
RUN apt-get update -qq \
&& apt-get install -y --no-install-recommends \
curl \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

# Run simple test to check if requests are answered
RUN /bin/bash -c "duckling-example-exe -p 8008 --no-access-log --no-error-log & \
sleep 5 && \
response=$(curl -XPOST http://0.0.0.0:8008/parse --data 'locale=en_GB&text=42€') && \
label='{\"value\":42,\"type\":\"value\",\"unit\":\"EUR\"}' && \
if [[ '$response' == *'$label'* ]] ; then echo 'ok' ; exit 0 ; else echo 'error' ; exit -1 ; fi;"

# Uninstall curl and it's dependencies again
RUN apt-get remove -y curl && apt-get autoremove -y

# Use port 8008 instead of default port, because port 8000 is already in use by portainer
EXPOSE 8008

CMD ["duckling-example-exe", "-p", "8008", "--no-access-log", "--no-error-log"]
16 changes: 16 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,22 @@ See `exe/ExampleMain.hs` for an example on how to integrate Duckling in your
project.
If your backend doesn't run Haskell or if you don't want to spin your own Duckling server, you can directly use [wit.ai](https://wit.ai)'s built-in entities.

### Docker

Alternatively build _duckling_ as a docker service:
```bash
docker build -t duckling_amd64 -f Dockerfile_amd64 .
docker run -p 8008:8008 --rm duckling_amd64

# Run request in another console window
# Note that the port is different, because the default 8000 port is already used by portainer
curl -XPOST http://0.0.0.0:8008/parse --data 'locale=en_GB&text=tomorrow at eight'

# It's also possible to cross-build the image for a different architecture, but this might take some hours.
sudo docker run --security-opt label=disable --rm --privileged multiarch/qemu-user-static --reset -p yes
docker buildx build --platform=linux/arm64 -t duckling_arm64 -f Dockerfile_arm64 .
```

## Supported dimensions

Duckling supports many languages, but most don't support all dimensions yet
Expand Down