Skip to content

Commit

Permalink
fix: update builder management, add a default value to argument
Browse files Browse the repository at this point in the history
  • Loading branch information
seppzer0 committed Apr 12, 2024
1 parent caf740d commit 70ef5ce
Show file tree
Hide file tree
Showing 9 changed files with 267 additions and 206 deletions.
10 changes: 8 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
FROM python:3.12.0-alpine3.18
FROM python:3.12-alpine3.19

ARG TARGETARCH

COPY mdocker /opt/mdocker
COPY pyproject.toml /opt
Expand All @@ -10,10 +12,14 @@ COPY LICENSE.md /opt
ENV PYTHONPATH /opt
WORKDIR /opt

# cffi wheel in Python dependencies needs to be built manually for arm64
RUN if [ "${TARGETARCH}" == "arm64" ]; then \
apk update && apk add gcc musl-dev libffi-dev; fi

RUN python3 -m pip install --upgrade pip && \
python3 -m pip install poetry twine && \
python3 -m poetry config virtualenvs.create false && \
python3 -m poetry install --no-root && \
python3 -m pytest tests/

CMD [ "/bin/bash" ]
CMD [ "/bin/sh" ]
45 changes: 41 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,22 +8,26 @@ An easy-to-use wrapper for multi-platform Docker image builds.
- [Contents](#contents)
- [Description](#description)
- [Usage](#usage)
- [Package Compatibility](#package-compatibility)
- [Examples](#examples)
- [Installation](#installation)
- [From PyPI (recommended)](#from-pypi-recommended)
- [Local from source](#local-from-source)
- [No installation, direct run from source](#no-installation-direct-run-from-source)
- [License](#license)

## Description

mdocker is a simple wrapper over Docker Buildx, which can be used for easy image building targeted for multiple platforms.
mdocker is a simple wrapper over Docker Buildx, which can be used for convenient image builds targeted for multiple platforms.

This tool was originally designed as a workaround to a [limitation](https://github.com/docker/buildx/issues/59) that Buildx has with the `--load` parameter.

***Note**: There are, however, some [workarounds and progress](https://github.com/docker/roadmap/issues/371) towards this issue.*
> [!NOTE]
> *There are, however, some [workarounds and progress](https://github.com/docker/roadmap/issues/371) towards this issue.*
So, the amount of target platforms specified for this wrapper will be equal to the amount of tags generated in local cache.
The amount of target platforms specified for this wrapper is equal to the amount of tags generated in local cache.

E.g., if `linux/arm64` and `linux/amd64` were specified as target platforms for a `demo` image, you will get `demo:arm64` and `demo:amd64` built and stored within your local Docker cache.
E.g., if `linux/arm64` and `linux/amd64` were specified as target platforms for the `demo` image, you will get `demo:arm64` and `demo:amd64` built and stored within your local Docker cache.

## Usage

Expand All @@ -47,6 +51,35 @@ options:
--push push image to remote registry
```

## Package Compatibility

In some cases, packages for Docker images may not be available across all target platforms.
E.g., a package "some_package" may be available for amd64, but may not be available for arm64 platform.

Is cases like this, you can define custom installation rules within your Dockerfile itself.

An example of such custom rule usage can be found [here](Dockerfile#L16).

Docker's default back-end (BuildKit) provides built-in variables that can be used for determining target platform set for the current build.

The full list of these variables can be found [here](https://docs.docker.com/reference/dockerfile#automatic-platform-args-in-the-global-scope).

## Examples

You can try out mdocker with it's own Dockerfile!

The command below will build the `demo` image for both `amd64` and `arm64` target architectures:

```sh
python3 -m mdocker demo --platforms linux/amd64,linux/arm64
```

Optionally, you can specify paths to the build context and Dockerfile:

```sh
python3 -m mdocker demo --context . --file ./Dockerfile --platforms linux/amd64,linux/arm64
```

## Installation

### From PyPI (recommended)
Expand Down Expand Up @@ -74,3 +107,7 @@ export PYTHONPATH=$(pwd)
python3 -m poetry install --no-root
python3 mdocker <arguments>
```

## License

[MIT](#license)
5 changes: 3 additions & 2 deletions mdocker/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ def parse_args() -> argparse.Namespace:
parser.add_argument(
"--context",
dest="bcontext",
default=Path("."),
help="specify a path to build context"
)
parser.add_argument(
Expand All @@ -30,7 +31,7 @@ def parse_args() -> argparse.Namespace:
parser.add_argument(
"--platforms",
default=[
f"linux/{ccmd.launch('uname -m', get_output=True, quiet=True).replace('x86_64', 'amd64')}"
f"linux/{ccmd.launch('uname -m', get_output=True, quiet=True).replace('x86_64', 'amd64')}" # type: ignore
],
help="specify target platforms (e.g., --platforms linux/amd64,linux/arm64)"
)
Expand Down Expand Up @@ -64,7 +65,7 @@ def process_platforms(platforms: str | list[str]) -> list[str]:


def main(args: argparse.Namespace) -> None:
# for logs to show in order in various CI/CD / Build systems
# for logs to show in proper order in various CI/CD / Build systems
sys.stdout = io.TextIOWrapper(open(sys.stdout.fileno(), "wb", 0), write_through=True)
validate_env()
ImageBuilder(
Expand Down
11 changes: 5 additions & 6 deletions mdocker/interfaces/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,21 +6,20 @@ class IImageBuilder(ABC):
"""An interface for Docker image builder."""

@abstractmethod
def _builder_instance_clear(self) -> list[CompletedProcess | str]:
"""Clear the builder instance from the host machine."""
def _builder_instance_clear(self) -> list[CompletedProcess | str | None]:
"""Clear the builder instance."""
raise NotImplementedError()

@abstractmethod
def _builder_instance_create(self) -> CompletedProcess | str | None:
def _builder_instance_create(self, platform: str) -> CompletedProcess | str | None:
"""Create new builder instance."""
raise NotImplementedError()

@abstractmethod
def _gen_build_cmds(self) -> list[str]:
def _gen_build_cmds(self, platform: str) -> list[str]:
"""Generate a list of Docker Buildx commands."""
raise NotImplementedError()

@abstractmethod
def run(self) -> None:
"""Run the logic."""
"""Execute the logic."""
raise NotImplementedError()
70 changes: 31 additions & 39 deletions mdocker/models/image_builder.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,58 +16,50 @@ class ImageBuilder(BaseModel, IImageBuilder):
:param push: Flag to push built Docker images to registry.
"""

_instance: str = "multi_instance"
_instance: str = "multi-target"

name: str
dfile: Path
bcontext: Path
push: bool
platforms: list[str]

def _builder_instance_clear(self) -> list[CompletedProcess | str]:
# collect completed processes
c_pcs = []
c_pcs.append(ccmd.launch(f"docker buildx stop {self._instance}", quiet=True, dont_exit=True))
c_pcs.append(ccmd.launch(f"docker buildx rm {self._instance}", quiet=True, dont_exit=True))
c_pcs.append(
ccmd.launch(
"docker buildx create --use --name {} --platform {} --driver-opt network=host"\
.format(self._instance, self.platforms)
)
)
return c_pcs

def _builder_instance_create(self) -> CompletedProcess | str | None:
def _builder_instance_clear(self) -> list[CompletedProcess | str | None]:
return [
ccmd.launch(f"docker buildx stop {self._instance}", quiet=True, dont_exit=True),
ccmd.launch(f"docker buildx rm {self._instance}", quiet=True, dont_exit=True)
]

def _builder_instance_create(self, platform: str) -> CompletedProcess | str | None:
return ccmd.launch(
"docker buildx create --use --name {} --platform {} --driver-opt network=host"\
.format(self._instance, self.platforms)
.format(self._instance, ",".join(self.platforms))
)

def _gen_build_cmds(self) -> list[str]:
all_b_cmds = []
for platform in self.platforms:
# only <arch> value is used in tag extension
tag = f'{self.name}:{platform.split("/")[1]}'
# define build commands
b_cmds = [
"docker buildx build --no-cache --platform {} --load -f {} {} -t {}"\
.format(platform, self.dfile, self.bcontext, tag),
"docker buildx stop {}"\
.format(self._instance),
"docker buildx rm {}"\
.format(self._instance),
"docker buildx prune --force"
]
# optionally push image to registry
if self.push:
b_cmds.append(f"docker push {tag}")
all_b_cmds.extend(b_cmds)
return all_b_cmds
def _gen_build_cmds(self, platform: str) -> list[str]:
# only <arch> value is used in tag extension
tag = f'{self.name}:{platform.split("/")[1]}'
# define build commands
b_cmds = [
"docker buildx build --no-cache --platform {} --load -f {} {} -t {}"\
.format(platform, self.dfile, self.bcontext, tag),
"docker buildx stop {}"\
.format(self._instance),
"docker buildx rm {}"\
.format(self._instance),
"docker buildx prune --force"
]
# optionally push image to registry
if self.push:
b_cmds.append(f"docker push {tag}")
return b_cmds

def run(self) -> None:
msg.note("Launching multi-platform Docker image build..")
self._builder_instance_clear()
self._builder_instance_create()
[ccmd.launch(cmd) for cmd in self._gen_build_cmds()]
self._builder_instance_clear()
for platform in self.platforms:
self._builder_instance_clear()
self._builder_instance_create(platform)
[ccmd.launch(cmd) for cmd in self._gen_build_cmds(platform)]
self._builder_instance_clear()
msg.done("Multiarch Docker image build finished!")
Loading

0 comments on commit 70ef5ce

Please sign in to comment.