Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Don't run as root when building protobuf files #251

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Don't run as root when building protobuf files #251

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
25288b3
Don't run as root when building protobuf files
kommendorkapten Mar 7, 2024
81c37b9
Updated rust target as first merge was wrong
kommendorkapten Mar 7, 2024
9027b78
white space fixes
kommendorkapten Mar 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 14 additions & 6 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,23 @@ RUN set -ex && \
apt-get install -y --no-install-recommends \
python3-pip

# Install Python dev dependencies.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing locally, I had to move lines 14-18, the apt-get for curl and build-essential, above 10-12, or else it failed with E: Failed to fetch http://deb.debian.org/debian/pool/main/p/perl/perl-modules-5.32_5.32.1-4%2bdeb11u2_all.deb 404 Not Found [IP: 151.101.22.132 80] (not sure why). Can we swap the order?

COPY ./dev-requirements.txt /tmp/
RUN python3 -m pip install --upgrade pip && \
python3 -m pip install --requirement /tmp/dev-requirements.txt

# Install Rust cargo.
RUN set -ex && \
apt-get install -y --no-install-recommends \
curl \
build-essential
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | bash -s -- -y
ENV PATH="/root/.cargo/bin:${PATH}"

# Install Python dev dependencies.
COPY ./dev-requirements.txt /tmp/
RUN python3 -m pip install --upgrade pip && \
python3 -m pip install --requirement /tmp/dev-requirements.txt
# Switch user
ARG uid=1000
RUN useradd -u ${uid} -s /bin/sh -m builder
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got an error: adduser: number 561799 is not in 0..256000 range. Looks like https://stackoverflow.com/questions/41807026/cant-add-a-user-with-a-high-uid-in-docker-alpine would be the fix?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting, I'll try to take a look later this week.


USER builder
WORKDIR /home/builder

RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | bash -s -- -y
ENV PATH="/home/builder/.cargo/bin:${PATH}"
12 changes: 12 additions & 0 deletions Dockerfile.jsonschema
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,19 @@
# 3.18.2
FROM alpine@sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
RUN apk add --update protoc protobuf-dev go git

# Switch user
ARG uid=1000
RUN adduser -u ${uid} -S builder
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alpine does not use the "standard" useradd command.


RUN mkdir -p /home/builder
RUN chown builder /home/builder

USER builder
WORKDIR /home/builder

RUN go install github.com/chrusty/protoc-gen-jsonschema/cmd/[email protected]

# This is required to get the field_behavior.proto file
# NOTE: --filter=tree:0 performs a treeless clone; we do this to optimize cloning
# this otherwise relatively heavy repository.
Expand Down
57 changes: 27 additions & 30 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,72 +21,69 @@ RUST_ACTION ?= run -p sigstore-protobuf-specs-codegen
# generate all language protobuf code
all: go python typescript ruby jsonschema rust

CUSER=$(shell id -u ${USER})
CGROUP=$(shell id -g ${USER})
DOCKER_RUN=docker run --platform linux/amd64 -u ${CUSER}:${CGROUP} -v ${PWD}:/defs
loosebazooka marked this conversation as resolved.
Show resolved Hide resolved
DOCKER_BUILD=docker build --platform linux/amd64 --build-arg uid=${CUSER}

# generate Go protobuf code
go: docker-image
@echo "Generating go protobuf files"
docker run --platform linux/amd64 -v ${PWD}:/defs ${PROTOC_IMAGE} -d protos -l go --go-module-prefix github.com/sigstore/protobuf-specs/gen/pb-go
${DOCKER_RUN} ${PROTOC_IMAGE} \
-d /defs/protos \
-o /defs/gen/pb-go \
-l go --go-module-prefix github.com/sigstore/protobuf-specs/gen/pb-go

python: docker-image
@echo "Generating python protobuf files"
# we need to manually fix the PYTHONPATH due to: https://github.com/namely/docker-protoc/pull/356
docker run \
--platform linux/amd64 \
-v ${PWD}:/defs \
-e PYTHONPATH="/opt/mypy-protobuf/" \
${DOCKER_RUN} -e PYTHONPATH="/opt/mypy-protobuf/" \
--entrypoint bash ${PROTOC_IMAGE} \
-c "cd ./gen/pb-python/sigstore_protobuf_specs && protoc -I/opt/include -I../../../protos/ --python_betterproto_out=. ../../../protos/*.proto"
-c "cd /defs/gen/pb-python/sigstore_protobuf_specs && protoc -I/opt/include -I../../../protos/ --python_betterproto_out=. ../../../protos/*.proto"

typescript: docker-image
@echo "Generating javascript protobuf files"
docker run \
--platform linux/amd64 \
-v ${PWD}:/defs \
${PROTOC_IMAGE} \
-d protos -l typescript -o ./gen/pb-typescript/src/__generated__ --ts_opt oneof=unions,forceLong=string,env=node,exportCommonSymbols=false,outputPartialMethods=false,outputEncodeMethods=false,unrecognizedEnum=false
${DOCKER_RUN} ${PROTOC_IMAGE} \
-d /defs/protos -l typescript \
-o /defs/gen/pb-typescript/src/__generated__ \
--ts_opt oneof=unions,forceLong=string,env=node,exportCommonSymbols=false,outputPartialMethods=false,outputEncodeMethods=false,unrecognizedEnum=false

ruby: docker-image
@echo "Generating ruby protobuf files"
docker run \
--platform linux/amd64 \
-v ${PWD}:/defs \
--entrypoint bash ${PROTOC_IMAGE} \
-c "cd ./gen/pb-ruby && protoc -I/opt/include -I../../protos/ --ruby_out=lib ../../protos/*.proto"
${DOCKER_RUN} --entrypoint bash ${PROTOC_IMAGE} \
-c "cd /defs/gen/pb-ruby && protoc -I/opt/include -I../../protos/ --ruby_out=lib ../../protos/*.proto"

jsonschema: docker-image-jsonschema
@echo "Generating JSON schema files"
docker run \
-v ${PWD}:/defs \
--entrypoint sh \
${JSONSCHEMA_IMAGE} \
-c "cd defs/gen/jsonschema && ./jsonschema.sh -I ../../protos -I /googleapis/ --jsonschema_out=schemas ../../protos/*.proto"
${DOCKER_RUN} --entrypoint sh \
${JSONSCHEMA_IMAGE} \
-c "cd /defs/gen/jsonschema && ./jsonschema.sh -I ../../protos -I /home/builder/googleapis/ --jsonschema_out=schemas ../../protos/*.proto"

rust: docker-image
docker run \
--platform linux/amd64 \
-v ${PWD}:/defs \
-e "RUST_BACKTRACE=1" \
@echo "Generating rust protobuf files"
${DOCKER_RUN} -e "RUST_BACKTRACE=1" \
-e "CARGO_REGISTRY_TOKEN" \
--entrypoint bash ${PROTOC_IMAGE} \
-c "cd gen/pb-rust && cargo ${RUST_ACTION}"
-c "cd /defs/gen/pb-rust && cargo ${RUST_ACTION}"

# docker already does its own caching so we can attempt a build every time
.PHONY: docker-image
docker-image:
@echo "Building development docker image"
docker build -t ${PROTOC_IMAGE} .
${DOCKER_BUILD} -t ${PROTOC_IMAGE} .
loosebazooka marked this conversation as resolved.
Show resolved Hide resolved

# to recover from a situation where a stale layer exist, just purging the
# docker image via `make clean` is not enough. Re-building without layer
# cache is the only solution.
.PHONY: docker-image-no-cache
docker-image-no-cache:
@echo "Building development docker image with disabled cache"
docker build --no-cache -t ${PROTOC_IMAGE} .
${DOCKER_BUILD} --no-cache -t ${PROTOC_IMAGE} .

.PHONY: docker-image-jsonschema
docker-image-jsonschema:
@echo "Building docker image for generating JSON schema files"
docker build -t ${JSONSCHEMA_IMAGE} -f Dockerfile.jsonschema .
${DOCKER_BUILD} -t ${JSONSCHEMA_IMAGE} -f Dockerfile.jsonschema .

# clean up generated files (not working? try sudo make clean)
clean:
Expand All @@ -95,7 +92,7 @@ clean:
gen/pb-python/sigstore_protobuf_specs/dev \
gen/pb-python/sigstore_protobuf_specs/io \
gen/pb-rust/target
docker rmi -f ${PROTOC_IMAGE}
docker rmi -f ${PROTOC_IMAGE} ${JSONSCHEMA_IMAGE}

help:
docker run --pull always --platform linux/amd64 -v ${PWD}:/defs ${PROTOC_IMAGE}
2 changes: 1 addition & 1 deletion gen/jsonschema/jsonschema.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
set -u
set -e

protoc --plugin=/root/go/bin/protoc-gen-jsonschema \
protoc --plugin=/home/builder/go/bin/protoc-gen-jsonschema \
--jsonschema_opt=disallow_additional_properties \
--jsonschema_opt=enforce_oneof \
--jsonschema_opt=enums_as_strings_only \
Expand Down