Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Stages #173

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
5266f1b
feat: Stages
gmpinder May 1, 2024
f1e1d1f
Add support for from-file
gmpinder May 1, 2024
f46fe77
Add shell property; add ARGs to make modules work; mount yq; adjust r…
gmpinder May 2, 2024
df1ee73
Source from correct spot for scripts
gmpinder May 2, 2024
c0c8455
Add --link to copy
gmpinder May 2, 2024
5af71bc
Add setup script to install programs compatible with bluebuild modules
gmpinder May 2, 2024
e52fcfe
Forgot file
gmpinder May 2, 2024
1fb9029
if statement error
gmpinder May 2, 2024
d53d609
Add debian variant
gmpinder May 2, 2024
6d19afb
Install gnu grep for alpine
gmpinder May 2, 2024
f24442b
cp yq over
gmpinder May 2, 2024
44bacfa
Add stages.yml test
gmpinder May 2, 2024
f6e4d0b
Set final image name to recipe name to make it easier to follow in th…
gmpinder May 2, 2024
1c78023
Add COPYs to get docker to run the stages
gmpinder May 2, 2024
dcef0c9
Properly check from-file for modules inside of a stage
gmpinder May 2, 2024
7fb1ce7
Add cache bust via no-cache
gmpinder May 3, 2024
7101e97
Add some comments to better signify what each stage is for
gmpinder May 3, 2024
92b63de
Add another comment
gmpinder May 3, 2024
19f72d1
Make from optional
gmpinder May 3, 2024
9dd35e5
Add circular dependency check
gmpinder May 4, 2024
a33fc44
Don't build bluebuild in integration test
gmpinder May 4, 2024
6f330de
Add module details
gmpinder May 4, 2024
6e7cb08
Add feature gates
gmpinder May 4, 2024
e2d55a5
Create different feature builds
gmpinder May 5, 2024
09e17a0
Merge branch 'main' into 108-feat-allow-adding-stages-to-run-builds-f…
gmpinder May 5, 2024
34c7c89
Merge branch 'main' into 108-feat-allow-adding-stages-to-run-builds-f…
gmpinder May 5, 2024
c74f219
Merge branch 'main' into 108-feat-allow-adding-stages-to-run-builds-f…
gmpinder May 10, 2024
67b7103
Change to using from for stage to make it clearer
gmpinder May 10, 2024
c26d050
Make templated file look nicer
gmpinder May 10, 2024
918d4cd
Update template/templates/modules/copy/README.md
gmpinder May 10, 2024
74b6323
Merge branch 'main' into 108-feat-allow-adding-stages-to-run-builds-f…
gmpinder May 14, 2024
4711cf1
Merge branch 'main' into 108-feat-allow-adding-stages-to-run-builds-f…
gmpinder May 14, 2024
581151f
Merge branch 'main' into 108-feat-allow-adding-stages-to-run-builds-f…
gmpinder May 14, 2024
b8a2988
Don't use --link
gmpinder May 14, 2024
70bcd71
Don't fail if we can't detect distro
gmpinder May 15, 2024
29ecc27
Always cp yq
gmpinder May 15, 2024
b7accaf
Add podman build job
gmpinder May 15, 2024
b18b1d0
Merge branch 'main' into 108-feat-allow-adding-stages-to-run-builds-f…
gmpinder May 15, 2024
9494317
Merge branch 'main' into 108-feat-allow-adding-stages-to-run-builds-f…
gmpinder May 15, 2024
f82d696
Merge branch 'main' into 108-feat-allow-adding-stages-to-run-builds-f…
gmpinder May 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .rusty-hook.toml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[hooks]
pre-commit = "cargo fmt --check && cargo test && cargo clippy -- -D warnings"
pre-commit = "cargo fmt --check && cargo test --all-features && cargo clippy --all-features -- -D warnings"

[logging]
verbose = true
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,8 @@ uuid.workspace = true

[features]
default = []
stages = ["blue-build-recipe/stages"]
copy = ["blue-build-recipe/copy"]

[dev-dependencies]
rusty-hook = "0.11.2"
Expand Down
44 changes: 32 additions & 12 deletions Earthfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
VERSION 0.8
PROJECT blue-build/cli

IMPORT github.com/blue-build/earthly-lib/cargo AS cargo
IMPORT github.com/earthly/lib/rust AS rust

ARG --global IMAGE=ghcr.io/blue-build/cli

Expand All @@ -21,17 +21,29 @@ build:

lint:
FROM +common
DO cargo+LINT
DO rust+CARGO --args="clippy -- -D warnings"
DO rust+CARGO --args="clippy --all-features -- -D warnings"
DO rust+CARGO --args="clippy --no-default-features -- -D warnings"

test:
FROM +common
DO cargo+TEST
DO rust+CARGO --args="test -- --show-output"
DO rust+CARGO --args="test --all-features -- --show-output"
DO rust+CARGO --args="test --no-default-features -- --show-output"

install:
FROM +common
ARG --required BUILD_TARGET

DO cargo+BUILD_RELEASE --BUILD_TARGET=$BUILD_TARGET
DO rust+CARGO --args="build --release --target $BUILD_TARGET" --output="$BUILD_TARGET/release/[^\./]+"

SAVE ARTIFACT target/$BUILD_TARGET/release/bluebuild

install-all-features:
FROM +common
ARG --required BUILD_TARGET

DO rust+CARGO --args="build --all-features --release --target $BUILD_TARGET" --output="$BUILD_TARGET/release/[^\./]+"

SAVE ARTIFACT target/$BUILD_TARGET/release/bluebuild

Expand All @@ -47,7 +59,7 @@ common:
COPY --keep-ts --dir .git/ /app
RUN touch build.rs

DO cargo+INIT
DO rust+INIT --keep_fingerprints=true

build-scripts:
FROM alpine
Expand All @@ -65,8 +77,6 @@ blue-build-cli:
FROM $BASE_IMAGE
LABEL org.opencontainers.image.base.name="$BASE_IMAGE"

BUILD +install --BUILD_TARGET="x86_64-unknown-linux-gnu"

RUN dnf -y install dnf-plugins-core \
&& dnf config-manager --add-repo https://download.docker.com/linux/fedora/docker-ce.repo \
&& dnf install --refresh -y \
Expand All @@ -84,7 +94,7 @@ blue-build-cli:

COPY +cosign/cosign /usr/bin/cosign

COPY (+install/bluebuild --BUILD_TARGET="x86_64-unknown-linux-gnu") /usr/bin/bluebuild
DO --pass-args +INSTALL --OUT_DIR="/usr/bin/" --BUILD_TARGET="x86_64-unknown-linux-gnu"

RUN mkdir -p /bluebuild
WORKDIR /bluebuild
Expand All @@ -97,14 +107,12 @@ blue-build-cli-alpine:
FROM $BASE_IMAGE
LABEL org.opencontainers.image.base.name="$BASE_IMAGE"

BUILD +install --BUILD_TARGET="x86_64-unknown-linux-musl"

RUN apk update && apk add buildah podman skopeo fuse-overlayfs jq

LABEL org.opencontainers.image.base.digest="$(skopeo inspect "docker://$BASE_IMAGE" | jq -r '.Digest')"

COPY +cosign/cosign /usr/bin/cosign
COPY (+install/bluebuild --BUILD_TARGET="x86_64-unknown-linux-musl") /usr/bin/bluebuild
DO --pass-args +INSTALL --OUT_DIR="/usr/bin/" --BUILD_TARGET="x86_64-unknown-linux-musl"

RUN mkdir -p /bluebuild
WORKDIR /bluebuild
Expand All @@ -121,7 +129,7 @@ installer:

LABEL org.opencontainers.image.base.digest="$(skopeo inspect "docker://$BASE_IMAGE" | jq -r '.Digest')"

COPY (+install/bluebuild --BUILD_TARGET="x86_64-unknown-linux-musl") /out/bluebuild
DO --pass-args +INSTALL --OUT_DIR="/out/" --BUILD_TARGET="x86_64-unknown-linux-musl"
COPY install.sh /install.sh

CMD ["cat", "/install.sh"]
Expand All @@ -146,6 +154,18 @@ version:

SAVE ARTIFACT /version

INSTALL:
FUNCTION
ARG TAGGED="false"
ARG --required BUILD_TARGET
ARG --required OUT_DIR

IF [ "$TAGGED" = "true" ]
COPY (+install/bluebuild --BUILD_TARGET="$BUILD_TARGET") $OUT_DIR
ELSE
COPY (+install-all-features/bluebuild --BUILD_TARGET="$BUILD_TARGET") $OUT_DIR
END

SAVE_IMAGE:
FUNCTION
ARG SUFFIX=""
Expand Down
10 changes: 8 additions & 2 deletions integration-tests/Earthfile
Original file line number Diff line number Diff line change
Expand Up @@ -92,11 +92,17 @@ secureblue-base:
DO +GEN_KEYPAIR

legacy-base:
FROM +test-base
FROM ../+blue-build-cli-alpine
ENV BB_TEST_LOCAL_IMAGE=/etc/bluebuild/cli_test.tar.gz
ENV CLICOLOR_FORCE=1

RUN rm -fr /test
COPY ./mock-scripts/ /usr/bin/

WORKDIR /test
COPY ./legacy-test-repo /test

DO ../+INSTALL --OUT_DIR="/usr/bin/" --BUILD_TARGET="x86_64-unknown-linux-musl" --TAGGED="true"

DO +GEN_KEYPAIR

test-base:
Expand Down
7 changes: 7 additions & 0 deletions integration-tests/test-repo/files/scripts/bluebuild.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/usr/bin/env bash

set -euo pipefail

cargo install blue-build --debug --all-features --target x86_64-unknown-linux-gnu
mkdir -p /out/
mv $CARGO_HOME/bin/bluebuild /out/bluebuild
12 changes: 12 additions & 0 deletions integration-tests/test-repo/recipes/bluebuild.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
stages:
- name: blue-build
image: rust
modules:
- type: script
scripts:
- bluebuild.sh
modules:
- type: copy
from: blue-build
src: /out/bluebuild
dest: /usr/bin/bluebuild
23 changes: 21 additions & 2 deletions integration-tests/test-repo/recipes/recipe.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
name: cli/test
description: This is my personal OS image.
base-image: ghcr.io/ublue-os/silverblue-main
alt-tags:
- gts
- stable
description: This is my personal OS image.
base-image: ghcr.io/ublue-os/silverblue-main
image-version: 40
stages:
- from-file: stages.yml
modules:
- from-file: akmods.yml
- from-file: flatpaks.yml
Expand Down Expand Up @@ -36,3 +38,20 @@ modules:
- labels
snippets:
- RUN echo "This is a snippet" && ostree container commit

- type: copy
from: alpine-test
src: /test.txt
dest: /
- type: copy
from: ubuntu-test
src: /test.txt
dest: /
- type: copy
from: debian-test
src: /test.txt
dest: /
- type: copy
from: fedora-test
src: /test.txt
dest: /
32 changes: 32 additions & 0 deletions integration-tests/test-repo/recipes/stages.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
stages:
- name: ubuntu-test
from: ubuntu
modules:
- from-file: stages.yml
- name: debian-test
from: debian
modules:
- from-file: stages.yml
- name: fedora-test
from: fedora
modules:
- from-file: stages.yml
- name: alpine-test
from: alpine
modules:
- from-file: stages.yml
modules:
- type: files
files:
- usr: /usr
- type: script
scripts:
- example.sh
snippets:
- echo "test" > /test.txt
- type: test-module
- type: containerfile
containerfiles:
- labels
snippets:
- RUN echo "This is a snippet"
3 changes: 2 additions & 1 deletion modules.json
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
[
"https://raw.githubusercontent.com/blue-build/cli/main/template/templates/modules/containerfile/module.yml"
"https://raw.githubusercontent.com/blue-build/cli/main/template/templates/modules/containerfile/module.yml",
"https://raw.githubusercontent.com/blue-build/cli/main/template/templates/modules/copy/module.yml"
]
6 changes: 6 additions & 0 deletions recipe/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,17 @@ chrono = "0.4"
indexmap = { version = "2", features = ["serde"] }

anyhow.workspace = true
colored.workspace = true
log.workspace = true
serde.workspace = true
serde_yaml.workspace = true
serde_json.workspace = true
typed-builder.workspace = true

[features]
default = []
stages = []
copy = []

[lints]
workspace = true
4 changes: 4 additions & 0 deletions recipe/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,12 @@ pub mod akmods_info;
pub mod module;
pub mod module_ext;
pub mod recipe;
pub mod stage;
pub mod stages_ext;

pub use akmods_info::*;
pub use module::*;
pub use module_ext::*;
pub use recipe::*;
pub use stage::*;
pub use stages_ext::*;
Loading