Skip to content

engineering: migrate image build definitions from test-images to trident#644

Open
bfjelds wants to merge 15 commits into
mainfrom
user/bfjelds/mjolnir/sync-test-images
Open

engineering: migrate image build definitions from test-images to trident#644
bfjelds wants to merge 15 commits into
mainfrom
user/bfjelds/mjolnir/sync-test-images

Conversation

@bfjelds
Copy link
Copy Markdown
Member

@bfjelds bfjelds commented May 13, 2026 •
edited
Loading

Summary

Migrates all trident image build definitions from the test-images repo into the trident repo, so trident pipelines no longer depend on test-images.

Changes

Image definitions copied from test-images:

  • trident-testimage, trident-container-testimage, trident-container-installer, trident-verity-testimage
  • trident-installer (standard + split configs)

Build system updates:

  • Added missing ImageConfig entries to testimages.py
  • Ported signing improvements from test-images (sign.py)
  • Replaced sign_boot_artifacts with inject-files iteration approach

Pipeline changes:

  • �uild-image.yml now calls local Build-image-template.yml instead of test-images
  • Removed repository: test-images from all 11 pipeline files
  • Removed CI trigger on test-images changes

Companion PR

test-images cleanup: removes trident image definitions that moved here.

Testing

https://dev.azure.com/mariner-org/ECF/_build/results?buildId=1117244&view=results

bfjelds and others added 9 commits May 13, 2026 09:54
@bfjelds
feat: copy image definitions from test-images repo
2eafb0e 
Copy image definitions that trident pipelines need but were only in
the test-images repo:
- trident-testimage/
- trident-container-testimage/
- trident-container-installer/
- trident-verity-testimage/ (used by verity, usrverity, container variants)
- trident-installer/base/baseimg.yaml (standard installer)
- trident-installer/base/baseimg-split.yaml (split installer)
- trident-installer supporting files (services, config placeholders)

Rename trident-installer/base/post-install.sh to
post-install-direct-streaming.sh for the direct-streaming variant,
and use the standard post-install.sh from test-images for the
regular installer images.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bfjelds
feat: port signing improvements from test-images
cb7cec9 
Update builder/sign.py with newer signing code from test-images:
- Detect efikeygen --kernel flag support via --help probe
- Handle pesign --certficate typo in older versions
- Stage PE and verity artifacts through temp dirs with sudo cp
  for root-owned artifact handling
- Improve sign_verity_hash with temp staging and logging

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bfjelds
feat: add missing ImageConfig entries to testimages.py
056eb45 
Add image definitions previously only in test-images repo:
- trident-installer (standard, split, arm64)
- trident-container-installer
- trident-testimage (+ arm64)
- trident-verity-testimage
- trident-usrverity-testimage
- trident-container-verity-testimage
- trident-container-usrverity-testimage
- trident-container-testimage

Organize entries by category (installers, test images, direct
streaming, AZL installer, VM test images).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bfjelds
feat: use local build template instead of @test-images
dee49e1 
Replace the call to build-image.yml@test-images with the local
build-image-template.yml. This runs testimages.py from trident's
own tests/images/ instead of checking out the test-images repo.

Also add nss-tools and pesign to the native dependency install
for SecureBoot signing support, and remove the migration TODO
comment since this completes that work.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bfjelds
chore: remove all test-images repository references from pipelines
e5e995c 
Remove repository: test-images resource blocks from 11 pipeline
files. Image builds now use local definitions and build templates
instead of checking out the test-images repo.

Also update the disabled mshvch-dom0 stage template with a comment
noting it needs updating if re-enabled.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bfjelds
fix: set requires_trident=False for trident-container-installer
00f8d8b 
The container-installer image runs trident via a container service,
not from local RPMs. Setting requires_trident=False prevents the
builder from requiring bin/RPMS which won't be populated for
container-runtime images (the pipeline stage skips RPM download
for runtimeEnv=container).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bfjelds
style: fix black formatting in sign.py
fbd037b 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bfjelds
fix: replace sign_boot_artifacts with inject-files iteration
880e9d6 
The old sign_boot_artifacts used items_to_sign list with regex
matching against inject-files.yaml, which fails with newer Image
Customizer output format.

Replace with the test-images approach that iterates injectFiles
entries directly, reading type/source/unsignedSource from each
entry. Supports both MIC v1.1+ (type field) and older versions
(filename-based type detection via get_artifact_type_from_name).

Remove unused get_artifact_path function and items_to_sign
parameter.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bfjelds
fix: remove chmod on root-owned artifact dirs in sign_boot_artifacts
c6b8901 
The signed_path.parent dir (e.g. ukis/) is created by Image
Customizer and owned by root. The mkdir+chmod in sign_boot_artifacts
fails with PermissionError. Remove it — sign_pe_artifact and
sign_verity_hash already handle permissions via sudo cp through
temp dirs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bfjelds bfjelds changed the title feat: migrate image build definitions from test-images to trident engineering: migrate image build definitions from test-images to trident May 13, 2026
@bfjelds
fix: enable sshd service in verity test images
f0ab5cd 
The verity test images install openssh-server but don't enable
the sshd service. Pipeline e2e tests require SSH access to the
deployed VMs. Add sshd to the enabled services list in:
- base/baseimg.yaml (root-verity host)
- base/baseimg-container.yaml (root-verity container)
- usr/host.yaml (usr-verity host)

usr/container.yaml already had sshd enabled.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bfjelds
Copy link
Copy Markdown
Member Author

bfjelds commented May 13, 2026

/azp run [GITHUB]-trident-pr-e2e

@azure-pipelines
Copy link
Copy Markdown

azure-pipelines Bot commented May 13, 2026

Azure Pipelines successfully started running 1 pipeline(s).

@bfjelds
fix: stage RPMs at top level of bin/RPMS, not in x86_64 subdir
33b4356 
Image Customizer's --rpm-source may not search subdirectories.
The old test-images pipeline staged RPMs directly in base/trident/
(flat). Our template was creating bin/RPMS/x86_64/ and placing
RPMs there, so IC couldn't find them in the top-level scan.

Use find + mv to flatten all RPMs into bin/RPMS/ regardless of
artifact directory structure.

Also fix duplicate sshd entry in baseimg-container.yaml.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bfjelds
Copy link
Copy Markdown
Member Author

bfjelds commented May 13, 2026

/azp run [GITHUB]-trident-pr-e2e

@azure-pipelines
Copy link
Copy Markdown

azure-pipelines Bot commented May 13, 2026

Azure Pipelines successfully started running 1 pipeline(s).

@bfjelds bfjelds force-pushed the user/bfjelds/mjolnir/sync-test-images branch from 8a86655 to d6211af Compare May 14, 2026 01:13
@bfjelds
fix: consolidate post-install.sh for standard and direct-streaming
145951a 
Merge the test-images improvements (ln -s -T, audit dir, override
mechanism) with the original direct-streaming conditional into a
single post-install.sh.

Standard installer path: sets up config symlink + override
Direct-streaming path: strips ConditionPathExists + sets up rcp-agent

Removes the separate post-install-direct-streaming.sh that was
created during the initial copy.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bfjelds bfjelds force-pushed the user/bfjelds/mjolnir/sync-test-images branch from d6211af to 145951a Compare May 14, 2026 01:15
@bfjelds
Copy link
Copy Markdown
Member Author

bfjelds commented May 14, 2026

/azp run [GITHUB]-trident-pr-e2e

@azure-pipelines
Copy link
Copy Markdown

azure-pipelines Bot commented May 14, 2026

Azure Pipelines successfully started running 1 pipeline(s).

@bfjelds
fix: restore execute permissions on scripts copied from test-images
0f0fff9 
Files copied from test-images on Windows lost their Unix execute
bits (100755 -> 100644). This caused etc-mount.sh to be
non-executable, which made etc-mount.service fail at boot on
root-verity images.

Fix execute permissions on all .sh scripts that were copied from
test-images.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bfjelds
Copy link
Copy Markdown
Member Author

bfjelds commented May 14, 2026

/azp run [GITHUB]-trident-pr-e2e

@azure-pipelines
Copy link
Copy Markdown

azure-pipelines Bot commented May 14, 2026

Azure Pipelines successfully started running 1 pipeline(s).

bfjelds and others added 2 commits May 14, 2026 08:16
@bfjelds
Revert changes to trident-mshvch-dom0-testimg.yml
d77254d 
Restore the original mshvch-dom0 test image build stage from main,
undoing the disabled-pending-migration placeholder.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bfjelds
Update test image READMEs for in-repo build system
b8fb823 
- Fix build commands: use testimages.py instead of old make targets
- Fix RPM paths: bin/RPMS/ via make bin/trident-rpms.tar.gz (not base/trident/*.rpm)
- Fix output paths: artifacts/<name>.cosi (not build/<name>/partition files)
- Note which images do/don't require Trident RPMs
- Remove stale test-images ADO build badge

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bfjelds bfjelds marked this pull request as ready for review May 14, 2026 15:40
@bfjelds bfjelds requested a review from a team as a code owner May 14, 2026 15:40
Copilot AI review requested due to automatic review settings May 14, 2026 15:40
@bfjelds
Copy link
Copy Markdown
Member Author

bfjelds commented May 14, 2026

/azp run [GITHUB]-trident-pr-e2e

@azure-pipelines
Copy link
Copy Markdown

azure-pipelines Bot commented May 14, 2026

Azure Pipelines successfully started running 1 pipeline(s).

Copilot AI reviewed May 14, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@bfjelds bfjelds requested a review from Copilot May 14, 2026 19:05
Copilot AI reviewed May 14, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 58 out of 58 changed files in this pull request and generated 6 comments.

Comments suppressed due to low confidence (4)

tests/images/trident-verity-testimage/base/etc-mount.sh:1

  • The overlay paths used by etc-mount.sh (/var/lib/trident-overlay/etc-rw/...) do not match the paths configured in base/etc.mount (/var/lib/trident-overlay/etc/...). With the current etc-mount.service requiring etc.mount, this mismatch can lead to inconsistent overlay state or mount failures. Align the upper/work dirs between the unit and the script, or remove one of the two mounting mechanisms to avoid double-mounting /etc.
    tests/images/trident-verity-testimage/base/create-web-dir.sh:1
  • This script is installed and executed as part of postCustomization but does not include a shebang or strict-mode settings. Add a #!/bin/bash shebang and basic safety flags (e.g., exit-on-error) to ensure consistent execution across environments.
    tests/images/trident-verity-testimage/usr/host.yaml:1
  • efibootmgr is listed twice in the package install list. Dropping the duplicate avoids confusion and keeps the manifest easier to audit.
    tests/images/trident-verity-testimage/usr/host.yaml:1
  • efibootmgr is listed twice in the package install list. Dropping the duplicate avoids confusion and keeps the manifest easier to audit.

Comment thread tests/images/builder/sign.py
Comment thread tests/images/builder/sign.py
Comment thread tests/images/builder/sign.py
Comment thread tests/images/trident-container-testimage/base/files/trident-container.service
Comment thread tests/images/trident-container-installer/base/files/trident-container.service
Comment thread .pipelines/templates/stages/build_image/build-image-template.yml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@bfjelds