aws4embeddedlinux-ci

The aws4embeddedlinux-ci library helps you deploy an AWS cloud infrastructure supporting the Embedded Linux builds for your project using AWS CDK.

Table of Contents

1. Architecture
2. API documentation
3. Project Build with AWS CodePipeline
4. Project Build with AWS CodeBuild as Action Runner
5. Setup
6. Development Setup
7. Known issues
8. Security
9. Contributing
10. License

Architecture

This is the overall architecture pattern used to deploy the build pipeline:

Bask to the top

API documentation

The API documentation is generated automatically using Typedoc.

Bask to the top

Project Build with AWS CodePipeline

Several pipelines are provided in this library, each one demonstrating a different aspect of how to build a Yocto image with AWS.

Once deployed, ou can review the pipeline execution from the Developer Tools > Pipeline - CodePipeline > Pipelines page in the AWS Console.

From the pipeline page, you can find the source repository (S3), the CodeBuild Project (with the build logs), and the S3 bucket that the image is uploaded to, at the end.

Each pipelines will refresh/re-run automatically every week using AWS EventBridge in order to grab the latest updates.

Poky

The pipeline name will end with poky.

This example will build the core-image-minimal image from Poky using the repo tool to manage layers. CVE checking is also enabled in the buildspec file.

Expected build times:

• First build: 32 minutes
• Rebuild (without any change, just use sstate cache): 8 minutes

Poky EC2 AMI

The pipeline name will end with poky-ami.

Yocto can be used to create an EC2 AMI. This example builds an AMI based on Poky and meta-aws and exports it to your AMI registry using the VM Import/Export Service.

Expected build times:

• First build: 52 minutes
• Rebuild (without any change, just use sstate cache): 17 minutes

Kas

The pipeline name will end with kas.

The Kas example shows how to use a Kas Config to manage layers.

This tool can help programatically manage layers and config with tighter Yocto integration than Git Submodules or the Repo tool.

Expected build times:

• First build: 36 minutes
• Rebuild (without any change, just use sstate cache): 11 minutes

QEmu

The pipeline name will end with qemu.

This example builds a Qemu based image using meta-aws-demos.

The Qemu image can be run in the CodeBuild environment. Using SLIRP networking, OEQA testing such as ptest can be run in the pipeline.

Expected build times:

• First build: 45 minutes
• Rebuild (without any change, just use sstate cache): 14 minutes

NXP / IMX

The pipeline name will end with nxp.

This example will build an image for the i.MX 6ULL EVK board.

NXP requires users to accept and comply with a EULA in order to build and, for this reason, the buildspec will require modification before the build succeeds.

See the IMX Yocto Users Guide for more detail.

Expected build times:

• First build: xx minutes
• Rebuild (without any change, just use sstate cache): xx minutes

Renesas

This example is based on the Yocto / Renesas R-Car work to build an image for Renesas R-Car-H3 Starter Kit Premier board (unofficial name - H3ULCB) including the proprietary graphics and multimedia drivers from Renesas.

You will need to download the Multimedia and Graphics library and related Linux drivers from the following link (registration necessary):

• https://www.renesas.com/us/en/application/automotive/r-car-h3-m3-h2-m2-e2-documents-software

Download the following files:

• R-Car_Gen3_Series_Evaluation_Software_Package_for_Linux-20220121
• R-Car_Gen3_Series_Evaluation_Software_Package_of_Linux_Drivers-20220121

Graphic drivers are required for Wayland. Multimedia drivers are optional.

Steps to build the image:

• Create a folder named proprietary in the root of the source repo
• Copy the downloaded files into the proprietary folder
• Uncomment the #TODO line in the build.sh file in the source repo
• Deploy the build pipeline

A build should automatically start. Once it succeeds you will get an image containing the proprietary graphics and multimedia drivers.

Expected build times:

• First build: 30 minutes
• Rebuild (without any change, just use sstate cache): 9 minutes

Bask to the top

Project Build with AWS CodeBuild as Action Runner

This will create an AWS CodeBuild project ready to build Embedded Linux which can be used to connect to an external source, e.g. GitHub Actions. This is not using any AWS CodePipeline as in the the Pipeline examples section.

You can use the following example for your GitHub Action.

We recommend you to clone the created AWS CodeBuild project and then proceed with the GitHub Action configuration. However, this will result in the use of the same EFS file system across projects.

Using the AWS CodeBuild project requires a source to be configured manually in the AWS Console at Developer Tools > Build - CodeBuild > Projects.

To configure the AWS CodeBuild source connection to a GitHub repository, you will need to:

• Select the AWS CodeBuild project (aws4el-ci-codebuild-project)
• Select the Project Details tab
• Scroll to the Source section and click on Edit.
• Select GitHub as a Source provider. Configure the access to your repository
• Scroll to the Primary source webhook events section, enable Rebuild every time a code change is pushed to this repository
• Expand the Webhook event filter groups section and click on Add filter group if no group is present, then select WORKFLOW_JOB_QUEUED as Filter group 1

Modify your repository GitHub action by replacing the following line:

runs-on: ${{ vars.CODEBUILD_RUNNER_NAME }}-${{ github.run_id }}-${{ github.run_attempt }}

with:

runs-on: codebuild-<project-name>-${{ github.run_id }}-${{ github.run_attempt }}

<project-name> should be replaced by your AWS CodeBuild project name (aws4el-ci-codebuild-project). There might be more than one occuerence to replace in the file.

Refer to the following example for more details.

Bask to the top

Setup

In order to use this library, you must first set up an AWS CDK project, including installing the CDK tool and bootstrapping the account you wish to deploy to.

Additionally, you must have NodeJS installed.

Note

This library is tested against Node Versions 22. If these version is not available for your system, we recommend using NVM to install a compatible version.

You can also use the sample project code provided to get started and deploy the stacks.

Bask to the top

Setting Up A New Project

In order to create a new project, you will need to initialize a new CDK project.

More details can be found in the CDK Getting Started Documentation.

The following commands will create a new CDK project named my-project:

mkdir my-project
cd my-project
cdk init app --language typescript

Then you will need to install the CDK library including the aws4embeddedlinux-ci library either using npm:

npm install aws4embeddedlinux/aws4embeddedlinux-ci

of `yarn':

yarn add github:aws4embeddedlinux/aws4embeddedlinux-ci
yarn install

If you are not familliar with Yarn, please refer to the documentation.

Once added, you can start creatin your application using the library.

For example, you can start by importing classes using:

import {
  EmbeddedLinuxCodePipelineBaseImageStack,
  EmbeddedLinuxCodePipelineStack,
  EmbeddedLinuxCodeBuildProjectStack,
  PipelineResourcesStack,
  ProjectType,
} from "aws4embeddedlinux-cdk-lib";

Then deploy the base resources and base image pipeline stacks:

const pipelineResourcesStack = new PipelineResourcesStack(app, `demo-resources`, {...});
const baseImageStack = new EmbeddedLinuxCodePipelineBaseImageStack(app, `demo-pipeline-base-image`, {...});

And ultimately deploy the target pipeline:

const projectPipeline = new EmbeddedLinuxCodePipelineStack(app, "demo-project", {
  projectType: ProjectType.Poky,
  ecrRepository: <ecrRepository>,
  ecrRepositoryImageTag: <ecrRepositoryImageTag>,
  pipelineSourceBucket: <SourceBucket>,
  pipelineArtifactBucket: <ArtifactBucket>,
  pipelineOutputBucket: <OutputBucket>,
  vpc: <vpc>,
  encryptionKey: <encryptionKey>,
});

Refer to the API Documentation and the sample for more details.

Once you have completed the code of your application, you can deploy the CDK stack using:

cdk deploy

After the CDK application is successfully deployed, the 'Base Image' pipeline needs to complete successfully.

This will create an Ubuntu based container for building the Yocto images.

Note

This container is used by the other pipelines. If the other pipelines are run before this container is created and pushed to ECR, they will fail.

The 'Base Image' pipeline will run weekly by default to keep the container patched and up to date.

Note

We recommend you to deploy first the 'Base Image' pipeline and once the pipeline completes successfully, then you can deploy the other pipelines in you application as described in the sample.

Once your pipelines completes successfully, the Yocto deploy directory generated content will be pushed into a S3 bucket.

Bask to the top

Development Setup

The repository is leveraging Yarn 2 and if you are not familliar with Yarn, please refer to the documentation.

If you are looking to develop new feature, you can use yarn link to develop with a local copy of this repo.

In this library repo, execute the following:

yarn install
yarn run build
yarn link

and in your project folder:

yarn install
yarn link "aws4embeddedlinux-cdk-lib"
yarn run build

This will link through the system node_modules install.

Note:

You should not install / reference the aws4embeddedlinux/aws4embeddedlinux-ci library in your package.json when using this approach.

Note:

After changing the code for the aws4embeddedlinux/aws4embeddedlinux-ci library, you will need to run yarn run build for changes to be available in your current project.

Note:

When using a system node install on Linux, this can require sudo access. To avoid this, use a node version manager or set a node prefix.

Using Git Credentials and Build Time Secrets

AWS Secrets Manager is the preferred method for adding and using secrets in your pipelines.

The service provides a structured means of access and avoids the pitfalls of putting secrets in environment variables, source repos, etc.

The following steps detaisl at a high level, how you can enable the use of AWS Secrets Manager in your pipelines:

• Create a Secret in Secrets Manager and add your secret value.

• Grant access permissions to the CodeBuild pipeline project.

• Create a Policy Statement which allows secretsmanager:GetSecretValue for your secret.

• Add this policy statement to the buildPolicyAdditions props for the EmbeddedLinuxPipelineStack. e.g.

  import * as iam from "aws-cdk-lib/aws-iam";
  const projectPipeline = new EmbeddedLinuxCodePipelineStack(app, "MyPokyPipeline", {
    projectType: ProjectType.Poky,
    ecrRepository: <ecrRepository>,
    ecrRepositoryImageTag: <ecrRepositoryImageTag>,
    pipelineSourceBucket: <SourceBucket>,
    pipelineArtifactBucket: <ArtifactBucket>,
    pipelineOutputBucket: <OutputBucket>,
    vpc: <vpc>,
    encryptionKey: <encryptionKey>,
    buildPolicyAdditions: [
      iam.PolicyStatement.fromJson({
        Effect: "Allow",
        Action: "secretsmanager:GetSecretValue",
        Resource:
          "arn:aws:secretsmanager:us-west-2:123456789012:secret:my-secret-??????",
      }),
    ],
  });

• The secret can then be used in the CodeBuild Project by adding it to the BuildSpec.

  env:
      secrets-manager:
          SECRET_VALUE: "<Secret ARN>"

See the CodeBuild Documentation for more details.

CVE Checking With Yocto

CVE checking is enabled in the reference implementations. Details on this can be found in the Yocto documentation.

Bask to the top

Known issues

• The use of this CDK library is currently not supported in Windows environments (you can still use WSL).
• When using AWS Cloud9, a micro instance type will run out of memory.
• Deletion of stacks while a CodePipeline is running can lead to unexpected behaviours.
• The NXP-IMX pipeline will fail unless you adjust the build spec file and address the licence acceptance requirement.

Bask to the top

Security

See SECURITY for more information about reporting issues with this project.

Bask to the top

Contributing

See CONTRIBUTING for more information.

Bask to the top

License

This library is licensed under the MIT-0 License. See the LICENSE file.

Bask to the top