Externalizing environment-specific microservice configuration for CI/CD

Note	This repository contains the guide documentation source. To view the guide in published form, view it on the Open Liberty website.

Learn how to create environment-specific configurations for microservices by using MicroProfile Config configuration profiles for easy management and portable deployments throughout the CI/CD lifecycle.

What you’ll learn

Managing configurations for microservices can be challenging, especially when configurations require adjustments across various stages of the software development and delivery lifecycle. The MicroProfile Config configuration profile feature, also known as the Config Profile, is a direct solution to this challenge. It simplifies the management of microservice configurations across diverse environments - from development to production and throughout the continuous integration/continuous delivery (CI/CD) pipeline. By externalizing and tailoring configuration properties to each environment, the CI/CD process becomes more seamless, so you can concentrate on perfecting your application code and capabilities.

You’ll learn how to provide environment-specific configurations by using the MicroProfile Config configuration profile feature. You’ll work with the MicroProfile Config API to create configuration profiles that use profile-specific configuration properties and configuration sources.

This guide builds on the Separating configuration from code in microservices guide and the Configuring microservices guide. If you are not familiar with externalizing the configuration of microservices, it will be helpful to read the External configuration of microservices document and complete the aforementioned guides before you proceed.

The application that you will work with is a query service, which fetches information about the running JVM from a system microservice. You’ll use configuration profiles to externalize and manage the configurations across the development, testing, and production environments.

https://raw.githubusercontent.com/OpenLiberty/guides-common/prod/gitclone.adoc

Creating a configuration profile for the dev environment

The dev environment is used to test, experiment, debug, and refine your code, ensuring an application’s functional readiness before progressing to subsequent stages in a software development and delivery lifecycle.

Navigate to the start directory to begin.

The starting Java project, which you can find in the start directory, is a multi-module Maven project comprised of the system and query microservices. Each microservice is in its own corresponding directory, system and query.

system/pom.xml

link:finish/system/pom.xml[role=include]

query/pom.xml

link:finish/query/pom.xml[role=include]

The system microservice contains the three Maven build profiles: dev, test, and prod, in which the dev profile is set as the default. Each build profile defines properties for a particular deployment configuration that the microservice uses.

The MicroProfile Config configuration profile feature supplies configurations for different environments when only a single profile is active. The active profile is set using the mp.config.profile property. You can set it in any of the configuration sources and it is read once during application startup. When a profile is active, its associated configuration properties are used. For the query service, the mp.config.profile property is set to dev in its Maven pom.xml. This Liberty configuration variable indicates to the runtime that dev is the active configuration profile.

When you run Open Liberty in dev mode, the dev mode listens for file changes and automatically recompiles and deploys your updates whenever you save a new change.

Open a command-line session and run the following commands to navigate to the system directory and start the system service in the dev environment:

cd system
mvn liberty:dev

Open another command-line session and run the following commands to navigate to the query directory and start the query service in the dev environment:

cd query
mvn liberty:dev

After you see the following message, your Liberty instance is ready in dev mode:

**************************************************
*     Liberty is running in dev mode.

Dev mode holds your command-line session to listen for file changes. Open another command-line session to continue, or open the project in your editor.

microprofile-config.properties

link:start/query/src/main/resources/META-INF/microprofile-config.properties[role=include]

In the dev environment, the dev configuration profile is set in the system/pom.xml file as the configuration profile to use for running the system service. The system service runs on HTTP port 9081 and HTTPS port 9444 using the context root system/dev. It uses a basic user registry with username alice and password alicepwd for resource authorization. Note that the basicRegistry element is a simple registry configuration for learning purposes. For more information on user registries, see the User registries documentation.

Point your browser to the http://localhost:9085/query/systems/localhost URL.

The query service returns the message: {"fail":"Failed to reach the client localhost."}. This is because the current query service uses the default properties in the query/src/main/resources/META-INF/microprofile-config.properties file to access the system service.

For proper communication with the development system service, the query service uses properties in the dev configuration profile.

There are two ways to define configuration properties that are associated with your configuration profile. The first is as individual configuration properties associated with a configuration profile that can be specified in any kind of MicroProfile configuration source. The second is through default microprofile-config.properties configuration files embedded in your application that can be associated with different configuration profiles. The former allows for flexibility in defining profile-specific configuration properties in the best configuration sources for your needs while the latter enables default profiles of configuration properties to be provided in your application.

Creating profile-specific configuration properties

This approach involves directly associating individual configuration properties with a configuration profile. To define a configuration property for a particular config profile, use the %<config_profile_id>.<property_name>=<value> syntax, where <config_profile_id> is the unique identifier for the configuration profile and <property_name> is the name of the property that you want to set.

Replace the microprofile-config.properties file.
query/src/main/resources/META-INF/microprofile-config.properties

microprofile-config.properties

link:finish/query/src/main/resources/META-INF/microprofile-config.properties[role=include]

system/pom.xml

link:finish/system/pom.xml[role=include]

Configure the %dev.* properties in the microprofile-config.properties file based on the values from the dev profile of the system service.

Because the active profile is set to dev, each %dev.* property overrides the value of the plain non-profile-specific property. For example, in this case, the %dev.system.httpsPort property overrides the system.httpsPort property and the value is resolved to 9444.

Because you are running the query service in dev mode, the changes that you made are automatically picked up.

Try out the application at the http://localhost:9085/query/systems/localhost URL. You can see the current OS and Java version in JSON format.

Creating profile-specific microprofile-config.properties configuration files

Creating profile-specific microprofile-config.properties configuration files is a structured way to provide and manage more extensive sets of default configurations. You can create a configuration file for each configuration profile in the META-INF folder on the classpath of your application by using the microprofile-config-<config_profile_id> naming convention, where <config_profile_id> is the unique identifier for a configuration profile. After you create the file, you can add your configuration properties to it with the standard <property_name>=<value> syntax.

Create the microprofile-config-dev.properties file.
query/src/main/resources/META-INF/microprofile-config-dev.properties

microprofile-config-dev.properties

link:finish/query/src/main/resources/META-INF/microprofile-config-dev.properties[role=include]

system/pom.xml

link:finish/system/pom.xml[role=include]

Define the system.* properties in the microprofile-config-dev.properties file based on the values from the dev profile of the system service.

Replace the microprofile-config.properties file.
query/src/main/resources/META-INF/microprofile-config.properties

microprofile-config.properties

link:finish/query/src/main/resources/META-INF/microprofile-config.properties[role=include]

server.xml

link:finish/system/src/main/liberty/config/server.xml[role=include]

Remove the %dev.* properties from the microprofile-config.properties file.

Because the active profile is set to dev, any system. properties specified in the microprofile-config-dev.properties file take precedence over the system. property values in the microprofile-config.properties file.

Now, point your browser to the http://localhost:9085/query/systems/localhost URL to check out the application again. You can see the current OS and Java version in JSON format.

When you are done checking out the application in dev environment, exit dev mode by pressing CTRL+C in the command-line sessions where you ran the system and query services.

Creating a configuration profile for the test environment

In CI/CD, the test environment is where integration tests ensure the readiness and quality of an application. A good testing configuration not only ensures smooth operations but also aligns the environment closely with potential production settings.

Create the microprofile-config-test.properties file.
query/src/main/resources/META-INF/microprofile-config-test.properties

microprofile-config-test.properties

link:finish/query/src/main/resources/META-INF/microprofile-config-test.properties[role=include]

system/pom.xml

link:finish/system/pom.xml[role=include]

Define the system.* properties in the microprofile-config-test.properties file based on the values from the test profile of the system service.

Create the QueryEndpointIT class.
query/src/test/java/it/io/openliberty/guides/query/QueryEndpointIT.java

QueryEndpointIT.java

link:finish/query/src/test/java/it/io/openliberty/guides/query/QueryEndpointIT.java[role=include]

Implement endpoint tests to test the basic functionality of the query microservice. If a test failure occurs, you might have introduced a bug into the code.

See the following descriptions of test cases:

• testQuerySystem() verifies the /query/systems/{hostname} endpoint.

• testUnknownHost() verifies that an unknown host or a host that does not expose their JVM system properties is correctly handled with a fail message.

Running the tests in the test environment

Now, navigate to the start directory.

scripts/testApp.sh|.bat

link:finish/scripts/testApp.sh[role=include]

microprofile-config-test.properties

link:finish/query/src/main/resources/META-INF/microprofile-config-test.properties[role=include]

Test the application under the test environment by running the following script that contains different Maven goals to build, start, test, and stop the services.

https://raw.githubusercontent.com/OpenLiberty/guides-common/prod/os-tabs.adoc

scripts\testApp.bat

./scripts/testApp.sh

If the tests pass, you see output similar to the following example:

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running it.io.openliberty.guides.system.SystemEndpointIT
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.539 s - in it.io.openliberty.guides.system.SystemEndpointIT

Results:

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

...

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running it.io.openliberty.guides.query.QueryEndpointIT
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.706 s - in it.io.openliberty.guides.query.QueryEndpointIT

Results:

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0

Next steps

Deploying the application to a Kubernetes environment using the Open Liberty Operator is an optional learning step in this guide.

To further explore deploying microservices using Kubernetes and the Open Liberty Operator, you can read the following guides:

• Deploying a microservice to Kubernetes using Open Liberty Operator

• Deploying a microservice to OpenShift 4 using Open Liberty Operator

A secure production environment is essential for application security. In the previous sections, you learned how to use the MicroProfile Config API to externalize credentials and other properties for accessing the system service. This strategy makes the application more adaptable to different environments without the need to change code and rebuild your application.

In the this section, you’ll learn how to use Kubernetes secrets to provide the credentials and how to pass them to the query service by using MicroProfile Config.

Deploying the application in the prod environment with Kubernetes

deploy.yaml

link:finish/deploy.yaml[role=include]

system/Dockerfile

link:finish/system/Dockerfile[role=include]

query/Dockerfile

link:finish/query/Dockerfile[role=include]

query/pom.xml

link:finish/query/pom.xml[role=include]

microprofile-config.properties

link:finish/query/src/main/resources/META-INF/microprofile-config.properties[role=include]

Before deploying, create the Dockerfile files for both system and query microservices. Then, build their .war files and Docker images in the start directory.

mvn -P prod clean package
docker build -t system:1.0-SNAPSHOT system/.
docker build -t query:1.0-SNAPSHOT query/.

The Maven clean and package goals can clean the target directories and build the .war application files from scratch. The microprofile-config-dev.properties and microprofile-config-test.properties files of the query microservice are excluded from the prod build. The default microprofile-config.properties file is automatically applied.

The Docker build command packages the .war files of the system and query microservices with their default configuration into your Docker images.

After building the images, you can create a Kubernetes secret for storing sensitive data such as credentials.

kubectl create secret generic sys-app-credentials \
        --from-literal username=[username] \
        --from-literal password=[password]

For more information about managing secrets, see the Managing Secrets using kubectl documentation.

Finally, write up the deploy.yaml deployment file to configure the deployment of the system and query microservices by using the Open Liberty Operator. The sys-app-credentials Kubernetes secrets set the environment variables DEFAULT_USERNAME and DEFAULT_PASSWORD for the system microservice, and SYSTEM_USER and SYSTEM_PASSWORD for the query microservice.

deploy.yaml

link:finish/deploy.yaml[role=include]

If you want to override another property, you can specify it in the env sections of the deploy.yaml file. For example, set the CONTEXT_ROOT environment variable in the system deployment and the SYSTEM_CONTEXTROOT environment variable in the query deployment.

After the images and the secret are ready, you can deploy the microservices to your production environment with Kubernetes.

kubectl apply -f deploy.yaml

Great work! You’re done!

You just learned how to use the MicroProfile Config’s configuration profile feature to configure your application for multiple CI/CD environments.

Feel free to try one of the related guides. They demonstrate new technologies that you can learn to expand on what you built in this guide.

https://raw.githubusercontent.com/OpenLiberty/guides-common/prod/attribution.adoc