Dynatrace Jenkins Plugin

This plugin for Jenkins pulls Test Automation data from Dynatrace AppMon and displays it through charts and tables on the project and build level.

• Download Latest Release: https://github.com/jenkinsci/dynatrace-plugin/releases
• Install from Jenkins Plugin Center: https://wiki.jenkins-ci.org/display/JENKINS/Dynatrace+Plugin
• Jenkins Plugin on the Dynatrace Community: https://community.dynatrace.com/community/display/DL/Test+Automation+Plugin+for+Jenkins

Special thanks to Wolfgang Gottesheim who contributed to the first version of this plugin.

Table of Contents

• Installation
  • Using Jenkins Update Center
  • Manual Installation
• Configuration
  • Global Settings
  • Build configuration
    • Option 1: Test Run Registration from Jenkins
    • Option 2: Test Run Registration from Maven/Ant/...
  • Post Build Action
• Examples
• Maven
  • Option 1: Test Run Registration from Jenkins
  • Option 2: Test Run Registration from Maven
• Ant
  • Option 1: Test Run Registration from Jenkins
  • Option 2: Test Run Registration from Ant
• NAnt
  • Option 1: Test Run Registration from Jenkins
• Problems? Questions? Suggestions?
• Additional Resources
  • Dynatrace AppMon Documentation
  • Recorded Webinar
  • Blogs

Installation

Using Jenkins Update Center

The recommended way of installing the plugin is by the Update Center (plugin directory). Navigate to "Manage Jenkins" / "Manage Plugins" page and switch to the "Available" tab. Search for the "Dynatrace" keyword and install the plugin.

Manual Installation

This procedure is meant for developers who want to install locally built plugin version.

• build the plugin from source using mvn package command
• in Jenkins, go to "Manage Jenkins" / "Manage Plugins" page
• switch to the "Advanced" tab
• upload the built plugin package from target/dynatrace-dashboard.hpi path in the "Upload Plugin" section
• restart Jenkins when the installation is complete

Configuration

If you are using Jenkins 2.5 or higher, you need to enable the use of environment variables - see Jenkins 2.5+ build step execution failed with java.lang.NullPointerException

Global settings

The global settings for the plugin are located under Manage Jenkins / Configure System / Dynatrace Application Monitoring. The connection to the Dynatrace AppMon Server is configured in this section. The configured user needs to have the right credentials to be able to access the Test Automation REST API.

The advanced section enables you to set a delay before retrieving the test results from the server. Change this settings if you are not getting all the test results in Jenkins.

Build configuration

In the build configuration (build name / configure), first enable Use Dynatrace AppMon to monitor tests in the Build Environment and fill the required fields.

Option 1: Test Run Registration from Jenkins

Use this option when:

• you want an easy integration and don't want to adapt your build scripts too much
• you are OK with defining the version in Jenkins to register the test run

Then, for each test category (Unit Test, Performance Test, Browser Test or Web API Test), you need to add a build step to register a test run to the Dynatrace AppMon server before running your tests.

The testrun id is available as environment variable which can be passed to the Dynatrace AppMon agent in the build script.

Example:

<jvmarg value="-agentpath:$/var/lib/dynatrace/agent/lib64/libdtagent.so=name=JavaAgent,
server=localhost:9998,loglevel=warning,optionTestRunIdJava=${dtTestrunID}" />

Maven tip:

Maven allows passing java arguments into surefire and failsafe plugins directly from commandline. Thus Dynatrace AppMon agent settings may be passed using jenkins build step definition (no changes required in maven build script).

Option 2: Test Run Registration from Maven/Ant/Gradle/...

Use this option when:

• you don't mind using an additional plug-in in your Ant/Maven scripts
• you want to re-use the Ant/Maven version to register the test run

• do not add the Register Test Run step in Jenkins
• install & configure the Ant/Maven/Gradle/... plug-in in your build script
• register the Test Run using the Ant/Maven/Gradle/... plug-in and make sure to pass the Jenkins build id ${BUILD_ID} in the meta data of the test run. The Jenkins Plug-in will use this build id to retrieve the results.

Post Build Action

At the end of the build, add the Dynatrace AppMon post-build action to retrieve the test results. You can also decide if the test results will change the build status.

Examples

Maven

Option 1

Option 1: Test Run Registration from Jenkins

in this case, the Test Run Id will be passed from Jenkins to your Maven build script (pom.xml) as an environment variable. You just need to make sure that the Java agent is injected and that the test run id ${dtTestrunID} is passed to the agent.

Example with Surefire:

<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-surefire-plugin</artifactId>
	<version>2.19.1</version>
	<configuration>
		<includes>
			<include>**/Unit*.java</include>
		</includes>
		<!-- dtTestrunID is passed from Jenkins as environment variable --> 
		<!-- dt_agent_path, dt_agent_name and dt_server needs to be configured in your script or passed as environment variable -->
		<argLine>-agentpath:"${dt_agent_path}"=name=${dt_agent_name},server=${dt_server},optionTestRunIdJava=${dtTestrunID}</argLine>
	</configuration>
</plugin>

Example with Failsafe:

<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-failsafe-plugin</artifactId>
	<version>2.19.1</version>
	<configuration>
		<includes>
			<include>**/Integration*.java</include>
		</includes>
		
		<!-- dtTestrunID is passed from Jenkins as environment variable --> 
		<!-- dt_agent_path, dt_agent_name and dt_server needs to be configured in your script or passed as environment variable -->
		<argLine>-agentpath:"${dt_agent_path}"=name=${dt_agent_name},server=${dt_server},optionTestRunIdJava=${dtTestrunID}</argLine>
	</configuration>
	<executions>
		<execution>
			<goals>
			 	<goal>integration-test</goal>
				<goal>verify</goal>
			</goals>
		</execution>
	</executions>
</plugin>

Option 2

In this case, the test run registration is done directly from Maven - enabling you to re-use version information and meta-data defined in the pom.xml file.

Download and install the Dynatrace Maven Plug-in as described here: https://github.com/Dynatrace/Dynatrace-Maven-Plugin

Add the Dynatrace Automation Plug-in in your dependency list:

<dependencies>
	<dependency>
		<groupId>dynaTrace</groupId>
		<artifactId>dtAutomation</artifactId>
		<version>${dynaTrace.version}</version>
	</dependency>
</dependencies>

Example with Surefire:

<plugin>
	<groupId>dynaTrace</groupId>
	<artifactId>dtAutomation</artifactId>
	<version>${dynaTrace.version}</version>
	<executions>
		<execution>
			<id>DT_StartTest_UnitTest</id>
			<configuration>
				<!-- the build id (passed from Jenkins) will be re-used by Dynatrace to retrieve the test results --> 
				<versionBuild>${BUILD_ID}</versionBuild>
				<profileName>junit-example-dt-maven</profileName>
				<category>unit</category>
			</configuration>
			<!-- start this test in the process-test-classes phase which is the one before the tests are executed -->
			<phase>process-test-classes</phase>
				<goals>
					<!-- call the startTest goal of the Dynatrace Maven plugin -->
					<goal>startTest</goal>
				</goals>
		</execution>
</plugin>
<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-surefire-plugin</artifactId>
	<version>2.19.1</version>
	<configuration>
		<includes>
			<include>**/Unit*.java</include>
		</includes>
		<!-- dtTestrunID is passed from the Dynatrace Maven Plug-in --> 
		<!-- dt_agent_path, dt_agent_name and dt_server needs to be configured in your script or passed as environment variable -->
		<argLine>-agentpath:"${dt_agent_path}"=name=${dt_agent_name},server=${dt_server},optionTestRunIdJava=${dtTestrunID}</argLine>
	</configuration>
</plugin>

Example with Failsafe:

<plugin>
	<groupId>dynaTrace</groupId>
	<artifactId>dtAutomation</artifactId>
	<version>${dynaTrace.version}</version>
	<executions>
		<execution>
		<id>DT_StartTest_IntegrationTest</id>
		<configuration>
			<!-- the build id (passed from Jenkins) will be re-used by Dynatrace to retrieve the test results --> 
			<versionBuild>${BUILD_ID}</versionBuild>
			<profileName>junit-example-dt-maven</profileName>
			<category>performance</category>
		</configuration>
		<phase>pre-integration-test</phase>
		<goals>
			<goal>startTest</goal>
		</goals>
		</execution>
	</executions>
</plugin>
<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-failsafe-plugin</artifactId>
	<version>2.19.1</version>
	<configuration>
		<includes>
			<include>**/Integration*.java</include>
		</includes>
		<!-- dtTestrunID is passed from the Dynatrace Maven Plug-in --> 
		<!-- dt_agent_path, dt_agent_name and dt_server needs to be configured in your script or passed as environment variable -->
		<argLine>-agentpath:"${dt_agent_path}"=name=${dt_agent_name},server=${dt_server},optionTestRunIdJava=${dtTestrunID}</argLine>
	</configuration>
	<executions>
		<execution>
			<goals>
				<goal>integration-test</goal>
				<goal>verify</goal>
			</goals>
		</execution>
	</executions>
</plugin>

Ant

Option 1

Option 1: Test Run Registration from Jenkins

in this case, the Test Run Id will be passed from Jenkins to your Ant script as an environment variable. You just need to make sure that the Java agent is injected and that the test run id ${dtTestrunID} is passed to the agent.

Example:

<target name="junit" depends="jar"> 
<junit printsummary="yes"> 
<!-- dtTestrunID is passed from Jenkins as environment variable --> 
<!-- dt_agent_path, dt_agent_name and dt_server needs to be configured in your script or passed as environment variable -->
<jvmarg value="-agentpath:${dt_agent_path}=name=${dt_agent_name},server=${dt_server},loglevel=warning,optionTestRunIdJava=${dtTestrunID}" /> 
<classpath> 
	<path refid="classpath"/> 
	<path refid="application"/> 
</classpath> 
<batchtest fork="yes"> 
	<fileset dir="${src.dir}" includes="*Test.java"/> 
</batchtest> 
</junit> 
</target> 

Option 2

In this case, the test run registration is done directly from Ant - enabling you to re-use version information and meta-data given in the Ant script.

Download and install the Dynatrace Ant Library as described here: https://github.com/Dynatrace/Dynatrace-Ant-Plugin/

Import the Dynatrace Ant Task definitions:

<property name="dtBaseDir" value="${libs}/dynaTrace" />
<import file="${dtBaseDir}/dtTaskDefs.xml"/>

Call DtStartTest before running your Tests:

<target name="test" depends="compile,test-compile" description="Run tests">
<mkdir dir="${test.result.dir}"/>
<mkdir dir="${test.report.dir}"/>

<!-- the build id (passed from Jenkins) will be re-used by Dynatrace to retrieve the test results --> 
<DtStartTest
	versionMajor="${version.major}"
	versionMinor="${version.minor}"
	versionRevision="${version.revision}"
	versionMilestone="test-parallel"
	versionBuild="${BUILD_ID}"
	profilename="${dynatrace.profile.name}"
	username="${dynatrace.server.user}"
	password="${dynatrace.server.pass}"
	serverurl="${dynatrace.server.url}"
	category="${dynatrace.test.category}">
</DtStartTest>

Finally run your Unit Tests:

<junit printsummary="yes"> 
<!-- dtTestrunID is passed from the Dynatrace Ant task --> 
<!-- dt_agent_path, dt_agent_name and dt_server needs to be configured in your script or passed as environment variable -->
<jvmarg value="-agentpath:${dt_agent_path}=name=${dt_agent_name},server=${dt_server},loglevel=warning,optionTestRunIdJava=${dtTestrunID}" /> 
<classpath> 
	<path refid="classpath"/> 
	<path refid="application"/> 
</classpath> 
<batchtest fork="yes"> 
	<fileset dir="${src.dir}" includes="*Test.java"/> 
</batchtest> 
</junit> 

NAnt

Option 1

Option 1: Test Run Registration from Jenkins

For the .NET agent, the test run id must be passed through the environment variable DT_TESTRUN_ID (see also .NET Agent Configuration). Using EnvInject Plugin in Jenkins you can inject the DT_TESTRUN_ID variable between the Register Test Run and the Execute Build steps.

Problems? Questions? Suggestions?

• Jenkins Plugin FAQ / Troubleshooting Guide
• Post any problems, questions or suggestions to the Dynatrace Community's Application Monitoring & UEM Forum.

Additional Resources

Dynatrace AppMon Documentation

• Continuous Delivery & Test Automation
• Capture Performance Data from Tests
• Integrate Dynatrace in Continous Integration Builds

Recorded Webinar

• Online Perf Clinic - Eclipse and Jenkins Integration
• Online Perf Clinic - Metrics-Driven Continuous Delivery with Dynatrace Test Automation

Blogs

• Continuous Performance Validation in Continuous Integration Environments
• Software Quality Metrics for your Continuous Delivery Pipeline – Part I
• Software Quality Metrics for your Continuous Delivery Pipeline – Part II: Database
• Software Quality Metrics for your Continuous Delivery Pipeline – Part III – Logging
• Automated Performance Analysis for Web API Tests