Batch Runner

Overview

A common approach with unit tests is to periodically run them without any involvement from the development team, and Jade provides the ability to run a batch of unit tests 'out of the box' which should suit basic needs. Should you require more advanced features a component has been provided which has the following extensions:

• support for advanced/query selection of which tests to run
• multi-threaded capability to improve performance
• collation of results into continuous integration friendly formats (for direct import into Jenkins)

Implementation

A batch run is made up of four distinct phases:

• Invocation (either by script or commandline)
• Collation of what tests to run
• Execution of the tests
• Output of the tests

Invoking the Batch run

The batch runner can be started by one of 3 options, via:

• Jadescript (see AutomatedTestSchema::JadeScript::runBatchForEnvironment)
• command-line (functionality to follow)
• command-line that invokes a Jadescript which drives the unit test run (see Jade documentation)

Collation of unit tests

Determining which tests to run is in the control of the developer, either by supplying your own list or utilising the ATLocator class to provide advanced filtering or as a simple means to find them all.

Execution of unit tests

Unit Tests will be run via an application running in the schema they are defined in to minimise application context issues from appearing. This requires the AutomatedTestPackage to be imported into the highest most user schema and the custom application defined as per the installation instructions.

Output of unit test results

The results are collated into a transient structure which is available for custom evaluation, or simply let the predefined output formats take care of outputting this to file or the interpreter window. The currently supported formats include csv and nunit, which was chosen given this can be imported into Jenkins.

Examples

Execute all tests in all schemas in environment and write the results to the interpreter:

vars
	runner	: ATBatchRunner;
	
begin
	create runner transient;
	runner.run();
end;

Execute all tests in the current schema and all below it, writing the output to a csv file located in the logs folder:

vars
	runner	: ATBatchRunner;
	
begin
	create runner transient;
	runner.locator.addSchema( currentSchema, true );
	
	runner.batchSettings.outputFormat	:= ATBatchSettings.OutputFormatCSV;
	runner.batchSettings.outputTarget	:= ATBatchSettings.OutputTargetFile;
		
	runner.run();
end;

Execute all #IntegrationTest's for schema "AutomatedTestSchema_TestInternals" and write the results to a xml file that can be loaded into Jenkins:

vars
	runner	: ATBatchRunner;
	
begin
	create runner transient;
	runner.locator.annotations.add( "#IntegrationTest" );
	runner.locator.addSchemaName( "AutomatedTestSchema_TestInternals" );
	
	runner.batchSettings.outputFormat	:= ATBatchSettings.OutputFormatNUnit;
	runner.batchSettings.outputTarget	:= ATBatchSettings.OutputTargetFile;
		
	runner.run();
end;