SCL Validator Testing Guide

Introduction

This guide aims at explaining the testing principles for the SCL validator CLI tool. The tests covered in this document are automatically run by RiseClipse's CI pipeline, but this guide will show you how to run these tests locally and update them if needed.

Requirements

First off, verify that the environment in which you plan to run the tests is UNIX based, and that Bash is installed (this will be needed to run the test shell scripts). You will then need to have a local copy of the riseclipse-validator-scl2003 repository.
You will also need a working java installation, as the tests will run a java -jar command on the SCL validator fatjar.

Speaking of the SCL validator, you will need to have it built locally in order to run the tests.
The resulting RiseClipseValidatorSCL-*version*.jar should be located in the riseclipse-validator-scl2003/fr.centralesupelec.edf.riseclipse.iec61850.scl.validator/target directory.

Tests structure

Below you can find the test directory structure:

riseclipse-validator-scl2003/test
├── input
│   └── ICD_test.xml
├── run_tests.sh
├── setup_vars.sh
├── snapshots
│   └── ICD_test.out
└── update_snapshots.sh

The input folder is where you will place the SCL .xml files that need to be validated, and the snapshots folder will contain the validator output for each input file.

The use for the remaining files of this directory will be detailed in the testing guide.

Testing guide

Snapshot setup

Upon adding a new file to be validated in the input folder, there will be no corresponding snapshot in the snapshots folder.
This is where the Bash script run_tests.sh comes into play. It has two purposes:

1. Compare the SCL validator output for each input file with its corresponding snapshot in the snapshots folder
2. Create a new snapshot based on the validator output for each input file that has no existing snapshot

Now let's say you added a other_test.xml file in the input folder. Your directory structure should look like this:

├── input
│   ├── ICD_test.xml
│   └── other_test.xml
├── run_tests.sh
├── setup_vars.sh
├── snapshots
│   └── ICD_test.out
└── update_snapshots.sh

As you can see there is no snapshots/other_test.out file yet, that is because we need to generate it.
When running the run_tests.sh script, if your installation is working you should get an output of this kind:

Running tests... Done !

=== Created snapshots: 1 ===
> snapshots/other_test.out

=== Passed tests: 1 ===
> input/ICD_test.xml

And now your directory tree should look like this:

├── input
│   ├── ICD_test.xml
│   └── other_test.xml
├── run_tests.sh
├── setup_vars.sh
├── snapshots
│   ├── ICD_test.out
│   └── other_test.out
└── update_snapshots.sh

Congratulations, you just created your first snapshot!
If you plan on integrating the test you just added to the production test suite, you should commit the input and snapshot files and make a pull request with your changes. Once your changes are integrated, your test will be checked at each CI pipeline run.