Test procedures can link directly into html rendering of ITI TF

[JohnMoehrke]

Now that we (ITI) have hyperlinks to specific sections in the TF, the question becomes, whether/how we include them in test definitions in Gazelle Master Model.

Note that this issue would/should not directly affect the ITI publication effort. It's a larger issue for methods to employ traceability from requirements to tests for IHE testing initativeS. We do not want to hash that out under the umbrella of ITI publication. I will document just a few issues below.

[lynnfel]

This is background information. As John mentioned above, there is a larger discussion to be had if IHE wants to consider implementing methods for rqmts/test traceability.

In my comments, I am going to specifically address pre-Connectathon and Connectathon test definitions maintained in Gazelle Master Model (GMM). I am going ignore other test cases (conformity assessment, projectathon…).

Our current TF does not “tag” individual requirements with specific identifiers. There is no imminent change to address this.

Pre-Connectathon test cases in GMM:
Unlike Connectathon test cases, the detailed instructions/test steps for pre-Connectathon test do not reside in GMM. This is because most pre-CAT tests are associated with a specific tool. Our tools are provided by many organizations and it was not desirable to have those developers maintain their instructions in a common tool like GMM.

Instead, each pre-Connectathon test case entry in GMM contains:
⁃ Test Keyword (unique id)
⁃ Test Name
⁃ Some other metadata (date/status…)
⁃ The profile/actor/option the test applies to
⁃ A link to the test instructions (ie outside GMM)
⁃ A link to the tool used (if applicable)
⁃
You will see that there is no database field to put a TF link/reference to identify the requirement(s) covered by the test. The test instructions themselves (maintained outside GMM) may have a TF reference, but this is not captured in the GMM database.

Connectathon test cases in GMM:
GMM provides a template for Connectathon test cases (the first 4 are in common with pre-CAT tests)
⁃ Test Keyword (unique id)
⁃ Test Name
⁃ Some other metadata (date/status…)
⁃ The profile/actor/option the test applies to
⁃ A narrative section in HTML format. The HTML contains 3 sections (Special Instruction, Test Definition (really a summary), Evaluation Criteria). The format of this HTML is not enforced by GMM, but TPMs follow it by convention so that tests in all domains ‘look the same’ to our users
⁃ Test steps. The structure of these in enforced by GMM.

Again, there is no database field to put a TF link/reference to identify the requirement(s) covered by the test.

In my test cases (not all), it is certainly helpful at times to point to specific TF text that is driving some test steps, or certain evaluation criteria. Up until now, I have done this using traditional TF references, e.g. in an Evaluation Criteria narrative, I might include a statement like, “see ITI TF-3: 1.2.3.4.”

BOTTOM LINE: The new, more granular links in the ITI TF will enable me to add more precise TF references into test definitions where it will be helpful. That is a benefit to users. The absence of a tag for each testbable rqmt in the TF, and no database field in GMM to put such a tag (or a TF link) is a basic gap. It would be a major undertaking to enable rqmts/test traceability in IHE. As stated above, that's not in the scope of the ITI HTML publication initiative.

(Another FYI note: There was an Assertion Manager feature added to the Gazelle suite of tools (https://gazelle.ihe.net/AssertionManagerGui). It was never used in the context of Connectathon testing. I believe Kereval used it for conformity assessment. I don’t know whether it’s actively used any more. The tool has some ‘features’ to enable some ‘linkage’ between TF and tests, but the tool was implemented before an overall was method in place to enable ‘real’ traceability. In IHE, that ‘real’ method still does not exist.)