1.1.0.0 - Support for RISK PRINCIPE

General remarks:

• This release is compatible with the Data Sharing Framework 1.3.0 or higher. This release does not work with older versions of the DSF.
• Additional information about the NUM-CODEX data transfer process plugin can be found int the repositories wiki.
• The configuration parameters required for a standard installation in a DIC can be found on the installation wiki page.
• The default SearchBundle and validation package config has been replaced to support the MII RISK PRINCIPE project.

Features include:

• The "data send" process is now compatible with additional FHIR resources (like Specimen) and contained resources.
• To enable transport without record linkage, the “data send” process is now able to get a DIC pseudonym for a local pseudonym (with system urn:ietf:rfc:4122).
• The FHIR resource validation step included in the "data send" process, implements workarounds for not yet published bug fixes of the MII KDS module Mikrobiologie (medizininformatik-initiative/kerndatensatzmodul-mikrobiologie/issues/15, medizininformatik-initiative/kerndatensatzmodul-mikrobiologie/issues/16, medizininformatik-initiative/kerndatensatzmodul-mikrobiologie/issues/18).
• Connections to the terminology- and validation package servers, now use the standard DSF forward proxy config variables DEV_DSF_PROXY ....

Process released in the binary assets and GitHub Packages Maven Registry:

• data-transfer: codex-process-data-transfer-1.1.0.0.jar
  Add jar to /opt/bpe/process and make sure it can be read by the bpe Linux user or group (uid: 2202, gid: 2202) chown root:bpe; chmod 440, restart/start bpe.

Stand-alone validator release in the binary assets:

• validator: codex-process-data-transfer-validator-1.1.0.0.zip
  See the Stand-Alone-Validation description in the Wiki on how to use the validator.

Issues closed:

• Add support for RISK PRINCIPE #126
• Deployment documentation still lacks configuration parameters #123
• Remove Special Terminology and Validation Package Forwarding Proxy Config Parameters #122

1.0.0.0 - Update to DSF 1.3.0

General remarks:

• This release is compatible with the Data Sharing Framework 1.3.0. This release does not work with older versions of the DSF.
• Additional information about the NUM-CODEX data transfer process plugin can be found int the repositories Wiki.

Features include:

• The process plugin has been ported to the DSF 1.x process plugin API and is now compatible with DSF 1.3.0.
• To enable transport of POLAR_MI data sets, the "data send" process is now compatible with additional FHIR resources.

Bugs fixed:

• Due to a limitation of the resource validator, false positive validation errors were generated for code from CodeSystems not available offline. A workaround has been implemented that removes not needed checks of code against the CodeSystem and expanded ValueSet.

Process released in the binary assets and GitHub Packages Maven Registry:

• data-transfer: codex-process-data-transfer-1.0.0.0.jar
  Add jar to /opt/bpe/process and make sure it can be read by the bpe Linux user or group (uid: 2202, gid: 2202) chown root:bpe; chmod 440, restart/start bpe.

Stand-alone validator release in the binary assets:

• validator: codex-process-data-transfer-validator-1.0.0.0.zip
  See the Stand-Alone-Validation description in the Wiki on how to use the validator.

Issues closed:

• Upgrade to DSF 1.x enhancement ready for release #118
• Start New Development Cycle ready for release #116
• Failed validation due to ICD Codes bug ready for release #114
• Hide fTTP API Key on startup connection test enhancement ready for release #113

0.7.0 - Improved Usability

General remarks:

• This release is compatible with the Data Sharing Framework version 0.9.0. This release does not work with older versions of the DSF.
• See NUM-CODEX internal mail communication for upgrade deadlines, especially regarding expiring client certificates of the GTH in the test- and production environments.
• To Update an existing DSF 0.7.0 installation, please see the 0.7.0 -> 0.8.0 Upgrade Guide.
• To Update an existing DSF 0.8.0 installation, please see the 0.8.0 -> 0.9.0 Upgrade Guide.
• For a fresh deployment please see the NUM‐CODEX DSF 0.9.0 Deployment and HiGHmed DSF 0.9.0 Deployment guides.
• Additional information about the NUM-CODEX data transfer process plugin can be found int the repositories Wiki.

Features include:

• The trigger process has been reworked. When started the trigger process, will signal existing process instances to stop executing.
• The process timeouts between the Data Integration Centers (DIC) and the Gecco Transfer Hub (GTH) as well as between the GTH and the Central Research Repository have been increased to support validation of larger data-sets at the CRR.
• The log level for some of the startup messages during validation support initialization has been reduced.
• A validation result summary has been added to the completed Task resource of send process executions (including dry-runs). The summary includes the number of resources validated successfully and the size (in bytes) of the resulting encrypted transfer bundle.
• An automatically send e-mail can be enabled for successful send process dry-runs. The mail will be send to the default recipient mail addresses configured in the DSF and will contain the completed Task resource for the dry-run execution.

Process released in the binary assets and GitHub Packages Maven Registry:

• data-transfer: codex-process-data-transfer-0.7.0.jar
  Add jar to /opt/bpe/process and make sure it can be read by the bpe Linux-user (chown root:bpe, chmod 440), restart/start bpe.

Stand-alone validator release in the binary assets:

• validator: codex-process-data-transfer-validator-0.7.0.zip
  See the Stand-Alone-Validation description in the Wiki on how to use the validator.

Issues closed:

• Send Mail if Dry-Run Successful #107
• Upgrade to DSF 0.9.0 #100
• Add a Validation Result Summary To the "start-data-send" Task Output #95
• Add a Warning Every Time the Stub Implementations Are Called #94
• Reduce Log Level at Startup #93
• Increase Process Timeouts #92
• GeccoFhirClientStub Throws Error #91
• Only one active instance of trigger process #90

0.6.0.1 - DSF 0.9.0 Backport for 0.6.0

General remarks:

• This is a backport of the 0.6.0 release for version 0.9.0 of the DSF. Allowing DSF 0.9.0 users to interact with DSF 0.7.0 deployments.
• Note: Do not deploy this backport release unless you know what you are doing.

Process released in binary assets and in GitHub Packages Maven Registry:

• data-transfer: codex-process-data-transfer-0.6.0.1.jar

Issues closed:

• Backport 0.6.0 Release to DSF 0.9.0 #106

0.6.0 - Improved FHIR Validation

With the 0.6.0 release false-positive validation errors have been fixed and a new parameter to run the data-send process in a dry-run mode has been added. With the dry-run mode, the data-send process will abort before storing the encrypted GECCO transfer dataset in the local DSF FHIR server and starting the data-translate process at the GECCO Transfer Hub (GTH). In the dry-mode no actual data is being transferred to the GTH.

A new version of the stand-alone Validator has been released as well. In order to fix the false-positive validation errors with the stand-alone validator, delete the three files listed below from the cache folder at default location ${java.io.tmpdir}/codex_gecco_validation_cache/.

Additional information on the new process release and the stand-alone validator can be found in the process Wiki.

To upgrade the process plugin from 0.5.1 perform the following steps:

1. Stop the DSF BPE

   docker-compose down
   

2. Replace the 0.5.1 process plugin with the new codex-process-data-transfer-0.6.0.jar

   sudo rm /opt/bpe/process/codex-process-data-transfer-0.5.1.jar
   sudo wget https://github.com/num-codex/codex-processes-ap1/releases/download/v0.6.0/codex-process-data-transfer-0.6.0.jar -P /opt/bpe/process
   sudo chmod 440 /opt/bpe/process/codex-process-data-transfer-0.6.0.jar
   sudo chown root:bpe /opt/bpe/process/codex-process-data-transfer-0.6.0.jar
   

3. Delete a profile snapshot in /opt/bpe/cache/StructureDefinition and two expanded ValueSets in /opt/bpe/cache/ValueSet

   sudo rm /opt/bpe/cache/StructureDefinition/https_www.netzwerk-universitaetsmedizin.de_fhir_StructureDefinition_sars-cov-2-ab-pnl-ser-pl-ia_1.0.4.json.gz
   sudo rm /opt/bpe/cache/ValueSet/https_www.netzwerk-universitaetsmedizin.de_fhir_ValueSet_rheumatological-immunological-diseases_1.0.5.json.gz
   sudo rm /opt/bpe/cache/ValueSet/https_www.netzwerk-universitaetsmedizin.de_fhir_ValueSet_vaccines-snomed_1.0.5.json.gz
   

   The files will be recreated during the next startup of the BPE to fix false-positive validation errors.

4. Modify the process exclude config in /opt/bpe/docker-compose.yml

   @@ -50,3 +50,3 @@
          ORG_HIGHMED_DSF_BPE_PROCESS_EXCLUDED: |
   -        wwwnetzwerk-universitaetsmedizinde_dataTranslate/0.5.1
   -        wwwnetzwerk-universitaetsmedizinde_dataReceive/0.5.1
   +        wwwnetzwerk-universitaetsmedizinde_dataTranslate/0.6.0
   +        wwwnetzwerk-universitaetsmedizinde_dataReceive/0.6.0

   Additional processes may be listed in your process exclude config.

5. Start the DSF BPE

   docker-compose up -d && docker-compose logs -f
   

Caution: To start the data-send process in version 0.6.0, you will need to use appropriate Task resources. For 0.6.0 examples see the wiki.

Fixes include:

• False-Positive validation errors generated for the SarsCoV2AbPnlSerPlIA, History of Vaccination and Rheumatological Immunological Diseases profiles have been fixed, requiring the deletion of three StructureDefinition/ValueSet cache enties.
• Process outcomes of the data-translate and data-receive processes where not saved as ouput parameters in their respective start-process Task resources.
• The test data generator used within the development test setup did not generate a UTF-8 encoded Bundle when executed on Windows machines.

Issues closed:

• Add Missing BpmnErrors #86
• Move Bundle Logging to Default off Logging Class #81
• Fix Test Data BundleGenerator Encoding #80
• Improve Status Output of Translate Process #79
• add simulate flag to send process to "just" get validation output from send process #77
• Validator Generates False-Positives #72

0.5.1 - Bug Fix Release For Version 0.5.0

This release fixes issues in the previous 0.5.0 release and adds support for blood-gas-panel resources. To upgrade from 0.5.0 perform the following steps:

1. Stop the DSF BPE

   docker-compose down
   

2. Replace process plugin

   sudo rm /opt/bpe/process/codex-process-data-transfer-0.5.0.jar
   sudo wget https://github.com/num-codex/codex-processes-ap1/releases/download/v0.5.1/codex-process-data-transfer-0.5.1.jar -P /opt/bpe/process
   sudo chmod 440 /opt/bpe/process/codex-process-data-transfer-0.5.1.jar
   sudo chown root:bpe /opt/bpe/process/codex-process-data-transfer-0.5.1.jar
   

3. Delete profile snapshots with issues in /opt/bpe/cache/StructureDefinition

   sudo rm /opt/bpe/cache/StructureDefinition/https_www.netzwerk-universitaetsmedizin.de_fhir_StructureDefinition_blood-gas-panel_1.0.5.json.gz
   sudo rm /opt/bpe/cache/StructureDefinition/https_www.netzwerk-universitaetsmedizin.de_fhir_StructureDefinition_carbon-dioxide-partial-pressure_1.0.4.json.gz
   sudo rm /opt/bpe/cache/StructureDefinition/https_www.netzwerk-universitaetsmedizin.de_fhir_StructureDefinition_diabetes-mellitus_1.0.json.gz
   sudo rm /opt/bpe/cache/StructureDefinition/https_www.netzwerk-universitaetsmedizin.de_fhir_StructureDefinition_inhaled-oxygen-concentration_1.0.4.json.gz
   sudo rm /opt/bpe/cache/StructureDefinition/https_www.netzwerk-universitaetsmedizin.de_fhir_StructureDefinition_organ-recipient_1.0.json.gz
   sudo rm /opt/bpe/cache/StructureDefinition/https_www.netzwerk-universitaetsmedizin.de_fhir_StructureDefinition_oxygen-partial-pressure_1.0.4.json.gz
   sudo rm /opt/bpe/cache/StructureDefinition/https_www.netzwerk-universitaetsmedizin.de_fhir_StructureDefinition_pH_1.0.4.json.gz
   sudo rm /opt/bpe/cache/StructureDefinition/https_www.netzwerk-universitaetsmedizin.de_fhir_StructureDefinition_symptoms-covid-19_1.0.json.gz
   

4. Modify the process exclude config in /opt/bpe/docker-compose.yml

   @@ -48,8 +48,8 @@
   services:
          ORG_HIGHMED_DSF_BPE_FHIR_SERVER_ORGANIZATION_IDENTIFIER_VALUE: TODO_ORGANIZATION_IDENTIFIER
          ORG_HIGHMED_DSF_BPE_FHIR_SERVER_BASE_URL: https://TODO_DSF_FRIR_SERVER_FQDN/fhir
          ORG_HIGHMED_DSF_BPE_PROCESS_EXCLUDED: |
   -        wwwnetzwerk-universitaetsmedizinde_dataTranslate/0.5.0
   -        wwwnetzwerk-universitaetsmedizinde_dataReceive/0.5.0
   +        wwwnetzwerk-universitaetsmedizinde_dataTranslate/0.5.1
   +        wwwnetzwerk-universitaetsmedizinde_dataReceive/0.5.1
   
          #TODO modify ORG_HIGHMED_DSF_BPE_PROCESS_EXCLUDED for later process versions
          #TODO add process specific environment variables, see process documentation

   Additional processes may be listed in your process exclude config.
5. Start the DSF BPE

   docker-compose up
   

Caution: Task resources used to start the 0.5.0 version of the process do not work with the new release. See Data Transfer Process Start - 0.5.1 on how to start a data transfer using the 0.5.1 release.

This release is only compatible with DSF starting from Version 0.7.0.

An installation guide for the DSF 0.7.0 in NUM-CODEX can be found here: DSF Wiki NUM-CODEX Deployment.
An installation guide for the DSF 0.7.0 in HiGHmed can be found here: DSF Wiki HiGHmed Deployment.

A detailed description of the process can be found here: NUM-CODEX Wiki Description v0.5.1.

The DSF process plugin can be found here: codex-process-data-transfer-0.5.1.jar (0.4 MB). This process-plugin requires hapi-fhir-client-5.1.0.jar to be installed as a regular DSF BPE plugin.

Information on how to configuration the DSF BPE for this process plugin, see Process Deployment and Configuration.

For information on how to use the process plugin as a stand-alone FHIR validator console tool, see Stand Alone Validation. The necessary file (jar and dependencies, can be found here: codex-process-data-transfer-validator-0.5.1.zip (79MB).

Fixes include:

• Bad Conditional Update URLs #74
• Validator Generates False-Positives #72

0.5.0 - FHIR Validation and Error Handling

With this release local FHIR validation is added to the process as well as feedback messages returning errors at the GECCO Transfer Hub (GTH) or Central Research Repository (CRR) back to the sending Data Integration Center (DIC). The new process release jar-file can be used as a stand-alone FHIR validator.

This release is only compatible with DSF starting from Version 0.7.0.

An installation guide for the DSF 0.7.0 in NUM-CODEX can be found here: DSF Wiki NUM-CODEX Deployment.
An installation guide for the DSF 0.7.0 in HiGHmed can be found here: DSF Wiki HiGHmed Deployment.

A detailed description of the process can be found here: NUM-CODEX Wiki Description v0.5.0.

The DSF process plugin can be found here: codex-process-data-transfer-0.5.0.jar (0.3 MB). This process-plugin requires hapi-fhir-client-5.1.0.jar to be installed as a regular DSF BPE plugin.

Information on how to configuration the DSF BPE for this process plugin, see Process Deployment and Configuration.

For information on how to use the process plugin as a stand-alone FHIR validator console tool, see Stand Alone Validation. The necessary file (jar and dependencies, can be found here: codex-process-data-transfer-validator-0.5.0.zip (79MB).

Fixes include:

• Generated FHIR search queries where not correctly escaped in the previous release, requiring a cumbersome workaround when using samply/blaze as the GECCO FHIR server. #53
• Missing recordedDate values (specified via data-absent-reason extensions) resulted in incompatible conditional update urls being generated. #4

Issues closed:

• Upgrade to DSF 0.7.0 #64
• Rename DB Volume in Docker Test Setup enhancement ready for release #62
• Fix Copy/Paste Error in Error Logging Statement bug ready for release #61
• Add the new #{date} place holder to bpmn and fhir files, switch to #{...} enhancement ready for release #59
• Move Connection Test to DSF 0.6.0 Method onProcessesDeployed enhancement ready for release #56
• Start New Development Cycle ready for release #54
• Bundle Search URLs Should Be URL Escaped bug ready for release #53
• CRR Pseudonym Needs Type Information enhancement ready for release #51
• Replace CodeSystem/ValueSet organization-type with organization-role enhancement ready for release #49
• Add Bundle Validation Implementation enhancement ready for release #39
• Add Feedback Messages and Delete of Binary Resources enhancement ready for release #36

This release contains contributions from @wetret and @hhund. Special thanks to @FloSeidel for helping to test this release.

First Release Candidate for 0.5.0

This pre-release (0.5.0-RC1) adds local FHIR validation to the process as well as feedback messages returning errors at the GECCO Transfer Hub (GTH) or central research repository (CRR) back to the sending data integration center (DIC). The new process release jar-file can be used as a stand-alone FHIR validator. This is a pre-release, we do not recommend to use this release in production.

This release is only compatible with DSF starting from Version 0.7.0-RC1.

An installation guide for the DSF 0.7.0 in NUM-CODEX can be found here: DSF Wiki NUM-CODEX Deployment (TODO).
An installation guide for the DSF 0.7.0 in HiGHmed can be found here: DSF Wiki HiGHmed Deployment (TODO).

A detailed description of the process can be found here: NUM-CODEX Wiki Description v0.5.0 (TODO).

The DSF process plugin can be found here: codex-process-data-transfer-0.5.0-RC1.jar (0.3 MB). This process-plugin requires hapi-fhir-client-5.1.0.jar to be installed as a regular DSF BPE plugin.

The deployment and configuration guide of the process can be found here: NUM-CODEX Wiki Deployment v0.5.0 (TODO).

For information on how to use the process plugin as a stand-alone FHIR validator console tool, see TODO. The necessary file (jar and dependencies, can be found here: codex-process-data-transfer-0.5.0-RC1.zip (77MB).

0.4.1.1 Conditional Update URL Improvements

This release improves the conditional update URLs within the transaction bundle send from DICs to the CRR. To upgrade to this release, replace the existing codex-process-data-transfer-0.4.1.jar with the new codex-process-data-transfer-0.4.1.1.jar. Data Integration Centers (DICs) are encouraged to upgrade, no upgrade is needed at the GTH or the CRR.

To Upgrade (may need sudo):

cd /opt/bpe
docker-compose stop
cd process
rm codex-process-data-transfer-0.4.1.jar
wget https://github.com/num-codex/codex-processes-ap1/releases/download/v0.4.1.1/codex-process-data-transfer-0.4.1.1.jar
chown root:bpe codex-process-data-transfer-0.4.1.1.jar
chmod 440 codex-process-data-transfer-0.4.1.1.jar
cd ..
docker-compose start

An installation guide for the DSF 0.6.0 in NUM-CODEX can be found here: DSF Wiki NUM-CODEX Deployment.
An installation guide for the DSF 0.6.0 in HiGHmed can be found here: DSF Wiki HiGHmed Deployment.

A detailed description of the process can be found here: NUM-CODEX Wiki Description v0.4.1.

The DSF process plugin can be found here: codex-process-data-transfer-0.4.1.1.jar. This process-plugin requires hapi-fhir-client-5.1.0.jar to be installed as a regular DSF BPE plugin.

The deployment and configuration guide of the process can be found here: NUM-CODEX Wiki Deployment v0.4.1.

Example Task-Resources to start the trigger and send processes can be found here: NUM-CODEX Task-Resources. Transaction-Bundles with example Patient-Resources among others can be found here: NUM-CODEX Bundle-Resources. The Patient-Resources include either an identifier with a DIC pseudonym or a Bloom Filter.

0.4.1 - Bug Fix Release For Version 0.4.0

This release fixes an issues with authenticating against the GECCO FHIR server using Basic Authentication (#45). To upgrade to this release, replace the existing codex-process-data-transfer-0.4.0.jar with the new codex-process-data-transfer-0.4.1.jar, modify the process exclude config within the BPE docker-compose.yml file and restart the DSF BPE container.

Be aware, using version 0.4.1 of the process requires using 0.4.1 instead of 0.4.0 within Task-Resources used for starting process instances.

An installation guide for the DSF 0.5.0 in NUM-CODEX can be found here: DSF Wiki NUM-CODEX Deployment.
An installation guide for the DSF 0.5.0 in HiGHmed can be found here: DSF Wiki HiGHmed Deployment.

A detailed description of the process can be found here: NUM-CODEX Wiki Description v0.4.1.

The DSF process plugin can be found here: codex-process-data-transfer-0.4.1.jar. This process-plugin requires hapi-fhir-client-5.1.0.jar to be installed as a regular DSF BPE plugin.

The deployment and configuration guide of the process can be found here: NUM-CODEX Wiki Deployment v0.4.1.

Example Task-Resources to start the trigger and send processes can be found here: NUM-CODEX Task-Resources. Transaction-Bundles with example Patient-Resources among others can be found here: NUM-CODEX Bundle-Resources. The Patient-Resources include either an identifier with a DIC pseudonym or a Bloom Filter.