Added member oid extension for each observation

[kant777]

This PR adds the member OID as the extension to all observations in the elr messages.

Test Steps:

1. Include steps to test these changes

Changes

• The ConditionStamper will now handle both condition lookups and Member OID stamping based on the logic and constants from ObservationMappingConstants

Checklist

Testing

• Tested locally? yes
• Ran ./prime test or ./gradlew testSmoke against local Docker ReportStream container? yes
• Added tests? yes

Linked Issues

• Fixes Map CSTE Member OID to member-oid extension during convert-step #14511

[github-actions]

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

OpenSSF Scorecard

Package	Version	Score	Details

Scanned Manifest Files

[github-actions]

Test Results

1 272 tests  +1   1 268 ✅ +1   7m 39s ⏱️ +11s
  165 suites ±0       4 💤 ±0 
  165 files   ±0       0 ❌ ±0 

Results for commit 090f937. ± Comparison against base commit f1890eb.

♻️ This comment has been updated with latest results.

[github-actions]

Integration Test Results

 60 files   60 suites   44m 23s ⏱️
428 tests 418 ✅ 10 💤 0 ❌
431 runs  421 ✅ 10 💤 0 ❌

Results for commit 090f937.

♻️ This comment has been updated with latest results.

[sonarqubecloud]

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
92.7% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[jalbinson]

Getting unit test failures. Other than that it looks good given the structure has been validated by Engagement.

[JFisk42]

I reviewed this from a perspective of trying to help @kant777 figure out the failing tests. You can ignore 90% of what I'm saying if you already figured them out :D

I did some manual testing and ran the tests locally. The new stamping works as expected (see details below) so great job there! As for the unit tests I left a couple comments, mostly it boils down to "the unit tests in FHIRConverterTests have some oddities". I'd like us to log a ticket to iron out some of that.

Once those are fixed the build is going to fail on the smoke tests because the output for those has changed, but apparently only for the MARS_OTC scenarios. (I didn't look into why that is but I'm assuming it is because of the specific condition that is being used in those scenarios).

To run our smoke tests locally you can use the command ./gradlew testEnd2EndUP. They can be a little confusing to parse at the moment but the basics of this test is that it runs through 5 scenarios. It begins by uploading each scenario. Then, one scenario at a time, it will poll the history endpoint to determine if that has been delivered. You'll see output similar to this 5 times:

Starting validation for scenario: Sending FHIR Report, Receiving FHIR (mars-otc-elr) // Scenario name and details
Polling for history endpoint for report ID: 6d57baa4-9899-4f96-8da7-4691b680bfab. (Max poll time 150 seconds) // Log of Report ID
Report 6d57baa4-9899-4f96-8da7-4691b680bfab status was Delivered after 5 seconds // Final status of history endpoint for that Scenario's Report
{
//  This is the output of the final history endpoint result. It's useful for finding out the names of the received reports //
}
Examining sent reports for the posted report ID 6d57baa4-9899-4f96-8da7-4691b680bfab // Now onto result verification
The report was received by MARS_OTC_ELR_FHIR_A_E2E // This is a/the expected receiver for that scenario
File successfully uploaded to blob store as 0507cdf1-f771-4523-826a-cbcd4957fce7.fhir // Verified that the report was found
The contents of 0507cdf1-f771-4523-826a-cbcd4957fce7.fhir matches the expected data // Verified the contents of the report 

Ultimately, to fix the test all that was needed was to paste in the new Observation.code results that included the stamping for the expected MARS FHIR results. We can walk through this after standup tomorrow. You can find the settings (and location of expected data files) for each scenario here.

Manual testing results were a success: I submitted an ORU_R01 report through ignore.ignore-full-elr-e2e. This successfully made its way through the pipeline and the resulting FHIR in my sftp folder matched the expected format. 🎉 🎉

"code": {  
  "coding": [  
    {  
      "extension": [  
        ...  
        {  
          "url": "https://reportstream.cdc.gov/fhir/StructureDefinition/condition-code",  
          "valueCoding": {  
            "extension": [  
              {  
                "url": "https://reportstream.cdc.gov/fhir/StructureDefinition/test-performed-member-oid",  
                "valueString": "2.16.840.1.113762.1.4.1146.1470"  
              }  
            ],  
            "system": "SNOMEDCT",  
            "code": "406575008",  
            "display": "Infection caused by vancomycin resistant enterococcus (disorder)"  
          }  
        },
        ... 

[JFisk42]

Worked with Kant over the last few days on the tests and a small implementation tweak. Everything looks good and is passing locally now 👍

[arnejduranovic]

Include new event structure in PR so Jessica can attend to the dashboard

[arnejduranovic]

Code changes look good and I did the following testing:

Submitted an HL7v2 message on main branch and this branch and compared. I confirmed:
- Application Insights custom events DO NOT contain the newly added info. This is good because it does not break current Dashboard functionality.

ITEM_ROUTED event on MAIN:

{"receiverName":"hhsprotect.mars-otc-elr","bundleDigest":{"observationSummaries":[{"testSummary":[{"conditions":[{"system":"SNOMEDCT","code":"651000146102","display":"Middle East respiratory syndrome (disorder)"}],"testPerformedSystem":"http://loinc.org","testPerformedCode":"94501-4","testPerformedDisplay":"Influenza virus B Ag [Presence] in Respiratory specimen by Rapid immunoassay"}]},{"testSummary":[{"conditions":[{"system":"SNOMEDCT","code":"840539006","display":"Disease caused by severe acute respiratory syndrome coronavirus 2 (disorder)"}],"testPerformedSystem":"http://loinc.org","testPerformedCode":"94500-6","testPerformedDisplay":"Influenza virus A Ag [Presence] in Respiratory specimen by Rapid immunoassay"}]},{"testSummary":[{"conditions":[{"system":"SNOMEDCT","code":"840539006","display":"Disease caused by severe acute respiratory syndrome coronavirus 2 (disorder)"}],"testPerformedSystem":"http://loinc.org","testPerformedCode":"94558-4","testPerformedDisplay":"SARS-CoV-2 (COVID-19) Ag [Presence] in Respiratory specimen by Rapid immunoassay"}]},{"testSummary":[{"conditions":[{"system":"ReportStream","code":"AOE","display":"Ask at order entry question"}],"testPerformedSystem":"http://loinc.org","testPerformedCode":"35659-2","testPerformedDisplay":"Age at specimen collection"}]}],"patientState":[],"performerState":[],"orderingFacilityState":[],"eventType":"ORU^R01^ORU_R01"}}

ITEM_ROUTED event on BRANCH:

{"receiverName":"hhsprotect.mars-otc-elr","bundleDigest":{"observationSummaries":[{"testSummary":[{"conditions":[{"system":"SNOMEDCT","code":"651000146102","display":"Middle East respiratory syndrome (disorder)"}],"testPerformedSystem":"http://loinc.org","testPerformedCode":"94501-4","testPerformedDisplay":"Influenza virus B Ag [Presence] in Respiratory specimen by Rapid immunoassay"}]},{"testSummary":[{"conditions":[{"system":"SNOMEDCT","code":"840539006","display":"Disease caused by severe acute respiratory syndrome coronavirus 2 (disorder)"}],"testPerformedSystem":"http://loinc.org","testPerformedCode":"94500-6","testPerformedDisplay":"Influenza virus A Ag [Presence] in Respiratory specimen by Rapid immunoassay"}]},{"testSummary":[{"conditions":[{"system":"SNOMEDCT","code":"840539006","display":"Disease caused by severe acute respiratory syndrome coronavirus 2 (disorder)"}],"testPerformedSystem":"http://loinc.org","testPerformedCode":"94558-4","testPerformedDisplay":"SARS-CoV-2 (COVID-19) Ag [Presence] in Respiratory specimen by Rapid immunoassay"}]},{"testSummary":[{"conditions":[{"system":"ReportStream","code":"AOE","display":"Ask at order entry question"}],"testPerformedSystem":"http://loinc.org","testPerformedCode":"35659-2","testPerformedDisplay":"Age at specimen collection"}]}],"patientState":[],"performerState":[],"orderingFacilityState":[],"eventType":"ORU^R01^ORU_R01"}}

Furthermore, the bundleDigestExtractor did not need its tests updated, which is another way of verifying that the bundleDigest remains unchanged: https://github.com/CDCgov/prime-reportstream/blob/63b4fd9eb1e272eee10e7abfebf321eff84db75b/prime-router/src/test/kotlin/azure/observability/bundleDigest/FhirPathBundleDigestExtractorStrategyTests.kt

I also downloaded the submitted blobs from the destination-filter folder via Azure Storage explorer and compared the stamping differences between the branch and main. I verified the branch is stamping the OID in such a way as to not chance the path of the condition code stamp and is valid FHIR:

BRANCH:

{
    "url": "https://reportstream.cdc.gov/fhir/StructureDefinition/condition-code",
    "valueCoding": {
      "extension": [
        {
          "url": "https://reportstream.cdc.gov/fhir/StructureDefinition/test-performed-member-oid",
          "valueString": "2.16.840.1.113762.1.4.1146.878"
        }
      ],
      "system": "SNOMEDCT",
      "code": "651000146102",
      "display": "Middle East respiratory syndrome (disorder)"
    }
  }

MAIN:

{
  "url": "https://reportstream.cdc.gov/fhir/StructureDefinition/condition-code",
  "valueCoding": {
    "system": "SNOMEDCT",
    "code": "651000146102",
    "display": "Middle East respiratory syndrome (disorder)"
  }
}

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
88.9% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud