Clinical Reasoning: $evaluate-measure returning faulty results after posting multiple Measure Bundles

[krauzer]

I'm running the latest version of the HAPI FHIR Server (7.2.1 FHIR 4.0.1/R4) with Clinical Reasoning Enabled. Using Postgres v. 16 as the database.

Using test data from ecqm-content-qicore-2024-subset, when I post the BreastCancerScreeningFHIR-bundle, I can successfully run $evaluate-measure on the Measure get the correct MeasureReport result back:

But if I subsequently post a Measure bundle like the ColonCancerScreenFHIR-bundle, running $evaluate-measure on BreastCancerScreeningFHIR now returns incorrect results:

The MeasureReports associated with BreastCancerScreeningFHIR still exist (e.g. MeasureReport/measurereport-denom-EXM125) so it seems like there is some sort of bug here.

Analyzing the results of posting the bundles indicates that shared value sets between them get updated, which might be the underlying cause.

For instance, if I post the BreastCancerScreeningFHIR-bundle twice in a row, the second time will send responses all with the code SUCCESSFUL_UPDATE_NO_CHANGE

But If I post ColonCancerScreenFHIR-bundle in between them (which breaks the $evaluate-measure for BreastCancerScreeningFHIR), the second posting of BreastCancerScreeningFHIR-bundle causes a number of the values sets (which have the same versions across bundles) to indicate they've been updated (parsed summaries below):

Response Resource Location: 200 OK - Library/FHIRHelpers/_history/9
Response Resource Type: OperationOutcome
Response Response Code SUCCESSFUL_UPDATE
-----------------------------------
Response Resource Location: 200 OK - ValueSet/2.16.840.1.113883.3.464.1003.101.12.1023/_history/7
Response Resource Type: OperationOutcome
Response Response Code SUCCESSFUL_UPDATE
-----------------------------------
Response Resource Location: 200 OK - ValueSet/2.16.840.1.113883.3.464.1003.118.12.1300/_history/7
Response Resource Type: OperationOutcome
Response Response Code SUCCESSFUL_UPDATE
-----------------------------------
Response Resource Location: 200 OK - ValueSet/2.16.840.1.113762.1.4.1/_history/9
Response Resource Type: OperationOutcome
Response Response Code SUCCESSFUL_UPDATE
-----------------------------------
Response Resource Location: 200 OK - ValueSet/2.16.840.1.113883.3.464.1003.101.12.1025/_history/7
Response Resource Type: OperationOutcome
Response Response Code SUCCESSFUL_UPDATE
-----------------------------------
Response Resource Location: 200 OK - ValueSet/2.16.840.1.113883.3.464.1003.113.12.1075/_history/7
Response Resource Type: OperationOutcome
Response Response Code SUCCESSFUL_UPDATE
-----------------------------------
Response Resource Location: 200 OK - ValueSet/2.16.840.1.114222.4.11.837/_history/9
Response Resource Type: OperationOutcome
Response Response Code SUCCESSFUL_UPDATE
-----------------------------------
Response Resource Location: 200 OK - ValueSet/2.16.840.1.113883.3.464.1003.113.12.1074/_history/7
Response Resource Type: OperationOutcome
Response Response Code SUCCESSFUL_UPDATE
-----------------------------------
Response Resource Location: 200 OK - ValueSet/2.16.840.1.114222.4.11.836/_history/9
Response Resource Type: OperationOutcome
Response Response Code SUCCESSFUL_UPDATE
-----------------------------------
Response Resource Location: 200 OK - ValueSet/2.16.840.1.113883.3.526.3.1240/_history/7
Response Resource Type: OperationOutcome
Response Response Code SUCCESSFUL_UPDATE
-----------------------------------
Response Resource Location: 200 OK - ValueSet/2.16.840.1.113883.3.464.1003.196.12.1510/_history/7
Response Resource Type: OperationOutcome
Response Response Code SUCCESSFUL_UPDATE
-----------------------------------
Response Resource Location: 200 OK - ValueSet/2.16.840.1.113883.3.464.1003.110.12.1082/_history/7
Response Resource Type: OperationOutcome
Response Response Code SUCCESSFUL_UPDATE
-----------------------------------
Response Resource Location: 200 OK - ValueSet/2.16.840.1.113883.3.464.1003.101.12.1001/_history/7
Response Resource Type: OperationOutcome
Response Response Code SUCCESSFUL_UPDATE
-----------------------------------
Response Resource Location: 200 OK - ValueSet/2.16.840.1.113883.3.464.1003.101.12.1088/_history/7
Response Resource Type: OperationOutcome
Response Response Code SUCCESSFUL_UPDATE
-----------------------------------
Response Resource Location: 200 OK - ValueSet/2.16.840.1.114222.4.11.3591/_history/9
Response Resource Type: OperationOutcome
Response Response Code SUCCESSFUL_UPDATE
-----------------------------------
Response Resource Location: 200 OK - ValueSet/2.16.840.1.113883.3.666.5.307/_history/9
Response Resource Type: OperationOutcome
Response Response Code SUCCESSFUL_UPDATE
-----------------------------------
Response Resource Location: 200 OK - ValueSet/2.16.840.1.113883.3.464.1003.101.12.1016/_history/7
Response Resource Type: OperationOutcome
Response Response Code SUCCESSFUL_UPDATE
-----------------------------------

It leads me to believe there is some sort of bug where updating the value sets causes the associated measure evaluation to break.

To replicate this, you can:

1. Post the BreastCancerScreeningFHIR-bundle
2. See that $evaluate-measure on BreastCancerScreeningFHIR works.
3. Post the ColonCancerScreenFHIR-bundle
4. See that $evaluate-measure on BreastCancerScreeningFHIR no longer works.
5. Post the BreastCancerScreeningFHIR-bundle again and see the valueSets have been updated in the response.
6. See that $evaluate-measure on BreastCancerScreeningFHIR works again.