Inconsistencies in uploading IG package resources

[fongsean]

I was using the following command to test uploading an IG to a local HAPI R4 server (with default config):
UploadFIG -s http://build.fhir.org/ig/aehrc/smart-forms-ig/package.tgz -d http://localhost:8080/fhir -df json

My goal is to recurrently update IG resources in the server whenever I update my IG's CI build: http://build.fhir.org/ig/aehrc/smart-forms-ig/. The IG consists of CodeSystems, Questionnaires, StructureDefinitions and ValueSets.

I ran the above command a total of four times - some runs provided different outcomes than others.

Run 1
All resources are created without fail.

Run 2:
All resources except Questionnaires fail to be created. Questionnaires copies are created.

Run 3:
All resources except Questionnaires fail to be created. Questionnaires copies are created again.

Run 4:
All resources except Questionnaires were updated successfully, Questionnaires cannot be updated as there are multiple instances with the same canonical url.

Expectation:
Run 1 aligns with expectations. Resources are created if they don't exist.
Run 2-3 is a little weird. Seems like its trying to create resources instead of updating them.
Run 4 aligns with expectations. I imagine if a resource already exists, the tool should attempt to update it - which run 4 did well in.

Here's the log file for reference:
uploadfig-smartformsig-log-2(2).txt

Cheers!

[brianpos]

There are 2 issues embedded in here

Failed to compare two elements in the array.

Which is a simple issue and I've just pushed a change to resolve that today.
The other looks like a HAPI server issue. I'm setting up a local one to try things out myself.
This appears to be a parsing issue, and side effect is attempting to push another instance of the same canonical resource

[fongsean]

Thanks Brian.

Copying our discussion/observations over from chat.fhir.org, it seems like the behaviour of run 2 and 3 attempting to create new resources is a result of an index update timing issue in HAPI.

The number of runs didn't matter, what mattered is how long it took HAPI to index the newly uploaded resources from run 1. Only after about a minute or so, the resources are indexed - therefore can be detected and updated. Before that indexing happens, the tool will try to create new resources. In the case of Questionnaires, the tool re-creates them successfully while for the other resources, the tool throws UnprocessableEntity errors.

Once the resources are indexed, the tool attempts to update the resources. Questionnaires cannot be updated since multiple instances was loaded from previous runs(the tool doesn't know which one to update!). The other resources were only loaded once so they can be updated successfully.

[brianpos]

If anyone else stumbles on this, the error message you'll get in the output logs is this:

ERROR: Canonical http://....canonicalUrl|0.1.0 has multiple instances already loaded - Must be resolved manually as 
unable to select which to update

Currently the only way to resolve this is to manually select one of the resources to delete.
I'm adding an option to delete the duplicate resource(s).