Support for fetching FHIR resources via Bulk export mode

[chandrashekar-s]

Description of what I changed

Fixes #1031

• Changes to support fetching of FHIR resources from FHIR server via Bulk Export API. Currently, changes are made only for FULL_RUN, INCREMENTAL_RUN changes will be made in a separate PR.
• A parameter called fhirFetchMode has been added, which explicitly specifies which mode the application should use to fetch FHIR resources from source (this avoids the confusion of dynamically figuring out the mode from the parameters configured).

E2E test

• Added a e2e test for BULK_EXPORT mode.
• Added unit tests.

TESTED:

• Manually tested the application by triggering a FULL run from the controller UI.

Checklist: I completed these to help reviewers :)

• I have read and will follow the review process.

• I am familiar with Google Style Guides for the language I have coded in.

  No? Please take some time and review Java and Python style guides.

• My IDE is configured to follow the Google code styles.

  No? Unsure? -> configure your IDE.

• I have added tests to cover my changes. (If you refactored existing code that was well tested you do not have to add tests)

• I ran mvn clean package right before creating this pull request and added all formatting changes to my commit.

• All new and existing tests passed.

• My pull request is based on the latest changes of the master branch.

  No? Unsure? -> execute command git pull --rebase upstream master

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 36.70412% with 169 lines in your changes missing coverage. Please review.

Project coverage is 51.74%. Comparing base (8ea0914) to head (8875523).

Files	Patch %	Lines
...c/main/java/com/google/fhir/analytics/FhirEtl.java	0.00%	56 Missing ⚠️
...main/java/com/google/fhir/analytics/FetchUtil.java	5.40%	35 Missing ⚠️
...java/com/google/fhir/analytics/BulkExportUtil.java	56.25%	14 Missing and 7 partials ⚠️
...com/google/fhir/analytics/BulkExportApiClient.java	66.66%	11 Missing and 6 partials ⚠️
...ava/com/google/fhir/analytics/PipelineManager.java	37.50%	15 Missing ⚠️
...ain/java/com/google/fhir/analytics/ReadJsonFn.java	47.61%	9 Missing and 2 partials ⚠️
...google/fhir/analytics/converter/JsonDateCodec.java	30.00%	5 Missing and 2 partials ⚠️
...java/com/google/fhir/analytics/DataProperties.java	22.22%	5 Missing and 2 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #1082      +/-   ##
============================================
- Coverage     52.05%   51.74%   -0.31%     
- Complexity      654      669      +15     
============================================
  Files            89       95       +6     
  Lines          5396     5577     +181     
  Branches        708      724      +16     
============================================
+ Hits           2809     2886      +77     
- Misses         2325     2412      +87     
- Partials        262      279      +17     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[chandrashekar-s]

Hi @bashir2, this PR contains changes to support FULL_RUN only. I will raise a separate PR for supporting INCREMENTAL_RUN (have created this issue for the same).

[bashir2]

Thanks @chandrashekar-s for the change and sorry for the delay caused by DevDays and issues around it. I had a look at most of the non-test code and my main/major comment is about whether we should create temporary files or not (I am suggesting not).

[chandrashekar-s]

Thanks @bashir2 for the review. I have addressed the comments in the latest commit.

[bashir2]

Thanks @chandrashekar-s for the changes. Most of my suggestions are minor; there is also a question about the expiry period left from the previous round (not sure if we can do anything about it from the client side). Otherwise please feel free to merge this after addressing the comments.

[chandrashekar-s]

Hi @bashir2, I have addressed your review comments in the latest commit. Can you please have a look (PR is not yet approved for merge)

[bashir2]

Thanks @chandrashekar-s for the changes and sorry that I forgot to flip the approve flag last time; please feel free to merge, this time for real :-)