WHFHRI-662: copy/push of all branch support-2.x files from internal g…

…h-repo to public alvearie gh-repo

Signed-off-by: Aram Openden <[email protected]>

.gitignore

@@ -0,0 +1,19 @@
+# Ignore IntelliJ files
+.idea
+.iml
+.ruby-version
+
+# Ignore Gradle project-specific cache directory
+.gradle
+
+# Ignore Gradle build output directory
+build
+
+.DS_Store
+test/test.iml
+test/demo/testenv.sh
+test/dependencies/
+test/test_data/synthea/
+test/test_data/synthetic_mass/
+
+hri-flink-validation-fhir.iml

CONTRIBUTING.md

@@ -0,0 +1,10 @@
+# Guiding Principles for Contribution
+First of all, thank you for taking the time to contribute! The HRI Team values your contribution. 
+
+In general, contributions can be made using the standard fork and pull request process. We use the [Git Flow](https://nvie.com/posts/a-successful-git-branching-model/) branching model, so branch off of and submit the pull request back to the `develop` branch. If updating an older release, submit a pull request against the associated `support-<major>.x` branch. If one does not exist, contact us, and we will create one.
+
+The GitHub actions may not run successfully in your forked repository without several secrets and external resources used for integration testing. You can ignore this and rely on the actions that will run in our repository when you create the pull request, but you should be able to run local unit tests to test your changes.
+
+Once the pull request is reviewed and approved and all the integration tests pass, we will merge it and handle releasing the updates.
+
+If making a significant contribution, please reach out to the development team's Slack channel, [#health-record-ingestion](https://alvearie.slack.com/archives/C01GM43LFJ6), so that we can coordinate the desired changes.

MAINTAINERS.md

@@ -0,0 +1,4 @@
+# MAINTAINERS
+
+- David N Perkins - [email protected]
+- Aram Openden - [email protected]

README.md

@@ -1,4 +1,104 @@
-# hri-flink-validation-fhir
-HRI Flink job which validates that the input data meets the FHIR schema specification 
+# HRI Flink Validation FHIR
 
-ready to copy/push support-2.x branch content
+The Alvearie Health Record Ingestion service: a common 'Deployment Ready Component' designed to serve as a “front door” for data for cloud-based solutions. See our [documentation](https://alvearie.io/HRI) for more details.
+
+This repo contains the code for the HRI Flink Validation FHIR Job of the HRI, which validates that the incoming (HRI Record's) message payload adheres to the [FHIR specification](https://www.hl7.org/fhir/overview.html).
+
+Note: This software uses Apache Flink (https://flink.apache.org/) for streaming data and is written in Scala (2.12.11).
+
+## Communication
+* Please [join](https://alvearie.io/contributions/requestSlackAccess/) our Slack channel for further questions: `#health-record-ingestion`
+* Please see recent contributors or [maintainers](MAINTAINERS.md)
+
+## Getting Started
+
+### Prerequisites
+* Java 1.8 - you can use an official [distribution](https://www.java.com/en/download/manual.jsp) or a package manager like `homebrew` for mac
+* Scala 2.12.11 - you can use an official [distribution](https://www.scala-lang.org/download/) or a package manager like `homebrew` for mac
+* Java/Scala IDE (Optional) - we use IntelliJ, but it requires a licensed version.
+* Ruby (Optional) - required for integration tests. See [testing](test/README.md) for more details.
+* IBM Cloud CLI (Optional) - useful for local testing. Installation [instructions](https://cloud.ibm.com/docs/cli?topic=cloud-cli-getting-started).
+
+### Building
+From the base directory, run `./gradlew clean build`. This will download dependencies and run all the unit tests. (Some output has been omitted for the sake of concision)
+
+This depends on the `hri-flink-pipeline-core` [GitHub repo](https://github.com/Alvearie/hri-flink-pipeline-core) and its published packages. 
+
+```
+hri-flink-validation-fhir % ./gradlew clean build
+
+> Task :validator:test
+Discovery starting.
+Discovery completed in 186 milliseconds.
+Run starting. Expected test count is: 10
+
+FhirJsonValidationTest:
+Run completed in 978 milliseconds.
+Total number of tests run: 10
+Suites: completed 2, aborted 0
+Tests: succeeded 10, failed 0, canceled 0, ignored 0, pending 0
+All tests passed.
+
+> Task :validator:reportTestScoverage
+[info] Found 1 subproject scoverage data directories [hri-flink-validation-fhir/validator/build/scoverage]
+
+> Task :validator:reportScoverage
+[info] Found 1 subproject scoverage data directories [/hri-flink-validation-fhir/validator/build/scoverage]
+Scoverage report:
+  hri-flink-validation-fhir/validator/build/reports/scoverage/index.html
+
+> Task :validation:test
+Discovery starting.
+Discovery completed in 100 milliseconds.
+Run starting. Expected test count is: 8
+FhirValidationJobTest:
+Run completed in 568 milliseconds.
+Total number of tests run: 8
+Suites: completed 2, aborted 0
+Tests: succeeded 8, failed 0, canceled 0, ignored 0, pending 0
+All tests passed.
+
+> Task :validation:reportTestScoverage
+[info] Found 1 subproject scoverage data directories [hri-flink-validation-fhir/validation/build/scoverage]
+
+> Task :validation:reportScoverage
+[info] Found 1 subproject scoverage data directories [hri-flink-validation-fhir/validation/build/scoverage]
+Scoverage report:
+  hri-flink-validation-fhir/validation/build/reports/scoverage/index.html
+
+BUILD SUCCESSFUL in 54s
+27 actionable tasks: 27 executed
+
+```
+
+## CI/CD
+GitHub actions is used for CI/CD. It runs unit tests, builds the code, and then runs integration tests on our Flink cluster using Event Streams and an HRI Management API instance. 
+
+Each branch uses its own topics, so different builds don't interfere with each other. Integration tests will clean up after themselves cancelling the Flink job, deleting the job jar, and the Event Streams topics.
+
+The Flink logs are available for trouble shooting. They can be viewed in the Flink UI or the Kubernetes logs. The logs for all jobs are combined together, so you may need to search to a specific time frame or look for specific keywords.
+
+
+## Releases
+Releases are created by creating Git tags, which trigger a GitHub Actions build that publishes a release version in GitHub Packages, see [Overall strategy](https://SOME_NEW_URL/docs/wiki/Overall-Project-Branching,-Test,-and-Release-Strategy) for more details.
+
+## Code Overview
+
+### Classes
+The code contains two sub projects: 
+- A FHIR validator library - is in its own project so that it can be published and used by others who might want to use it in a custom validation job.
+- A Flink FHIR validation job - uses the validator library; currently validates that the records meet the 'bundle' FHIR schema specification. A future goal is to support FHIR profile validation.
+
+### Tests
+This repository contains both unit tests and end-to-end integration tests.
+
+- The validator and validation job have files containing respective unit tests
+- The fhir.jobtest package within the Validation project contains examples of an end-to-end streaming job test that use the Flink Framework Unit Test support of the [MiniClusterWithClientResource](https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/testing.html#junit-rule-miniclusterwithclientresource)
+
+#### Test Coverage
+The HRI team requires that your code has at least 90% unit test code coverage. Anything less will likely not be accepted as a new contribution.
+
+The build automatically creates a Scoverage Test Report that contains test coverage percentages and a coverage file at `hri-flink-validation-fhir/validator/build/reports/scoverage/index.html` and `hri-flink-validation-fhir/validation/build/reports/scoverage/index.html`. 
+
+## Contribution Guide
+Please read [CONTRIBUTING.md](CONTRIBUTING.md) for details on our code of conduct, and the process for submitting pull requests to us.

build.gradle

@@ -0,0 +1,85 @@
+/*
+ * (C) Copyright IBM Corp. 2021
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+allprojects {
+    apply plugin: 'scala'
+    group = 'org.alvearie.hri.flink'
+    version = '2.1-1.0.1'
+
+    ext {
+        pipelineCoreVersion = '2.1-1.0.1'
+        javaVersion = '1.8'
+        flinkVersion = '1.10.0'
+        scalaBinaryVersion = '2.12'
+        scalaVersion = '2.12.11'
+        scalaTestVersion = '3.1.1'
+        jacksonVersion = '2.12.0'
+        slf4jVersion = '1.7.7'
+        log4jVersion = '1.2.17'
+    }
+
+    sourceCompatibility = javaVersion
+    targetCompatibility = javaVersion
+    tasks.withType(JavaCompile) {
+        options.encoding = 'UTF-8'
+    }
+
+    repositories {
+      //TODO: replace with GH Packages Maven repo location
+        ['wh-wffh-maven-local'].each { repo ->
+            maven {
+                credentials {
+                    username = findProperty('user') ?: System.getenv('user')
+                    password = findProperty('password') ?: System.getenv('password')
+                }
+
+                url "NEW_GH_PACKAGES_URL"
+            }
+        }
+        mavenCentral()
+        mavenLocal()
+    }
+
+    ext {
+        branch = System.getenv('BRANCH_NAME') != null
+                ? System.getenv('BRANCH_NAME')
+                : getWorkingBranch()
+    }
+
+    // If not running in travis add 'local' to the version to support local development
+    if (System.getenv('BRANCH_NAME') == null || System.getenv('BRANCH_NAME') == "") {
+        version = "${branch}-local-SNAPSHOT"
+    } else if (System.getenv('GIT_TAG') == null || System.getenv('GIT_TAG') == "") {
+        version = "${branch}-SNAPSHOT"
+    } else if (System.getenv('GIT_TAG') == "v${version}") {
+        version = "${version}"
+    } else {
+        throw new InvalidUserDataException(String.format("The tag '%s' does not match with the current release version '%s'",System.getenv('TRAVIS_TAG'),"${version}"));
+    }
+}
+
+dependencies {
+    // Scala lib
+    implementation "org.scala-lang:scala-library:${scalaVersion}"
+}
+
+// prevents parent project from producing build output
+jar.enabled = false
+
+/**
+ * Get the name of the working branch of the project
+ *
+ * @return Name of the working branch
+ */
+def getWorkingBranch() {
+    // Triple double-quotes for the breaklines
+    def workingBranch = """git --git-dir=${rootDir}/.git
+                               --work-tree=${rootDir}
+                               rev-parse --abbrev-ref HEAD""".execute().text.trim()
+    println "Working branch: " + workingBranch
+    return workingBranch
+}
+

elastic.crt

@@ -0,0 +1,19 @@
+-----BEGIN CERTIFICATE-----
+MIIDHTCCAgWgAwIBAgIUL5g6t1HmNOt8OOqUWyuQZeWsrUAwDQYJKoZIhvcNAQEL
+BQAwHjEcMBoGA1UEAwwTSUJNIENsb3VkIERhdGFiYXNlczAeFw0xODExMjExMTQ3
+MjdaFw0yODExMTgxMTQ3MjdaMB4xHDAaBgNVBAMME0lCTSBDbG91ZCBEYXRhYmFz
+ZXMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC3hxfosYeXdmrQRHlU
+vMg91IWSCuGZ16RbHbZ2p62SEUfBJemr4MkTr48YZTfiTCZjckWWGlBlvUYr0cYY
+KM3OPtkhk3lmkAQlJcMCPfiF2+mBxAUqrIibCgmQijylRJW4sEFQgSbrXewDywEd
+8JUgm+82DwLYy6vnkKJjzcUZk5OKS1Uv7w0pHUtUMtNLaSyKksDdE9d9sj6lWTFJ
+W9CyXCp5iffJ97oewIh47nIFYU5D5MptCDYr6wifOWvhiduYD6WMR7wDgLMv/ra9
+Zt4nt7MYglRmKnO77DZI5Y9lT4gw4FiU0k65aWnXrk+1DZ4jqFdWk3VpoXhA0IuX
+ZMbPAgMBAAGjUzBRMB0GA1UdDgQWBBQYWTX4dsUL3/LnQAXQLLzh46yDATAfBgNV
+HSMEGDAWgBQYWTX4dsUL3/LnQAXQLLzh46yDATAPBgNVHRMBAf8EBTADAQH/MA0G
+CSqGSIb3DQEBCwUAA4IBAQAPwYrOIQMEreNuHY/68lFJimB6GCinGsJw8YIBBQwA
+tdJMOXU8bXGrEHEuZ6Qaho7Q5dV2bB6GVKTMqhSEdrvuyxKniQ90jdkQSk2VHhDe
+H+6i04hA9TkKT6ooLwMPc1LYYzqDljEkfKlLIPWCkOAozD3cyc26pV/35nG7WzAF
+xw7S3jAyB3WcJDlWlSWGTn58w3EHxzVXvKT6Y9eAdKp4SjUHyVFsL5xtSyjH8zpF
+pZKK8wWNUwgWQ66MNh8Ckq732JZ+so6RAfb4BbNj45I3s9fuZSYlvjkc5/+da3Ck
+Rp6anX5N6yIrzhVmAgefjQdBztYzdfPhsJBkS/TDnRmk
+-----END CERTIFICATE-----

gradle/wrapper/gradle-wrapper.properties

@@ -0,0 +1,5 @@
+distributionBase=GRADLE_USER_HOME
+distributionPath=wrapper/dists
+distributionUrl=https\://services.gradle.org/distributions/gradle-6.4-bin.zip
+zipStoreBase=GRADLE_USER_HOME
+zipStorePath=wrapper/dists