Add support for ahead-of-time compilation (#248)

* Add more logging messages

* Fix log4j2 configuration

* Implement apache HTTP client

* Implement apache HTTP async client 5

* Fix timeout

* Fixes

* Fix hashing bug

* Fix conversion of byte stream to string

* Implement POST request streaming

* Disable the storing and hashing of responses when the parseResults parameter in the config is false

* Move utility classes

* StreamEntityProducer can send fixed-sized data and is reproducible now

* Make QueryHandler return stream supplier and info about query being cached

* Change RequestFactory behavior

* cached queries will be sent with fixed-sizes request
* requests of cached queries will be cached as well (addresses #223)

* Cleanup

* Preload requests

* Fix IDE warnings

* Fix tests

* Remove unneeded test class

* Add Javadocs

* Add the GraalVM native-maven-plugin for ahead-of-time compilation

* Switch to Logback implementation of SLF4J, as Log4j2 is not supported with GraalVM

* Update native-maven-plugin version

* Native-image builder optimizations

* Remove pre-made graalvm config

* Update native profile

* Catch exceptions inside TriplestoreStorage

* Reset workerId after warmup

* Update native image plugin configuration

* Add scripts for working with native images

* Remove spring

* Rename directory

* Add test workflow

* Fix permissions

* Remove periods

* Fix script

* Fix workflow

* Update workflow

* Test directory upload

* Update workflows

* Update Test Workflow

* Fix workflow

* Another fix

* Rename job

* Remove test workflow

* Make workerID go out of scope

* Add comment for registering LanguageProcessors

* Clean up logging config

* Fix deploy workflow

* Disable non supported tests

* Update pom.xml to automatically generate configuration files for native image

* Update workflows

* Update documentation

* Fix symlink

* Add cpu micro architectures

* Add cpu micro architectures 2

* Update generate-config.sh

* Fix unstable tests

* Fix regex cleanup

* Enable long running tests on environment variable

* Increase the thread count for the apache http client

* Disable re-usage of bbaos and create bbaos of optimal size when possible

* Try to fix something

* Debug logging

* Debug logging 2

* Attempt to fix something

* Attempt to fix something 2

* Attempt to fix something 3

* Attempt to fix something 4

* Attempt to fix something 5

* Attempt to fix something 6

* Make thread dump

* Make thread dump 2

* Attempt to fix something 7

* Attempt to fix something 8

* Attempt to fix something 9

* Attempt to fix something 10

* Attempt to fix something 11

* Finetuning test

* Finetuning test 2

* Cleanup httpclient configuration

* Cleanup tests

* Disable compressed references by default

This option needs be set before compilation and it allows the heap to use more than 32gb.

* Remove test configurations

* Re-enable configurations and decrease timeout in tests

* Add workaround for failing tests

* Adjust test configurations

* Adjust test configurations 2

* Adjust test configurations 3

* Adjust test configurations 4

* Revert "Adjust test configurations 4"

This reverts commit 9bf8cc8.

* Shorten http client configuration

* Add ByteArrayList output and inputstream

* Update SPARQLProtocolWorker to use ByteArrayListOutputStream when response body has unknown length

* Fix bad merge conflict resolve

* Fix size calculation in ByteArrayListOutputStream

* Add test + fix for ByteArrayListInputStream

* Add test for ByteArrayListOutputStream

* Change single log message

* Update exception handling in TriplestoreStorage

* Add execution parameter to configuration generation

* Fix dry-run parameter

* Add comment in TriplestoreStorage

* Change behavior of ByteArrayListInputStream

* Add comments and access modifiers

* Update src/main/java/org/aksw/iguana/cc/storage/impl/TriplestoreStorage.java

Co-authored-by: Alexander Bigerl <[email protected]>

* Update github workflow

---------

Co-authored-by: Alexander Bigerl <[email protected]>

.github/workflows/lint.yml → .github/workflows/check_version.yml

@@ -1,4 +1,5 @@
-name: lint
+# Checks if version number has been updated 
+name: Version Check
 on: pull_request
 
 jobs:

.github/workflows/deploy.yml

@@ -0,0 +1,156 @@
+name: Deployment
+
+on:
+  push:
+    branches:
+      - main
+
+jobs:
+  find_version:
+    name: Find Release Version
+    runs-on: ubuntu-latest
+    outputs:
+      RELEASE_VERSION: ${{ steps.step_find.outputs.RELEASE_VERSION }}
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-java@v3
+        with:
+          java-version: '17'
+          distribution: 'adopt'
+          cache: 'maven'
+      - name: 'Find velease version'
+        run: echo "RELEASE_VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout)" >> $GITHUB_OUTPUT
+        id: step_find
+
+  deploy_to_maven:
+    name: Deploy to Maven Repository
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up JDK 17
+        uses: actions/setup-java@v3
+        with:
+          java-version: '17'
+          distribution: 'adopt'
+          cache: 'maven'
+      - name: Publish package
+        run: mvn --batch-mode deploy -Dmaven.test.skip=true
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      - name: 'Upload artifact'
+        uses: actions/upload-artifact@v4
+        with:
+          if-no-files-found: error
+          name: 'iguana-jar'
+          path: 'target/'
+
+
+  compile_native:
+    name: Compile Native Executable
+    runs-on: ubuntu-latest
+    needs: find_version
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up GraalVM
+        uses: graalvm/setup-graalvm@v1
+        with:
+          java-version: '21'
+          cache: 'maven'
+      - name: 'Compile native-binary'
+        run: 'mvn -Dagent=true -Pnative package'
+      - name: 'Upload artifact'
+        uses: actions/upload-artifact@v4
+        with:
+          name: 'iguana-native'
+          path: 'target/iguana'
+          if-no-files-found: error
+
+  deploy_docs:
+    name: Deploy Documentation
+    runs-on: ubuntu-latest
+    needs: find_version
+    env:
+      RELEASE_VERSION: ${{ needs.find_version.outputs.RELEASE_VERSION }}
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up JDK 17
+        uses: actions/setup-java@v3
+        with:
+          java-version: '17'
+          distribution: 'adopt'
+          cache: 'maven'
+      - name: Set up Python
+        uses: actions/setup-python@v2
+        with:
+          python-version: 3.x
+          cache: 'pip'
+      - run: pip install mkdocs-material
+      - run: pip install mkdocs-macros-plugin
+      - run: sed -i "s/\$VERSION/${{ env.RELEASE_VERSION }}/g" mkdocs.yml
+      - run: sed -i "s/\$RELEASE_VERSION/${{ env.RELEASE_VERSION }}/g" mkdocs.yml
+      - run: mkdocs build -d site/${{ env.RELEASE_VERSION }}
+      - run: mvn javadoc:javadoc
+      - run: sed -i "s/\$VERSION/${{ env.RELEASE_VERSION }}/g" .github/pages/latest.html
+      - run: sed -i "s/\$VERSION/${{ env.RELEASE_VERSION }}/g" .github/pages/javadoc-latest.html
+      - name: Deploy Site
+        uses: peaceiris/actions-gh-pages@v3
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          publish_dir: ./site/${{ env.RELEASE_VERSION }}
+          destination_dir: ./docs/${{ env.RELEASE_VERSION }}
+      - name: Deploy Javadoc
+        uses: peaceiris/actions-gh-pages@v3
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          publish_dir: ./javadoc/${{ env.RELEASE_VERSION }}
+          destination_dir: ./javadoc/${{ env.RELEASE_VERSION }}
+      - name: Deploy latest.html
+        uses: peaceiris/actions-gh-pages@v3
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          publish_dir: .github/pages/
+          keep_files: true
+          destination_dir: ./docs/
+      - name: Deploy latest.html
+        uses: peaceiris/actions-gh-pages@v3
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          publish_dir: .github/pages/
+          keep_files: true
+          destination_dir: ./docs/
+
+  deploy_gh_release:
+    runs-on: ubuntu-latest
+    needs: [compile-jar, deploy_to_maven, find_version]
+    env:
+      RELEASE_VERSION: ${{ needs.find_version.outputs.RELEASE_VERSION }}
+
+    steps:
+      - name: Download artifacts from previous jobs
+        uses: actions/download-artifact@v4
+        with:
+          path: artifacts/
+          merge-multiple: true
+      - name: Prepare files
+        run: |
+          mkdir iguana
+          cp artifacts/start-iguana.sh iguana/
+          cp artifacts/iguana.jar iguana/iguana.jar
+          cp artifacts/iguana iguana/iguana
+          cp example-suite.yml iguana/
+          zip -r iguana-${{ env.RELEASE_VERSION }}.zip iguana/
+      - name: Create Release
+        uses: softprops/action-gh-release@v2
+        with:
+          tag_name: v${{ env.RELEASE_VERSION }}
+          name: version ${{ env.RELEASE_VERSION }}
+          draft: false
+          prerelease: false
+          body: ""
+          fail_on_unmatched_files: true
+          make_latest: true
+          token: ${{ secrets.GITHUB_TOKEN }}
+          files: |
+            iguana-${{ env.RELEASE_VERSION }}.zip
+            artifacts/iguana.jar
+            artifacts/iguana

.github/workflows/tests.yml

@@ -0,0 +1,45 @@
+name: Tests
+
+on:
+  push:
+    branches:
+      - develop
+  pull_request:
+    branches:
+      - develop
+      - main
+
+jobs:
+  tests:
+    name: Compile and Run Tests
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - name: Set up JDK 17
+        uses: actions/setup-java@v3
+        with:
+          java-version: '17'
+          distribution: 'adopt'
+      - name: Cache Maven packages
+        uses: actions/cache@v2
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
+          restore-keys: ${{ runner.os }}-m2
+      - name: Testing the Java code
+        run:  mvn package
+
+  # Only run for pull request on main or if pushed to develop
+  compile_native:
+    if: github.base_ref == 'main' || github.event_name == 'push'
+    name: Test Native Executable Compilation
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up GraalVM
+        uses: graalvm/setup-graalvm@v1
+        with:
+          java-version: '21'
+          cache: 'maven'
+      - name: 'Compile native-binary and run tests'
+        run: 'mvn -Pnative -Dagent=true package'