Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for ahead-of-time compilation #248

Merged
merged 111 commits into from
Jul 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
111 commits
Select commit Hold shift + click to select a range
734b261
Add more logging messages
nck-mlcnv Dec 5, 2023
8a62c90
Fix log4j2 configuration
nck-mlcnv Dec 5, 2023
07b3052
Implement apache HTTP client
nck-mlcnv Jan 12, 2024
78408b9
Implement apache HTTP async client 5
nck-mlcnv Jan 12, 2024
476f528
Fix timeout
nck-mlcnv Jan 12, 2024
9587d60
Fixes
nck-mlcnv Feb 6, 2024
1831b60
Fix hashing bug
nck-mlcnv Feb 7, 2024
a9ca2a5
Fix conversion of byte stream to string
nck-mlcnv Feb 8, 2024
48be6c1
Implement POST request streaming
nck-mlcnv Feb 8, 2024
744b09e
Disable the storing and hashing of responses when the parseResults pa…
nck-mlcnv Feb 13, 2024
d454f6b
Move utility classes
nck-mlcnv Feb 14, 2024
d517e9c
StreamEntityProducer can send fixed-sized data and is reproducible now
nck-mlcnv Feb 16, 2024
c238a40
Make QueryHandler return stream supplier and info about query being c…
nck-mlcnv Feb 16, 2024
d9190cb
Change RequestFactory behavior
nck-mlcnv Feb 16, 2024
c25b560
Cleanup
nck-mlcnv Feb 16, 2024
94bcb3c
Preload requests
nck-mlcnv Feb 16, 2024
52ce0bb
Fix IDE warnings
nck-mlcnv Feb 16, 2024
e2f8199
Fix tests
nck-mlcnv Feb 16, 2024
fc523f0
Remove unneeded test class
nck-mlcnv Feb 16, 2024
bac8488
Add Javadocs
nck-mlcnv Feb 16, 2024
5480342
Add the GraalVM native-maven-plugin for ahead-of-time compilation
nck-mlcnv Feb 14, 2024
4823a8b
Switch to Logback implementation of SLF4J, as Log4j2 is not supported…
nck-mlcnv Feb 14, 2024
986423b
Update native-maven-plugin version
nck-mlcnv Feb 20, 2024
10abf6c
Native-image builder optimizations
nck-mlcnv Feb 20, 2024
57a5217
Merge branch 'develop' into feature/ahead-of-time-compiler
nck-mlcnv Mar 21, 2024
75810ce
Remove pre-made graalvm config
nck-mlcnv Mar 26, 2024
9cc4d7c
Update native profile
nck-mlcnv Mar 26, 2024
f970759
Catch exceptions inside TriplestoreStorage
nck-mlcnv Mar 27, 2024
4611e11
Reset workerId after warmup
nck-mlcnv Mar 27, 2024
eddb49f
Update native image plugin configuration
nck-mlcnv Mar 27, 2024
69538a9
Add scripts for working with native images
nck-mlcnv Mar 27, 2024
5654ee5
Remove spring
nck-mlcnv Apr 24, 2024
f10c504
Rename directory
nck-mlcnv May 22, 2024
3860a3b
Add test workflow
nck-mlcnv May 22, 2024
ad9d99b
Fix permissions
nck-mlcnv May 22, 2024
c3f4606
Remove periods
nck-mlcnv May 22, 2024
180e725
Fix script
nck-mlcnv May 22, 2024
1e4e328
Fix workflow
nck-mlcnv May 22, 2024
caa8805
Update workflow
nck-mlcnv May 22, 2024
a25ec5e
Test directory upload
nck-mlcnv May 22, 2024
fc97825
Update workflows
nck-mlcnv May 22, 2024
d5cc274
Update Test Workflow
nck-mlcnv May 22, 2024
2c88c0e
Fix workflow
nck-mlcnv May 22, 2024
a80beb6
Another fix
nck-mlcnv May 22, 2024
677ccdd
Rename job
nck-mlcnv May 22, 2024
069c7ae
Remove test workflow
nck-mlcnv May 23, 2024
749dfa7
Make workerID go out of scope
nck-mlcnv Jun 11, 2024
151af88
Add comment for registering LanguageProcessors
nck-mlcnv Jun 11, 2024
9b876b1
Clean up logging config
nck-mlcnv Jun 11, 2024
7d84ac0
Fix deploy workflow
nck-mlcnv Jun 11, 2024
286529e
Merge branch 'develop' into feature/ahead-of-time-compiler
nck-mlcnv Jun 11, 2024
8c721d1
Disable non supported tests
nck-mlcnv Jun 13, 2024
8d0adfd
Update pom.xml to automatically generate configuration files for nati…
nck-mlcnv Jun 13, 2024
5d38131
Update workflows
nck-mlcnv Jun 13, 2024
f3a641c
Update documentation
nck-mlcnv Jun 13, 2024
c99582a
Fix symlink
nck-mlcnv Jun 13, 2024
bcd0f9c
Add cpu micro architectures
nck-mlcnv Jun 13, 2024
8628f86
Add cpu micro architectures 2
nck-mlcnv Jun 13, 2024
fdcb710
Update generate-config.sh
nck-mlcnv Jun 17, 2024
e80516e
Fix unstable tests
nck-mlcnv Jun 28, 2024
70f845f
Fix regex cleanup
nck-mlcnv Jun 28, 2024
9fbf565
Enable long running tests on environment variable
nck-mlcnv Jun 28, 2024
70d0a2b
Increase the thread count for the apache http client
nck-mlcnv Jul 4, 2024
3b5c344
Disable re-usage of bbaos and create bbaos of optimal size when possible
nck-mlcnv Jul 4, 2024
871a874
Try to fix something
nck-mlcnv Jul 4, 2024
5daf544
Debug logging
nck-mlcnv Jul 4, 2024
89cb31b
Debug logging 2
nck-mlcnv Jul 4, 2024
a03ac9f
Attempt to fix something
nck-mlcnv Jul 4, 2024
1975ac4
Attempt to fix something 2
nck-mlcnv Jul 4, 2024
f910cec
Attempt to fix something 3
nck-mlcnv Jul 4, 2024
858fc80
Attempt to fix something 4
nck-mlcnv Jul 5, 2024
f35f9aa
Attempt to fix something 5
nck-mlcnv Jul 5, 2024
eeaf789
Attempt to fix something 6
nck-mlcnv Jul 5, 2024
e23b02a
Make thread dump
nck-mlcnv Jul 5, 2024
bd4669f
Make thread dump 2
nck-mlcnv Jul 5, 2024
26867ad
Attempt to fix something 7
nck-mlcnv Jul 5, 2024
1c062ba
Attempt to fix something 8
nck-mlcnv Jul 5, 2024
7ff7acd
Attempt to fix something 9
nck-mlcnv Jul 5, 2024
b613a47
Attempt to fix something 10
nck-mlcnv Jul 5, 2024
ac82a6d
Attempt to fix something 11
nck-mlcnv Jul 5, 2024
2650bfc
Finetuning test
nck-mlcnv Jul 5, 2024
64d7445
Finetuning test 2
nck-mlcnv Jul 5, 2024
ec4749c
Cleanup httpclient configuration
nck-mlcnv Jul 5, 2024
632cce9
Cleanup tests
nck-mlcnv Jul 5, 2024
3bb2e7e
Disable compressed references by default
nck-mlcnv Jul 16, 2024
04b85fe
Remove test configurations
nck-mlcnv Jul 17, 2024
04158d0
Re-enable configurations and decrease timeout in tests
nck-mlcnv Jul 17, 2024
a9f3115
Add workaround for failing tests
nck-mlcnv Jul 17, 2024
6bedf3c
Adjust test configurations
nck-mlcnv Jul 18, 2024
305bc59
Adjust test configurations 2
nck-mlcnv Jul 18, 2024
5488327
Adjust test configurations 3
nck-mlcnv Jul 18, 2024
9bf8cc8
Adjust test configurations 4
nck-mlcnv Jul 18, 2024
25a7ca6
Revert "Adjust test configurations 4"
nck-mlcnv Jul 18, 2024
69b0c0b
Shorten http client configuration
nck-mlcnv Jul 19, 2024
b6928e5
Add ByteArrayList output and inputstream
nck-mlcnv Jul 19, 2024
feb8a18
Update SPARQLProtocolWorker to use ByteArrayListOutputStream when res…
nck-mlcnv Jul 19, 2024
24c5b55
Merge branch 'develop' into feature/ahead-of-time-compiler
nck-mlcnv Jul 19, 2024
2f12e49
Fix bad merge conflict resolve
nck-mlcnv Jul 19, 2024
deea528
Fix size calculation in ByteArrayListOutputStream
nck-mlcnv Jul 19, 2024
09e73a8
Add test + fix for ByteArrayListInputStream
nck-mlcnv Jul 22, 2024
19d0282
Add test for ByteArrayListOutputStream
nck-mlcnv Jul 24, 2024
97cfa4c
Change single log message
nck-mlcnv Jul 24, 2024
bedf55a
Update exception handling in TriplestoreStorage
nck-mlcnv Jul 24, 2024
2214fd8
Add execution parameter to configuration generation
nck-mlcnv Jul 24, 2024
ff6d771
Fix dry-run parameter
nck-mlcnv Jul 24, 2024
edf2a26
Add comment in TriplestoreStorage
nck-mlcnv Jul 24, 2024
d842fa1
Change behavior of ByteArrayListInputStream
nck-mlcnv Jul 24, 2024
38344ab
Add comments and access modifiers
nck-mlcnv Jul 24, 2024
1d28690
Update src/main/java/org/aksw/iguana/cc/storage/impl/TriplestoreStora…
nck-mlcnv Jul 25, 2024
cd25e5c
Update github workflow
nck-mlcnv Jul 25, 2024
5ee69a1
Merge remote-tracking branch 'origin/feature/ahead-of-time-compiler' …
nck-mlcnv Jul 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
name: lint
# Checks if version number has been updated
name: Version Check
on: pull_request

jobs:
Expand Down
99 changes: 0 additions & 99 deletions .github/workflows/ci.yml

This file was deleted.

156 changes: 156 additions & 0 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
name: Deployment

on:
push:
branches:
- main

jobs:
find_version:
name: Find Release Version
runs-on: ubuntu-latest
outputs:
RELEASE_VERSION: ${{ steps.step_find.outputs.RELEASE_VERSION }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v3
with:
java-version: '17'
distribution: 'adopt'
cache: 'maven'
- name: 'Find velease version'
run: echo "RELEASE_VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout)" >> $GITHUB_OUTPUT
id: step_find

deploy_to_maven:
name: Deploy to Maven Repository
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up JDK 17
uses: actions/setup-java@v3
with:
java-version: '17'
distribution: 'adopt'
cache: 'maven'
- name: Publish package
nck-mlcnv marked this conversation as resolved.
Show resolved Hide resolved
run: mvn --batch-mode deploy -Dmaven.test.skip=true
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: 'Upload artifact'
uses: actions/upload-artifact@v4
with:
if-no-files-found: error
name: 'iguana-jar'
path: 'target/'


compile_native:
name: Compile Native Executable
runs-on: ubuntu-latest
needs: find_version
steps:
- uses: actions/checkout@v4
- name: Set up GraalVM
uses: graalvm/setup-graalvm@v1
with:
java-version: '21'
cache: 'maven'
- name: 'Compile native-binary'
run: 'mvn -Dagent=true -Pnative package'
- name: 'Upload artifact'
uses: actions/upload-artifact@v4
with:
name: 'iguana-native'
path: 'target/iguana'
if-no-files-found: error

deploy_docs:
name: Deploy Documentation
runs-on: ubuntu-latest
needs: find_version
env:
RELEASE_VERSION: ${{ needs.find_version.outputs.RELEASE_VERSION }}
steps:
- uses: actions/checkout@v4
- name: Set up JDK 17
uses: actions/setup-java@v3
with:
java-version: '17'
distribution: 'adopt'
cache: 'maven'
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: 3.x
cache: 'pip'
- run: pip install mkdocs-material
- run: pip install mkdocs-macros-plugin
- run: sed -i "s/\$VERSION/${{ env.RELEASE_VERSION }}/g" mkdocs.yml
- run: sed -i "s/\$RELEASE_VERSION/${{ env.RELEASE_VERSION }}/g" mkdocs.yml
- run: mkdocs build -d site/${{ env.RELEASE_VERSION }}
- run: mvn javadoc:javadoc
- run: sed -i "s/\$VERSION/${{ env.RELEASE_VERSION }}/g" .github/pages/latest.html
- run: sed -i "s/\$VERSION/${{ env.RELEASE_VERSION }}/g" .github/pages/javadoc-latest.html
- name: Deploy Site
uses: peaceiris/actions-gh-pages@v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./site/${{ env.RELEASE_VERSION }}
destination_dir: ./docs/${{ env.RELEASE_VERSION }}
- name: Deploy Javadoc
uses: peaceiris/actions-gh-pages@v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./javadoc/${{ env.RELEASE_VERSION }}
destination_dir: ./javadoc/${{ env.RELEASE_VERSION }}
- name: Deploy latest.html
uses: peaceiris/actions-gh-pages@v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: .github/pages/
keep_files: true
destination_dir: ./docs/
- name: Deploy latest.html
uses: peaceiris/actions-gh-pages@v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: .github/pages/
keep_files: true
destination_dir: ./docs/

deploy_gh_release:
runs-on: ubuntu-latest
needs: [compile-jar, deploy_to_maven, find_version]
env:
RELEASE_VERSION: ${{ needs.find_version.outputs.RELEASE_VERSION }}

steps:
- name: Download artifacts from previous jobs
uses: actions/download-artifact@v4
with:
path: artifacts/
merge-multiple: true
- name: Prepare files
run: |
mkdir iguana
cp artifacts/start-iguana.sh iguana/
cp artifacts/iguana.jar iguana/iguana.jar
cp artifacts/iguana iguana/iguana
cp example-suite.yml iguana/
zip -r iguana-${{ env.RELEASE_VERSION }}.zip iguana/
- name: Create Release
uses: softprops/action-gh-release@v2
with:
tag_name: v${{ env.RELEASE_VERSION }}
name: version ${{ env.RELEASE_VERSION }}
draft: false
prerelease: false
body: ""
fail_on_unmatched_files: true
make_latest: true
token: ${{ secrets.GITHUB_TOKEN }}
files: |
iguana-${{ env.RELEASE_VERSION }}.zip
artifacts/iguana.jar
artifacts/iguana
29 changes: 0 additions & 29 deletions .github/workflows/maven.yml

This file was deleted.

45 changes: 45 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: Tests

on:
push:
branches:
- develop
pull_request:
branches:
- develop
- main

jobs:
tests:
name: Compile and Run Tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up JDK 17
uses: actions/setup-java@v3
with:
java-version: '17'
distribution: 'adopt'
- name: Cache Maven packages
uses: actions/cache@v2
with:
path: ~/.m2
key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-m2
- name: Testing the Java code
run: mvn package

# Only run for pull request on main or if pushed to develop
compile_native:
if: github.base_ref == 'main' || github.event_name == 'push'
name: Test Native Executable Compilation
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up GraalVM
uses: graalvm/setup-graalvm@v1
with:
java-version: '21'
cache: 'maven'
- name: 'Compile native-binary and run tests'
run: 'mvn -Pnative -Dagent=true package'
Loading
Loading