Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adapt and build intermediate images #22

Merged
merged 27 commits into from
Nov 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
88f4490
wip: first version of PR test workflow
CodeLionX Nov 15, 2023
6653d5f
feat: adapt first algorithm to new infrastructure
CodeLionX Nov 15, 2023
221fd40
wip
CodeLionX Nov 15, 2023
97cd8cf
fix: job generation script
CodeLionX Nov 15, 2023
51ed06b
feat: adapt second algorithm to new infrastructure
CodeLionX Nov 15, 2023
7a6358d
fix: python version
CodeLionX Nov 15, 2023
74b50e4
wip
CodeLionX Nov 15, 2023
cdd0660
wip
CodeLionX Nov 15, 2023
45cce51
wip
CodeLionX Nov 15, 2023
9ae03b8
feat: allow matrix generation script to run in different contexts (fo…
CodeLionX Nov 16, 2023
6387ae6
refactor: split up base images and intermediate images; rename folders
CodeLionX Nov 16, 2023
cd5424b
feat: adjust workflow to build images in order
CodeLionX Nov 16, 2023
1c13e8b
fix: build matrix computation script
CodeLionX Nov 16, 2023
3e9af8f
wip
CodeLionX Nov 16, 2023
b8e4d13
wip
CodeLionX Nov 16, 2023
e4b727c
test empty matrix
CodeLionX Nov 16, 2023
c840567
test empty matrix
CodeLionX Nov 16, 2023
182c762
wip
CodeLionX Nov 16, 2023
672dc1f
wip
CodeLionX Nov 16, 2023
77107e4
feat: prepare image publishing and adapt docker images
CodeLionX Nov 17, 2023
9c9089c
wip
CodeLionX Nov 17, 2023
d0b584a
chore: restore lof and sublof algorithms and use kmeans; also add lic…
CodeLionX Nov 17, 2023
e0199f6
wip
CodeLionX Nov 17, 2023
299fa53
feat: adjust r base image
CodeLionX Nov 20, 2023
7903e27
chore: cleanup workflow definition
CodeLionX Nov 20, 2023
7b395ab
Merge branch 'feat/build-algos-ci' into feat/build-imanges-ci-3
CodeLionX Nov 20, 2023
cf5418b
feat: adapt intermediate images
CodeLionX Nov 20, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions 1-intermediate-images/pyod/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
FROM ghcr.io/timeeval/python3-base:0.3.0

LABEL maintainer="[email protected]"

# install pyod library and cleanup afterwards
RUN set -eux; \
pip install --no-cache-dir pyod==0.9.2; \
find /usr/local -depth \
\( \
\( -type d -a \( -name test -o -name tests -o -name idle_test \) \) \
-o \
\( -type f -a \( -name '*.pyc' -o -name '*.pyo' \) \) \
\) -exec rm -rf '{}' +; \
rm -rf /tmp/* /var/tmp/* ~/.cache/pip
21 changes: 21 additions & 0 deletions 1-intermediate-images/pyod/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2020-2023 Phillip Wenig and Sebastian Schmidl

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
1 change: 1 addition & 0 deletions 1-intermediate-images/pyod/version.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
0.3.0
5 changes: 5 additions & 0 deletions 1-intermediate-images/python3-torch/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
FROM ghcr.io/timeeval/python3-base:0.3.0

LABEL maintainer="[email protected]"

RUN pip install --no-cache-dir torch==1.7.1
21 changes: 21 additions & 0 deletions 1-intermediate-images/python3-torch/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2020-2023 Phillip Wenig and Sebastian Schmidl

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
1 change: 1 addition & 0 deletions 1-intermediate-images/python3-torch/version.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
0.3.0
8 changes: 8 additions & 0 deletions 1-intermediate-images/timeeval-test-algorithm/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
FROM ghcr.io/timeeval/python3-base:0.3.0

LABEL maintainer="[email protected]"

ENV ALGORITHM_MAIN=/app/algorithm.py

COPY manifest.json /app/
COPY algorithm.py /app/
21 changes: 21 additions & 0 deletions 1-intermediate-images/timeeval-test-algorithm/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2020-2023 Phillip Wenig and Sebastian Schmidl

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
41 changes: 41 additions & 0 deletions 1-intermediate-images/timeeval-test-algorithm/algorithm.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
import numpy as np
import pandas as pd
import json
import time
import argparse
import sys


class AlgorithmArgs(argparse.Namespace):
@property
def ts(self) -> np.ndarray:
dataset = pd.read_csv(self.dataInput)
return dataset.values[:, 1:-1]

@staticmethod
def from_sys_args() -> 'AlgorithmArgs':
args = json.loads(sys.argv[1])
return AlgorithmArgs(**args)


def main():
args = AlgorithmArgs.from_sys_args()
will_raise = args.customParameters.get("raise", False)
sleep_seconds = args.customParameters.get("sleep", 10)
write_preliminary_results = args.customParameters.get("write_prelim_results", False)

results = np.zeros(args.ts.shape[0])

if write_preliminary_results:
results.tofile(args.dataOutput, sep="\n")

if will_raise:
raise Exception("from within")

time.sleep(sleep_seconds)

results.tofile(args.dataOutput, sep="\n")


if __name__ == "__main__":
main()
37 changes: 37 additions & 0 deletions 1-intermediate-images/timeeval-test-algorithm/manifest.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
{
"title": "timeeval-test-algorithm",
"description": "Test algorithm image for TimeEval.",
"inputDimensionality": "multivariate",
"version": "0.3.0",
"authors": "Sebastian Schmidl, Phillip Wenig",
"language": "Python",
"type": "Detector",
"mainFile": "algorithm.py",
"learningType": "unsupervised",
"executionStep": {
"parameters": [
{
"name": "raise",
"type": "bool",
"defaultValue": false,
"optional": "true",
"description": "Force the algorithm to raise an exception and return a non-zero exit-code."
},
{
"name": "sleep",
"type": "int",
"defaultValue": 10,
"optional": "true",
"description": "Controls the duration of the algorithm 'processing' via sleeping for a certain amount of time before writing the results."
},
{
"name": "write_prelim_results",
"type": "bool",
"defaultValue": false,
"optional": "true",
"description": "If the algorithm should write some results before raising an exception or sleeping."
}
],
"modelInput": "none"
}
}
12 changes: 12 additions & 0 deletions 1-intermediate-images/tsmp/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
FROM ghcr.io/timeeval/r-base:0.3.0

LABEL maintainer="[email protected]"

# install tsmp library and cleanup afterwards
RUN set -eux; \
R -e 'install.packages("tsmp")'; \
find /usr/lib/R/library -depth \
\( \
-type d -a \( -name doc -o -name html \) \
\) -exec rm -rf '{}' +; \
rm -rf /tmp/* /var/tmp/*
21 changes: 21 additions & 0 deletions 1-intermediate-images/tsmp/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2020-2023 Phillip Wenig and Sebastian Schmidl

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
1 change: 1 addition & 0 deletions 1-intermediate-images/tsmp/version.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
0.3.0