Clustering example

[DanInci]

TODO list for example clustering:

• Create benchmark YAML
• Implement method modules in custom repositories
• Update software backend
• Setup storage
• Update documentation on website

Note for reviewer:

• Please review also the example module repositories for clustering, since I wrote some pretty basic analysis. They are available here.

Topology plot:

---
title: clustering_benchmark
---
flowchart LR
	classDef param fill:#f96
	subgraph data
		iris
		penguins
	end
	subgraph distances
		D1
		iris --> D1
		penguins --> D1
	end
	subgraph methods
		kmeans
		D1 --> kmeans
		ward
		D1 --> ward
	end
	subgraph metrics
		ari
		kmeans --> ari
		ward --> ari
		accuracy
		kmeans --> accuracy
		ward --> accuracy
	end
	subgraph params_D1
		8948341830810125333['--measure', 'cosine']
		-6750798201292827140['--measure', 'euclidean']
		-5289070520883795694['--measure', 'manhattan']
		1540997255918402294['--measure', 'chebyshev']
	end
	params_D1:::param --o D1

Loading

Computational plot:

[retogerber]

Added the possibility to use the remote storage with play.min.io
use with (currently only works with https://github.com/omnibenchmark/omnibenchmark/tree/io_tests):
OB_STORAGE_S3_CONFIG=.play_minio.json ob run benchmark -b Clustering.yaml

[imallona]

Cross-posting omnibenchmark/omnibenchmark#35

[imallona]

The omnibenchmark/omnibenchmark#36 patches the run using envmodules, e.g. when snakemake -s Snakefile --use-envmodules once the Snakefile is generated after running ob run benchmark -b Clustering.yaml --local (see note) with omnibenchmark/omnibenchmark@e6dab70 and Clustering.yaml f636fe5. Currently, the validator checks whether envmodules as files exist or not. There are (lua) files defining the envmodules somewhere, but I think is easier we check via module avail checks (see the drafted PR).

Note: The ob run benchmark using envmodules does not work directly but that's an independent issue, probably due to the modulepath environmental variable not being propagated to children processes by snakemake; opening a PR.

[DanInci]

Added the possibility to use the remote storage with play.min.io use with (currently only works with https://github.com/omnibenchmark/omnibenchmark/tree/io_tests): OB_STORAGE_S3_CONFIG=.play_minio.json ob run benchmark -b Clustering.yaml

@retogerber Arent these sensitive tokens, or just part of a free sandbox environment?
Would it make sense to store them as github secrets and only use remote storage examples with the CI jobs?

[retogerber]

Added the possibility to use the remote storage with play.min.io use with (currently only works with https://github.com/omnibenchmark/omnibenchmark/tree/io_tests): OB_STORAGE_S3_CONFIG=.play_minio.json ob run benchmark -b Clustering.yaml

@retogerber Arent these sensitive tokens, or just part of a free sandbox environment? Would it make sense to store them as github secrets and only use remote storage examples with the CI jobs?

@DanInci those are not sensitive but from the official minio sandbox. E.g. here: https://min.io/docs/minio/linux/developers/python/minio-py.html

[DanInci]

Added the possibility to use the remote storage with play.min.io use with (currently only works with https://github.com/omnibenchmark/omnibenchmark/tree/io_tests): OB_STORAGE_S3_CONFIG=.play_minio.json ob run benchmark -b Clustering.yaml

@retogerber Arent these sensitive tokens, or just part of a free sandbox environment? Would it make sense to store them as github secrets and only use remote storage examples with the CI jobs?

@DanInci those are not sensitive but from the official minio sandbox. E.g. here: https://min.io/docs/minio/linux/developers/python/minio-py.html

Alright great, just double checking.