Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update webknossos api version to v8 #1185

Merged
merged 10 commits into from
Dec 18, 2024
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Merge branch 'master' into wk-api-v8
markbader committed Dec 18, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit 7fa231a7ec56aa17ee124f020b80ac893c4b4aa9
7 changes: 7 additions & 0 deletions .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
@@ -25,6 +25,9 @@ jobs:
uses: astral-sh/setup-uv@v3
with:
version: "0.4.19"

- name: Install proxay
run: npm install -g proxay

- name: Set up Python ${{ matrix.python-version }}
run: uv python install ${{ matrix.python-version }}
@@ -34,6 +37,10 @@ jobs:
git diff --no-ext-diff --quiet --exit-code
[[ -z $(git status -s) ]]

- name: Use currently deployed Webknossos version
run: |
sed -i "s/export DOCKER_TAG=.*$/export DOCKER_TAG=master__$(curl https://webknossos.org/api/buildinfo | jq -r .webknossos.version)/" local_wk_setup.sh

- name: Python tests, refreshing the network snapshots
env:
WK_TOKEN: ${{ secrets.WK_TOKEN }}
1 change: 0 additions & 1 deletion docs/mkdocs.yml
Original file line number Diff line number Diff line change
@@ -234,7 +234,6 @@ nav:
- webknossos-py/examples/download_image_data.md
- webknossos-py/examples/download_tiff_stack.md
- webknossos-py/examples/remote_datasets.md
- webknossos-py/examples/zarr_and_dask.md
- webknossos-py/examples/convert_4d_tiff.md
- webknossos-py/examples/announce_dataset_upload.md
- webknossos-py/examples/accessing_metadata.md
9 changes: 0 additions & 9 deletions docs/src/webknossos-py/examples/zarr_and_dask.md

This file was deleted.

255 changes: 71 additions & 184 deletions docs/uv.lock

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions webknossos/Changelog.md
Original file line number Diff line number Diff line change
@@ -13,11 +13,13 @@ For upgrade instructions, please check the respective _Breaking Changes_ section
[Commits](https://github.com/scalableminds/webknossos-libs/compare/v0.16.1...HEAD)

### Breaking Changes
- `MagView.get_zarr_array` now returns a `tensorstore` array instead of a `zarr-python` array. [#1174](https://github.com/scalableminds/webknossos-libs/pull/1174)

### Added

### Changed
- Updated to WEBKNOSSOS API version 8. [#1185](https://github.com/scalableminds/webknossos-libs/pull/1185)
- Using tensorstore for reading and writing zarr 2 and 3 arrays. Removed `zarrita` and `zarr` dependency. [#1174](https://github.com/scalableminds/webknossos-libs/pull/1174)

### Fixed

26 changes: 12 additions & 14 deletions webknossos/examples/apply_merger_mode.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
from os import environ
from pathlib import Path
from typing import Tuple, cast

@@ -84,19 +83,18 @@ def apply_mapping_for_chunk(args: Tuple[wk.View, wk.View, int]) -> None:
# Optionally, downsample and re-upload #
########################################

if "PYTEST_CURRENT_TEST" not in environ:
out_layer.downsample()
dataset.delete_layer("segmentation")
dataset.upload(
"l4_sample_remapped",
layers_to_link=[
wk.LayerToLink(
dataset_name="l4_sample",
layer_name="color",
organization_id="scalable_minds",
)
],
)
out_layer.downsample()
dataset.delete_layer("segmentation")
dataset.upload(
"l4_sample_remapped",
layers_to_link=[
wk.LayerToLink(
dataset_name="l4_sample",
layer_name="color",
organization_id="scalable_minds",
)
],
)


if __name__ == "__main__":
27 changes: 0 additions & 27 deletions webknossos/examples/zarr_and_dask.py

This file was deleted.

2 changes: 1 addition & 1 deletion webknossos/local_wk_setup.sh
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
function export_vars {
export WK_TOKEN=1b88db86331a38c21a0b235794b9e459856490d70408bcffb767f64ade0f83d2bdb4c4e181b9a9a30cdece7cb7c65208cc43b6c1bb5987f5ece00d348b1a905502a266f8fc64f0371cd6559393d72e031d0c2d0cabad58cccf957bb258bc86f05b5dc3d4fff3d5e3d9c0389a6027d861a21e78e3222fb6c5b7944520ef21761e
export WK_URL=http://localhost:9000
export DOCKER_TAG=master__30776
export DOCKER_TAG=master__31116
}

function ensure_local_test_wk {
12 changes: 6 additions & 6 deletions webknossos/pyproject.toml
Original file line number Diff line number Diff line change
@@ -38,25 +38,22 @@ dependencies = [
"cattrs >=22.0.0",
"certifi>=2023",
"cluster-tools",
"fsspec ~=2024.6.0",
"httpx ~=0.27.0",
"loxun ~=2.0.0",
"natsort ~=8.4.0",
"networkx ~=3.2.1",
"numcodecs ~=0.12",
"numpy ~=2.0.0",
"pims ~=0.7.0",
"psutil ~=6.0.0",
"python-dateutil ~=2.8.0",
"python-dotenv ~=1.0.1",
"rich ~=13.8.0",
"scipy ~=1.13",
"tensorstore~=0.1.69",
"typer ~=0.12.5",
"typing-extensions ~=4.0",
"universal-pathlib ~=0.2",
"wkw ==1.1.24",
"zarr ~=2.18.0",
"zarrita ==0.2.7",
"zipp ~=3.5.0",
]

@@ -91,7 +88,7 @@ all = [
[tool.uv]
extra-index-url = ["https://pypi.scm.io/simple"]
dev-dependencies = [
"dask[distributed] ~=2023.9.1; python_version >='3.9'",
"fsspec ~=2024.6.0",
"hypothesis ~=6.35.0",
"icecream ~=2.1.1",
"inducoapi ~=2.0.2",
@@ -160,7 +157,7 @@ target-version = "py39"
# PERF = Perflint https://docs.astral.sh/ruff/rules/#perflint-perf
# FIX = Flake8 fixme https://docs.astral.sh/ruff/rules/#flake8-fixme-fix
# ARG = Flake8 unused-arguments https://docs.astral.sh/ruff/rules/#flake8-unused-arguments-arg
select = ["F", "E", "W", "I", "A", "PERF", "FIX", "ARG"]
select = ["F", "E", "W", "I", "A", "PERF", "FIX", "ARG", "T201"]

# Allow fix for all enabled rules (when `--fix`) is provided.
fixable = ["ALL"]
@@ -169,6 +166,9 @@ fixable = ["ALL"]
# E712 true-false-comparison https://docs.astral.sh/ruff/rules/true-false-comparison/
ignore = ["E501", "E712"]

[tool.ruff.lint.per-file-ignores]
"{examples,tests,script_collection,webknossos/cli}/*" = ["T201"]

[tool.ruff.format]
# Like Black, use double quotes for strings.
quote-style = "double"
15 changes: 15 additions & 0 deletions webknossos/test.sh
Original file line number Diff line number Diff line change
@@ -31,6 +31,21 @@ if [ $# -gt 0 ] && [ "$1" = "--refresh-snapshots" ]; then

stop_local_test_wk

exit $PYTEST_EXIT_CODE
elif [ $# -gt 0 ] && [ "$1" = "--add-snapshots" ]; then
ensure_local_test_wk

# Starts a proxy server in record mode on port 3000 and sets the HTTP_PROXY env var
proxay --mode record --host http://localhost:9000 --tapes-dir tests/cassettes &
PROXAY_PID=$!

shift
$PYTEST "-m" "use_proxay" "$@"
PYTEST_EXIT_CODE=$?
trap 'kill $PROXAY_PID' EXIT

stop_local_test_wk

exit $PYTEST_EXIT_CODE
elif [ $# -gt 0 ] && [ "$1" = "--debug-cassettes" ]; then
# This will start a proxay server in replay mode so that the stored cassettes can be used for debugging tests.
Binary file not shown.
6 changes: 3 additions & 3 deletions webknossos/testdata/simple_zarr3_dataset/color/1/zarr.json
Original file line number Diff line number Diff line change
@@ -16,17 +16,17 @@
"configuration": {
"chunk_shape": [3, 16, 16, 16],
"codecs": [
{ "name": "endian", "configuration": { "endian": "little" } }
{ "name": "bytes", "configuration": { "endian": "little" } }
],
"index_codecs": [
{ "name": "endian", "configuration": { "endian": "little" } },
{ "name": "bytes", "configuration": { "endian": "little" } },
{ "name": "crc32c" }
]
},
"name": "sharding_indexed"
}
],
"dimension_names": null,
"dimension_names": ["c", "x", "y", "z"],
"zarr_format": 3,
"node_type": "array"
}
Loading
You are viewing a condensed version of this merge commit. You can view the full changes here.