openeo-processes-dask 2024.1.1 gives AttributeError: module 'openeo_processes_dask.specs' has no attribute 'apply_kernel'

[soxofaan]

Just played with dask runner, and one of the current struggles is

    @pytest.fixture(scope="module")
    def connection(
        request, runner: str, auto_authenticate: bool, pytestconfig
    ) -> ProcessTestRunner:
        # TODO: this fixture override changes the return type of the original `connection` fixture,
        #       which might lead to problems due to broken assumptions
    
        if runner == "dask":
>           from openeo_test_suite.lib.process_runner.dask import Dask

src/openeo_test_suite/tests/processes/processing/conftest.py:40: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
src/openeo_test_suite/lib/process_runner/dask.py:48: in <module>
    registry = create_process_registry()
src/openeo_test_suite/lib/process_runner/dask.py:35: in create_process_registry
    specs = {
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

.0 = <list_iterator object at 0x7fce38927ca0>

    specs = {
>       func.__name__: getattr(specs_module, func.__name__)
        for func in processes_from_module
    }
E   AttributeError: module 'openeo_processes_dask.specs' has no attribute 'apply_kernel'

src/openeo_test_suite/lib/process_runner/dask.py:36: AttributeError

apparently downgrading openeo-processes-dask to 23.11.6 instead of 2024.1.1 (currenlty highest) is workaround

[soxofaan]

cc @m-mohr

[soxofaan]

as quickfix I prevented installing 2024.1.1 in b97d890

Better solution might be to skip this apply_kernel in some way. Or maybe something has to be fixed in openeo-processes-dask. Haven't looked much into the details yet

[m-mohr]

Looks like a bug introduced through Open-EO/openeo-processes-dask#186 @GeraldIr

[GeraldIr]

This issue should have been fixed by the newest openeo-processes-dask version: https://github.com/Open-EO/openeo-processes-dask/releases/tag/2024.1.2

[soxofaan]

with 2024.1.2 the "apply_kernel" issue is indeed gone, but I now get the problem with "mask":

e.g.

___________________________________________________________________________ ERROR at setup of test_process[min-example7-file7-L1-False] ___________________________________________________________________________

request = <SubRequest 'connection' for <Function test_process[min-example0-file0-L1-False]>>, runner = 'dask', auto_authenticate = True, pytestconfig = <_pytest.config.Config object at 0x7fd334fb0d10>

    @pytest.fixture(scope="module")
    def connection(
        request, runner: str, auto_authenticate: bool, pytestconfig
    ) -> ProcessTestRunner:
        # TODO: this fixture override changes the return type of the original `connection` fixture,
        #       which might lead to problems due to broken assumptions
    
        if runner == "dask":
>           from openeo_test_suite.lib.process_runner.dask import Dask

src/openeo_test_suite/tests/processes/processing/conftest.py:40: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
src/openeo_test_suite/lib/process_runner/dask.py:48: in <module>
    registry = create_process_registry()
src/openeo_test_suite/lib/process_runner/dask.py:35: in create_process_registry
    specs = {
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

.0 = <list_iterator object at 0x7fd2ebd67e80>

    specs = {
>       func.__name__: getattr(specs_module, func.__name__)
        for func in processes_from_module
    }
E   AttributeError: module 'openeo_processes_dask.specs' has no attribute 'mask'

src/openeo_test_suite/lib/process_runner/dask.py:36: AttributeError

[soxofaan]

In 505e6d3 I also black-listed 2024.1.2 in pyproject.toml for now