EODataAccessGateway async init fails with: "pkg_resources.UnknownExtra: eodag 3.0.1 has no such extra feature 'ecmwf'"

[andretheronsa]

Describe the bug
I have Eodag 3.0.1 as a dependency of my project. Installed without any options: dependencies = ["eodag"].
When I instantiate the dag I sometimes get a warning for pkg_resources.UnknownExtra: eodag 3.0.1 has no such extra feature 'ecmwf'. Sometimes it warns for 'csw' and not 'ecmf'.

I override the default providers and user conf with only a single provider (geodes) to simplify debugging. So it should not even be aware of any other providers.

The EODataAccessGateway is instantiated in an async event loop and if multiple of these are instantiated at the same time it causes this error. This makes it a little tricky to debug and reproduce.

It appears as though

• CSWSearch is registered as an entry point and its optional dependencies are not installed (as expected).
• pkg_resources tries to load CSWSearch, fails to resolve eodag[csw], and raises UnknownExtra.
• This behavior only happens sometimes because pkg_resources lazily loads entry points(?), meaning the failure occurs only when CSWSearch is actually accessed or it only happens due to some async / eodag shared object issue

Code To Reproduce

 pip install eodag

import asyncio
import functools
from eodag import EODataAccessGateway

for iter in range(3):
    await async_download()

async def async_download():
  loop = asyncio.get_running_loop()
  pfunc = functools.partial(sync_download)
  return await asyncio.wait_for(loop.run_in_executor(None, pfunc))

def sync_download()
    os.environ["EODAG_CFG_FILE"] = "user_conf_geodes.yml"
    os.environ["EODAG_PROVIDERS_CFG_FILE"] = "providers_geodes.yml"
    dag = EODataAccessGateway()

Output

Exception has occurred: UnknownExtra
eodag 3.0.1 has no such extra feature 'ecmwf'
KeyError: 'ecmwf'

The above exception was the direct cause of the following exception:

  File "/usr/local/app/app/downloader.py", line 346, in _download
    dag = EODataAccessGateway()
  File "/usr/local/app/app/downloader.py", line 308, in download
    return await asyncio.wait_for(loop.run_in_executor(None, pfunc), timeout=self.total_timeout)
  File "/usr/local/lib/python3.11/site-packages/eodag/api/core.py", line 163, in __init__
    self._plugins_manager = PluginManager(self.providers_config)
  File "/usr/local/lib/python3.11/site-packages/eodag/plugins/manager.py", line 108, in __init__
    entry_point.load()
  File "/usr/local/lib/python3.11/site-packages/pkg_resources/__init__.py", line 2772, in load
    self.require(*args, **kwargs)  # type: ignore[arg-type]
  File "/usr/local/lib/python3.11/site-packages/pkg_resources/__init__.py", line 2799, in require
    reqs = self.dist.requires(self.extras)
  File "/usr/local/lib/python3.11/site-packages/pkg_resources/__init__.py", line 3105, in requires
    raise UnknownExtra(f"{self} has no such extra feature {ext!r}") from e
pkg_resources.UnknownExtra: eodag 3.0.1 has no such extra feature 'csw'
OR
pkg_resources.UnknownExtra: eodag 3.0.1 has no such extra feature 'ecmwf'

Environment:

• Python version: 3.11
• EODAG version: 3.0.1

Summary: Earth Observation Data Access Gateway
Home-page: https://github.com/CS-SI/eodag
Author: CS GROUP - France
Author-email: [email protected]
License: Apache 2.0
Location: /usr/local/lib/python3.11/site-packages
Requires: annotated-types, boto3, botocore, click, geojson, jsonpath-ng, lxml, orjson, pydantic, pydantic-core, PyJWT, pyproj, pyshp, pystac, python-dateutil, PyYAML, requests, setuptools, shapely, stream-zip, tqdm, typing-extensions, urllib3, Whoosh

python -c "import pkg_resources; print(list(pkg_resources.iter_entry_points('eodag.plugins.search')))"
<string>:1: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
[EntryPoint.parse('BuildPostSearchResult = eodag.plugins.search.build_search_result:BuildPostSearchResult'), EntryPoint.parse('BuildSearchResult = eodag.plugins.search.build_search_result:BuildSearchResult'), EntryPoint.parse('CSWSearch = eodag.plugins.search.csw:CSWSearch [csw]'), EntryPoint.parse('CopMarineSearch = eodag.plugins.search.cop_marine:CopMarineSearch'), EntryPoint.parse('CreodiasS3Search = eodag.plugins.search.creodias_s3:CreodiasS3Search'), EntryPoint.parse('DataRequestSearch = eodag.plugins.search.data_request_search:DataRequestSearch'), EntryPoint.parse('ODataV4Search = eodag.plugins.search.qssearch:ODataV4Search'), EntryPoint.parse('PostJsonSearch = eodag.plugins.search.qssearch:PostJsonSearch'), EntryPoint.parse('PostJsonSearchWithStacQueryables = eodag.plugins.search.qssearch:PostJsonSearchWithStacQueryables'), EntryPoint.parse('QueryStringSearch = eodag.plugins.search.qssearch:QueryStringSearch'), EntryPoint.parse('StacSearch = eodag.plugins.search.qssearch:StacSearch'), EntryPoint.parse('StaticStacSearch = eodag.plugins.search.static_stac_search:StaticStacSearch')

Additional remarks

• When I run it in a debugger and let each concurrent dag instantiate before continuing it does not happen.
• I do not expect Eodag to be async safe as I don't think the package sells itself as an async package but it would be an improvement to not get this unrelated exception upon instantiating a dag async.

[sbrunato]

Hello @andretheronsa and thanks for pointing out this issue. We'll try to reproduce it and see how this can be handled