Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retrying on known Semantic Scholar flaky SSL error in get_s2_doc_details_from_doi #522

Merged
merged 2 commits into from
Oct 3, 2024

Conversation

jamesbraza
Copy link
Collaborator

I hit this flaky SSL error last night, now with Semantic Scholar:

    | Traceback (most recent call last):
    |   File "/path/to/repo/.venv/lib/python3.12/site-packages/paperqa/agents/search.py", line 407, in process_file
    |     await tmp_docs.aadd(
    |   File "/path/to/repo/.venv/lib/python3.12/site-packages/paperqa/docs.py", line 360, in aadd
    |     doc = await metadata_client.upgrade_doc_to_doc_details(
    |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/path/to/repo/.venv/lib/python3.12/site-packages/paperqa/clients/__init__.py", line 202, in upgrade_doc_to_doc_details
    |     if doc_details := await self.query(**kwargs):
    |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/path/to/repo/.venv/lib/python3.12/site-packages/paperqa/clients/__init__.py", line 150, in query
    |     await gather_with_concurrency(
    |   File "/path/to/repo/.venv/lib/python3.12/site-packages/paperqa/utils.py", line 111, in gather_with_concurrency
    |     return await asyncio.gather(*(sem_coro(c) for c in coros))
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/path/to/repo/.venv/lib/python3.12/site-packages/paperqa/utils.py", line 109, in sem_coro
    |     return await coro
    |            ^^^^^^^^^^
    |   File "/path/to/repo/.venv/lib/python3.12/site-packages/paperqa/clients/client_models.py", line 108, in query
    |     return await self._query(client_query)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/path/to/repo/.venv/lib/python3.12/site-packages/paperqa/clients/semantic_scholar.py", line 325, in _query
    |     return await get_s2_doc_details_from_doi(
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/path/to/repo/.venv/lib/python3.12/site-packages/paperqa/clients/semantic_scholar.py", line 273, in get_s2_doc_details_from_doi
    |     details = await _get_with_retrying(
    |               ^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/path/to/repo/.venv/lib/python3.12/site-packages/tenacity/asyncio/__init__.py", line 189, in async_wrapped
    |     return await copy(fn, *args, **kwargs)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/path/to/repo/.venv/lib/python3.12/site-packages/tenacity/asyncio/__init__.py", line 111, in __call__
    |     do = await self.iter(retry_state=retry_state)
    |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/path/to/repo/.venv/lib/python3.12/site-packages/tenacity/asyncio/__init__.py", line 153, in iter
    |     result = await action(retry_state)
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/path/to/repo/.venv/lib/python3.12/site-packages/tenacity/_utils.py", line 99, in inner
    |     return call(*args, **kwargs)
    |            ^^^^^^^^^^^^^^^^^^^^^
    |   File "/path/to/repo/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 398, in <lambda>
    |     self._add_action_func(lambda rs: rs.outcome.result())
    |                                      ^^^^^^^^^^^^^^^^^^^
    |   File "/Users/jamesbraza/.pyenv/versions/3.12.5/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/_base.py", line 449, in result
    |     return self.__get_result()
    |            ^^^^^^^^^^^^^^^^^^^
    |   File "/Users/jamesbraza/.pyenv/versions/3.12.5/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    |     raise self._exception
    |   File "/path/to/repo/.venv/lib/python3.12/site-packages/tenacity/asyncio/__init__.py", line 114, in __call__
    |     result = await fn(*args, **kwargs)
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/path/to/repo/.venv/lib/python3.12/site-packages/paperqa/utils.py", line 413, in _get_with_retrying
    |     async with session.get(
    |   File "/path/to/repo/.venv/lib/python3.12/site-packages/aiohttp/client.py", line 1355, in __aenter__
    |     self._resp: _RetType = await self._coro
    |                            ^^^^^^^^^^^^^^^^
    |   File "/path/to/repo/.venv/lib/python3.12/site-packages/aiohttp/client.py", line 659, in _request
    |     conn = await self._connector.connect(
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/path/to/repo/.venv/lib/python3.12/site-packages/aiohttp/connector.py", line 557, in connect
    |     proto = await self._create_connection(req, traces, timeout)
    |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/path/to/repo/.venv/lib/python3.12/site-packages/aiohttp/connector.py", line 1002, in _create_connection
    |     _, proto = await self._create_direct_connection(req, traces, timeout)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/path/to/repo/.venv/lib/python3.12/site-packages/aiohttp/connector.py", line 1293, in _create_direct_connection
    |     raise ClientConnectorError(req.connection_key, exc) from exc
    | aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host api.semanticscholar.org:443 ssl:default [nodename nor servname provided, or not known]
    +------------------------------------

Applying the fix from #444 to Semantic Scholar get_s2_doc_details_from_doi

@jamesbraza jamesbraza added the bug Something isn't working label Oct 3, 2024
@jamesbraza jamesbraza self-assigned this Oct 3, 2024
@dosubot dosubot bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Oct 3, 2024
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Oct 3, 2024
@jamesbraza jamesbraza merged commit e2627b4 into main Oct 3, 2024
5 checks passed
@jamesbraza jamesbraza deleted the s2-error branch October 3, 2024 18:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working lgtm This PR has been approved by a maintainer size:M This PR changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants