[BUG] test_load fails if Biomodels website happens to be down.

[djinnome]

Is there an existing issue for this?

• I have searched the existing issues

Problem description

When I submitted pull request: #1450, github actions ran all the unit tests, and the job failed, not because my contribution had a bug, but because Biomodels happened to be down.

When I asked github copilot to explain the error, this is what it said:

The job failed because the test test_remote_load for the model_id 'BIOMD0000000633' encountered a ReadTimeout when trying to download the model from the BioModels repository. The test is defined in tests/test_io/test_web/test_load.py and tries to load the model from a remote source:

@pytest.mark.parametrize(
    "model_id, num_metabolites, num_reactions",
    [("e_coli_core", 72, 95), ("BIOMD0000000633", 50, 35)],
)
def test_remote_load(model_id: str, num_metabolites: int, num_reactions: int) -> None:
    model = load_model(model_id, cache=False)
    assert len(model.metabolites) == num_metabolites
    assert len(model.reactions) == num_reactions

The failure was caused by a network timeout:

httpx.ReadTimeout: The read operation timed out
...
raise RuntimeError(
    f"The connection to the BioModels repository failed."
) from error

Solution:

1. Mark this test with @pytest.mark.flaky or @pytest.mark.network to indicate it is network-dependent and may occasionally fail due to external issues.
2. Increase the timeout parameter (if supported) in the code that fetches remote models.
3. Skip remote tests in CI by adding a conditional skip, e.g., using an environment variable:

   import os
   import pytest
   
   @pytest.mark.parametrize(...)
   def test_remote_load(...):
       if os.getenv("CI") == "true":
           pytest.skip("Skipping remote load tests on CI")
       ...

4. Alternatively, mock the remote response in CI, as is already done for other tests in this file.

Implementing any of the above will make CI more robust to transient network issues and avoid false negatives. For reference, see the test file here (ref: ca0ed13):
https://github.com/opencobra/cobrapy/blob/ca0ed139aa3ea6775ccf149bb91af1c26e71e6b0/tests/test_io/test_web/test_load.py

Code sample

Code run:

pytest tests/test_io/test_web/test_load.py

Traceback:

=================================== FAILURES ===================================
  ___________________ test_remote_load[BIOMD0000000633-50-35] ____________________
  
      @contextlib.contextmanager
      def map_httpcore_exceptions() -> typing.Iterator[None]:
          global HTTPCORE_EXC_MAP
          if len(HTTPCORE_EXC_MAP) == 0:
              HTTPCORE_EXC_MAP = _load_httpcore_exceptions()
          try:
  >           yield
  
  .tox/py39/lib/python3.9/site-packages/httpx/_transports/default.py:101: 
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  .tox/py39/lib/python3.9/site-packages/httpx/_transports/default.py:250: in handle_request
      resp = self._pool.handle_request(req)
  .tox/py39/lib/python3.9/site-packages/httpcore/_sync/connection_pool.py:256: in handle_request
      raise exc from None
  .tox/py39/lib/python3.9/site-packages/httpcore/_sync/connection_pool.py:236: in handle_request
      response = connection.handle_request(
  .tox/py39/lib/python3.9/site-packages/httpcore/_sync/connection.py:103: in handle_request
      return self._connection.handle_request(request)
  .tox/py39/lib/python3.9/site-packages/httpcore/_sync/http11.py:136: in handle_request
      raise exc
  .tox/py39/lib/python3.9/site-packages/httpcore/_sync/http11.py:106: in handle_request
      ) = self._receive_response_headers(**kwargs)
          Parameters
          ----------
          model_id : str
              The identifier of the desired metabolic model. This is typically repository
              specific.
          repositories : iterable
              An iterable of repository accessor instances. The model_id is searched in order.
      
          Returns
          -------
          bytes
              A gzip-compressed, UTF-8 encoded SBML document.
      
          """
          for repository in repositories:
              logger.info(
                  f"Attempting to fetch '{model_id}' from the {repository.name} repository."
              )
              try:
                  return repository.get_sbml(model_id=model_id)
              except OSError:
                  logger.debug(
                      f"Model '{model_id} not found in the local "
                      f"repository {repository.name}.'"
                  )
              except httpx.HTTPStatusError as error:
                  if error.response.status_code == 404:
                      logger.debug(
                          f"Model '{model_id}' not found in the {repository.name} repository."
                      )
                      continue
                  raise RuntimeError(
                      f"The connection to the {repository.name} repository failed."
                  ) from error
              except httpx.RequestError as error:
  >               raise RuntimeError(
                      f"The connection to the {repository.name} repository failed."
                  ) from error
  E               RuntimeError: The connection to the BioModels repository failed.
  
  .tox/py39/lib/python3.9/site-packages/cobra/io/web/load.py:175: RuntimeError

Environment

Github actions environment

Anything else?

No response

[cdiener]

Yeah Biomodels is flaky or is blocking Github at times. I like solution 1 but AFAICT those marks do not come with the normal pytest. Are those in an additional package? Or did you mean adding the xfail mark with a specific reason?