Fix hybrid newton python tests imports #6522

lechevaa · 2025-10-08T07:01:01Z

This PR is linked to OPM/opm-simulators#6424 on Hybrid Newton flag.

The related python tests are failing due to some import errors. I am not sure that I reproduced exactly the setup that is run during tests.

The tests have a tensorflow dependency for creating dummy neural networks. I added tensorflow-cpu in the requirements as it is lighter than its gpu version.

Regarding other imports, I mimicked the other tests way of importing.

I'm providing few details for clarity:
I followed installation using Option 1, and then run unittest from top of test folder using:
python -m unittest discover -s test -p "test_*.py" -v and the hybrid newton tests are working.

daavid00 · 2025-10-08T07:06:07Z

jenkins build this please

akva2 · 2025-10-08T07:40:27Z

You can test using python/docker and python/docker/test_wheels.

various issues;
python 3.8 & python 3.9:

======================================================================
ERROR: test.ml.hybrid_newton.test_hybrid_newton (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test.ml.hybrid_newton.test_hybrid_newton
Traceback (most recent call last):
  File "/root/.pyenv/versions/3.8.19/lib/python3.8/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/root/.pyenv/versions/3.8.19/lib/python3.8/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/tmp/opm-simulators-tests/test/ml/hybrid_newton/test_hybrid_newton.py", line 7, in <module>
    from .utils import collect_input_features, compute_output_vars, write_config, extract_newtit_from_file, extract_unrst_variables
  File "/tmp/opm-simulators-tests/test/ml/hybrid_newton/utils.py", line 223, in <module>
    def extract_newtit_from_file(filename: str | Path) -> list[int]:
TypeError: unsupported operand type(s) for |: 'type' and 'type'

3.10-3.13:

======================================================================
ERROR: test.ml.hybrid_newton.test_hybrid_newton (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test.ml.hybrid_newton.test_hybrid_newton
Traceback (most recent call last):
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/tmp/opm-simulators-tests/test/ml/hybrid_newton/test_hybrid_newton.py", line 11, in <module>
    from kerasify import export_model
ModuleNotFoundError: No module named 'kerasify'

it is safe to assumes 3.8 and 3.9 would also miss the kerasify module of course.

akva2 · 2025-10-08T07:50:31Z

i'll make some noise here, just ignore..

akva2 · 2025-10-08T07:50:43Z

jenkins build wheels please

lechevaa · 2025-10-08T07:54:44Z

Thanks for pointing that out, I will try to use python/docker and python/docker/test_wheels

Regarding kerasify, as it comes from opm-common. How would you recommend to manage it?

akva2 · 2025-10-08T08:04:24Z

well, for starters, you cannot import what you do not package, the ml directory is not included in the opm-common wheels.

akva2 · 2025-10-08T08:08:04Z