negative-arg sqrt calls happened on Apple Silicon CI run around moments (non-deterministic, log attached)

[slayoo]

for the record, here's the error log:

examples/PySDM_examples/Jensen_and_Nugent_2017/simulation.py:118: in run
    output_products = super()._run(
examples/PySDM_examples/utils/basic_simulation.py:20: in _run
    self._save(output)
examples/PySDM_examples/Jensen_and_Nugent_2017/simulation.py:128: in _save
    super()._save(output)
examples/PySDM_examples/utils/basic_simulation.py:10: in _save
    value = v.get()
PySDM/products/impl/product.py:99: in get
    result = self._impl(**kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <PySDM.products.size_spectral.size_standard_deviation._SizeStandardDeviation object at 0x33aa6f3e0>
kwargs = {}, tmp = array([-6.89228949e-13])

    def _impl(self, **kwargs):
        ActivationFilteredProduct.impl(self, attr=self.attr, rank=1)
        tmp = np.empty_like(self.buffer)
        tmp[:] = -self.buffer**2
        ActivationFilteredProduct.impl(self, attr=self.attr, rank=2)
        tmp[:] += self.buffer
>       tmp[:] = np.sqrt(tmp)
E       RuntimeWarning: invalid value encountered in sqrt

PySDM/products/size_spectral/size_standard_deviation.py:42: RuntimeWarning

non deterministic :(

Originally posted by @slayoo in #1458 (comment)

[AgnieszkaZaba]

I can't recreate it. Tried to run on MacOs 14.4 with python 3.12. Works fine.

[AgnieszkaZaba]

In the same folder one of tests is not passing:
(So sorry for the formatting)

======================= 1 xfailed, 3 warnings in 13.78s ========================
XFAIL [100%]
variables = {'Settings': <class 'PySDM_examples.Jensen_and_Nugent_2017.settings.Settings'>, 'Simulation': <class 'PySDM_examples.J...ass 'AttributeError'>, 'BaseException': <class 'BaseException'>, ...}, 'figure': <function figure at 0x144f77560>, ...}

@staticmethod
@pytest.mark.xfail(strict=True, reason="TODO #1266")
def test_supersaturation_maximum(variables):
    supersaturation = np.asarray(variables["output"]["products"]["S_max"])
    assert signal.argrelextrema(supersaturation, np.greater)[0].shape[0] == 1

  assert 1.2 * PER_CENT < np.nanmax(supersaturation) < 1.4 * PER_CENT

E assert (1.2 * 0.01) < np.float64(0.006053614904341087)
E + where np.float64(0.006053614904341087) = <function nanmax at 0x103e8a070>(array([ nan, -0.14217759, -0.14036018, -0.13853827, -0.13671186,\n -0.13488092, -0.13304545, -0.13120544, -0.12936087, -0.12751172,\n -0.125658 , -0.12379967, -0.12193674, -0.12006919, -0.11819701,\n -0.11632017, -0.11443868, -0.11255252, -0.11066167, -0.10876612,\n -0.10686586, -0.10496088, -0.10305117, -0.1011367 , -0.09921747,\n -0.09729347, -0.09536468, -0.09343109, -0.09149269, -0.08954946,\n -0.08760139, -0.08564847, -0.08369069, -0.08172803, -0.07976048,\n -0.07778802, -0.07581066, -0.07382837, -0.07184114, -0.06984896,\n -0.06785181, -0.0658497 , -0.0638426 , -0.0618305 , -0.05981339,\n -0.05779127, -0.05576413, -0.05373195, -0.05169472, -0.04965244,\n -0.04760511, -0.04555271, -0.04349525, -0.04143271, -0.03936511,\n -0.03729245, -0.03521472, -0.03313195, -0.03104414, -0.02895133,\n -0.02685353, -0.0247508 , -0.02264319, -0.02053079, -0.01841368,\n -0.01629203, -0.01416601, -0.01203591, -0.00990212, -0.00776521,\n -0.0056261 , -0.00348639, -0.00134949, 0.00077447, 0.00284298,\n 0.00467379, 0.00583357, 0.00605361, 0.00597113, 0.00532725,\n 0.0044723 , 0.00373555, 0.00320559, 0.00285031, 0.00261229,\n 0.00244572, 0.00232164, 0.00222352, 0.00214222, 0.00207265,\n 0.00201178, 0.0019577 , 0.00190912, 0.00186509, 0.0018249 ,\n 0.001788 , 0.00175393, 0.00172235, 0.00169295, 0.00166549,\n 0.00163975, 0.00161556, 0.00159276, 0.00157123, 0.00155084,\n 0.00153151, 0.00151313, 0.00149563, 0.00147895, 0.00146302,\n 0.00144778, 0.00143319, 0.0014192 , 0.00140577, 0.00139286,\n 0.00138045, 0.00136849, 0.00135697, 0.00134586, 0.00133513,\n 0.00132476, 0.00131473, 0.00130503, 0.00129564, 0.00128654,\n 0.00127771, 0.00126915, 0.00126083, 0.00125276, 0.00124492,\n 0.00123729, 0.00122987, 0.00122265, 0.00121562, 0.00120877,\n 0.00120209, 0.00119559, 0.00118924, 0.00118305, 0.00117701,\n 0.00117111, 0.00116534, 0.00115972, 0.00115421, 0.00114883,\n 0.00114357, 0.00113843, 0.00113339, 0.00112846, 0.00112364,\n 0.00111892, 0.00089385, -0.00076808, -0.00092783, -0.00094654,\n -0.00095194, -0.00095614, -0.00096031, -0.00096456, -0.0009689 ,\n -0.00097335, -0.00097789, -0.00098253, -0.00098729, -0.00099216,\n -0.00099714, -0.00100225, -0.00100748, -0.00101285, -0.00101835,\n -0.00102399, -0.00102978, -0.00103573, -0.00104184, -0.00104812,\n -0.00105457, -0.0010612 , -0.00106803, -0.00107506, -0.00108229,\n -0.00108975, -0.00109745, -0.00110538, -0.00111358, -0.00112204,\n -0.00113079, -0.00113984, -0.00114921, -0.00115892, -0.00116898,\n -0.00117942, -0.00119026, -0.00120153, -0.00121326, -0.00122548,\n -0.00123821, -0.00125151, -0.0012654 , -0.00127994, -0.00129518,\n -0.00131117, -0.00132797, -0.00134566, -0.00136432, -0.00138403,\n -0.00140489, -0.00142703, -0.00145058, -0.00147567, -0.00150251,\n -0.00153129, -0.00156224, -0.00159567, -0.00163191, -0.00167137,\n -0.00171455, -0.00176206, -0.00181466, -0.00187331, -0.00193925,\n -0.00201406, -0.00209989, -0.00219962, -0.00231734, -0.00245895,\n -0.00263338, -0.00285524]))
E + where <function nanmax at 0x103e8a070> = np.nanmax

jensen_and_nugent_2017/test_fig_5.py:57: AssertionError

Process finished with exit code 0

[AgnieszkaZaba]

Full log

[AgnieszkaZaba]

I captured it using
pytest --count=1000 -x tests/smoke_tests/parcel_c/jensen_and_nugent_2017/test_fig_6.py::TestFig6::test_radii -W error::RuntimeWarning

[AgnieszkaZaba]

Found this release note of python 3.12.8.

[AgnieszkaZaba]

@slayoo this issue now is more urgent.
I have another idea that it might be connected to new release of anyio-4.7.0
When I reinstall it to the previous version I don't have errors (at least for the past 30k test runs).