Tests failing consistency

[cvanelteren]

This acts as a reminder to solve the issue for some tests failing within the image compare

ultraplot/tests/test_1dplots.py::test_networks
❌ Image files did not match
RMS: 26.85 (Tolerance: 2)
ultraplot/tests/test_legend.py::test_contour_legend_with_label
❌ Image files did not match
RMS: 5.71 (Tolerance: 2)
ultraplot/tests/test_legend.py::test_contour_legend_without_label
❌ Image files did not match
RMS: 5.71 (Tolerance: 2)

Will use this issue as a scratchbook to jot down some notes.

[cvanelteren]

In mpl 3.10 the filtering of no labels is no longer applied (see https://matplotlib.org/stable/api/prev_api_changes/api_changes_3.10.0.html). Since we are applying _no_label we are not seeing it in mpl3.9 but we are seeing it in 3.10 which I can confirm.

Within each run we are not comparing across mpl versions, so this should not be the root of the problem.

[cvanelteren]

Ok so the network stuff is not our fault but how nx passed the decorator. It uses an index which is not how we parse our seed. Will follow up with them (see networkx/networkx#8116)

[cvanelteren]

It's a bit annoying that locally I cannot reproduce the problem. I can now but still don't understand why it is happening

update after resetting the error disappears again.. bit frustrating.

[cvanelteren]

Ok figured out the problem. With those two tests there is a modification of the rc params which is sometimes is affected by a racing conditions. We either fully make the RC bullet proof or fix it with --dist load

[beckermr]

The networkx tests are still not reproducible. We should fix that separately.

[beckermr]

See #278

[beckermr]

One more: #279

[beckermr]

Ok the network test now is stable. So it's only the threading issue we have to deal with if we want to.

[beckermr]

We should add a ci test for prs only that compares the code in the pr to itself. This will help us catch these instabilities earlier.

[cvanelteren]

I am trying to restore the PR to its pre-edit stage -- nearly there.

[cvanelteren]

For introducing xdist we need to break the circular dependency between the colour maps and the re initialization which is currently causing an issue to make the reading and writing thread safe of the rc object.

On a longer time scale I think we should also get rid as much as possible the global or dynamic assignments present in many files.

#281 is a recent example of this that I just happened to have open and seemed easy enough implement

[beckermr]

How do we get around global state in matplotlib itself?

[cvanelteren]

The thread local copies are protected currently with the thread local variable and if we move all the interactions through configuration it will work as it will hold two rc one for ultraplot and one for matplotlib. Currently the issue is that we get a infinite regress due to colormaps calling image.lut from RC -- the changes in #281 are nice to have irregardless of the xdist stuff as it is currently a headache to find where stuff is added and sometimes they are difficult to find due to the dynamic codes as in #281. We should move these to ultraplot/__init__.py if we really cannot get around them to have them in a centralized space.