Re-add Safari render test job in the CI

[stepankuzmin]

Disabling homebrew updates helped to eliminate the Safari timeouts, so re-adding them. At first, I tried to run Safari render tests on Apple Silicon in CircleCI, but it seems to work on the Intel chip with disabled homebrew updates for 6 consecutive runs.

• Re-add Safari render test job in the CI with disabled homebrew updates
• Increase the allowed value for 3 render tests

Launch Checklist

• briefly describe the changes in this PR
• apply changelog label ('bug', 'feature', 'docs', etc) or use the label 'skip changelog'
• add an entry inside this element for inclusion in the mapbox-gl-js changelog: <changelog></changelog>

[stepankuzmin]

The CodeBuild is failing because its Node version is incompatible with the required Node version for selenium-webdriver

error [email protected]: The engine "node" is incompatible with this module. Expected version ">= 14.20.0". Got "14.19.3"

[mourner]

Cool! How did you trace the timeouts to the brew update thing? Seems pretty unrelated at first...

[stepankuzmin]

How did you trace the timeouts to the brew update thing

That was pretty random, actually. I just wanted to try if disabling updates would help with timeouts without switching to Apple Silicon, and it did. At least the timeouts are not reproduced for several runs.

[stepankuzmin]

Disabling homebrew updates doesn't help with Safari timeouts. I've switched the test-render-mac-safari-dev job to the new macos.m1.large.gen1 resource class to see how that would impact Safari timeouts.

[mourner]

I've switched the test-render-mac-safari-dev job to the new macos.m1.large.gen1 resource class to see how that would impact Safari timeouts.

The timings look nice, looks very fast! Let's land this if timeouts don't seem to happen. BTW, I wonder what the costs will be with this — if one sequential render test job runs faster than 3x parallel Intel jobs, maybe we should switch the Chrome/Mac ones as well (removing parallelization too) and the cost will be offset by less time in the executor?

[stepankuzmin]

Currently, we are using the default macOS resource class, medium. The price for medium is 50 credits/min ($0.0300). CircleCI will deprecate the macOS medium resource classes automatically upgrading jobs to Gen2 resources (macos.x86.medium.gen2) which is 75 credits/min ($0.045). According to the resource classes list, the macos.m1.large.gen1 price is 400 credits/min ($0.2400, subject to change).

There is some issue with the Safari render tests in CircleCI, but the job duration with the M1 resource class goes down from 15min to 5min approximately. So I assume we'd spend a bit more on CircleCI if we switched to macos.m1.large.gen1.

[mourner]

@stepankuzmin so in summary, M1 is 5x more expensive but 3x faster, so we'd pay about 1.5x more. Seems fair to me for the benefit of faster CI times.

[stepankuzmin]

After several more runs, some Safari render tests might still be flaky due to the issue with the missing tiles. However, the same applies to the other browsers, so I think we are good to merge.

[stepankuzmin]

Noted that sometimes, test-render-mac-safari-dev fails with

Global error: AssertionError: null == true at http://localhost:7357/dist/mapbox-gl-dev.js, line 1774