Prevent puppeteer protocol timeouts

[bisswanger]

FAQ

• Yes, my issue is not about variability or throttling.
• Yes, my issue is not about a specific accessibility audit (file with axe-core instead).

URL

https://www.adobe.com

What happened?

We are using lighthouse for in-house analysis of Webpage performance.
We are rarely experiencing some issues where the page load seems to "take forever"

In these cases, we are receiving the attached exception log after around 3 minutes.
The exception is not thrown from the call to await lighthouse but instead thrown as an uncaught exception which causes the node process to crash.

What did you expect?

Exception is either thrown / or caught on lighthouse side

What have you tried?

Replication is not easy as it happens rarely on our end-2-end test pages.

Though the uncaught exception can be simulated with a small code change:

lighthouse/core/gather/driver/navigation.js

Line 100 in 6a4c751

session.setNextProtocolTimeout(Infinity);

Changing the value to 10 will simulate a similar behavior / exception.

In that case the node handler for an uncaught exception will be caught:

    process.on('uncaughtException', function(err) {
        console.log('Caught exception');
        console.log(err);
    });

How were you running Lighthouse?

node

Lighthouse Version

11.0.0

Chrome Version

117

Node Version

18.18.0

OS

Alpine 3.18

Relevant log output

file:///eaas-lighthouse-runner/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/Connection.js:39
    #error = new ProtocolError();
             ^

ProtocolError: Page.navigate timed out. Increase the 'protocolTimeout' setting in launch/connect calls for a higher timeout if needed.
    at <instance_members_initializer> (file:///eaas-lighthouse-runner/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/Connection.js:39:14)
    at new Callback (file:///eaas-lighthouse-runner/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/Connection.js:43:16)
    at CallbackRegistry.create (file:///eaas-lighthouse-runner/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/Connection.js:82:26)
    at Connection._rawSend (file:///eaas-lighthouse-runner/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/Connection.js:203:26)
    at CdpCDPSession.send (file:///eaas-lighthouse-runner/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/CDPSession.js:58:33)
    at ProtocolSession.sendCommand (file:///eaas-lighthouse-runner/node_modules/lighthouse/core/gather/session.js:96:44)
    at gotoURL (file:///eaas-lighthouse-runner/node_modules/lighthouse/core/gather/driver/navigation.js:101:42)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async _navigate (file:///eaas-lighthouse-runner/node_modules/lighthouse/core/gather/navigation-runner.js:100:55)
    at async _navigation (file:///eaas-lighthouse-runner/node_modules/lighthouse/core/gather/navigation-runner.js:242:26)

Node.js v18.18.0

[adamraine]

We have a PROTOCOL_TIMEOUT error that is emitted if a CDP command takes longer than 30s. When we do session.setNextProtocolTimeout(Infinity) it prevents Lighthouse form throwing an error on the 30s timeout but puppeteer can still emit a timeout error if the command exceeds their default timeout of 180s. In general, there should be a way to upstream our protocol timeout overrides to puppeteer.

That being said, something is probably going on in the browser backend for Page.navigate to take longer than 180s. Unfortunately these issues are difficult to debug (see #6512)

[adamraine]

puppeteer/puppeteer#11535