PageSpeed Insights suddenly returning FAILED_DOCUMENT_REQUEST on Square sites

[privatenumber]

Apologies if this is not the right place to report PageSpeed Insights related bugs.

Multiple Square sites I was monitoring via the PageSpeed Insights API suddenly started returning FAILED_DOCUMENT_REQUEST on March 3rd, 2021 15:40 EST. I assume this is affecting all Square sites.

Provide the steps to reproduce

Enter https://benchmarks-empty-site.square.site/ on PageSpeed Insights

Another example: https://boba-guys.square.site/

What is the current behavior?

Error:

Lighthouse returned error: FAILED_DOCUMENT_REQUEST. Lighthouse was unable to reliably load the page you requested. Make sure you are testing the correct URL and that the server is properly responding to all requests. (Details: net::ERR_CONNECTION_FAILED)

What is the expected behavior?

For a successful page speed report to be returned. There were no changes on the site, and Lighthouse runs fine on them via Chrome and lighthouse CLI.

I checked the PSI release notes but no red flags.

Environment Information

• Affected Channels: PageSpeed Insights
• Lighthouse version: Confirmed working on v7.2.0
• Chrome version: Confirmed working on Version 88.0.4324.192 (Official Build) (x86_64)
• Node.js version: v14.15.4
• Operating System: macOS 11.2.1

Related issues

[devtools-bot]

Thanks! Appreciate you filing this bug. 👏

This is a known issue, most well described in #2784. So, we'll automatically close this as a duplicate.

However, if you believe your bug is different than the cases described there, please comment here with "necessarily-wide-alpaca" and I'll reopen this bug. 🤖 Beep beep boop.

[privatenumber]

necessarily-wide-alpaca

[privatenumber]

(#2784 is a Lighthouse issue, whereas this is a PSI issue)

[privatenumber]

After checking our access logs, it seems like PSI tries to access the http version of the URL (despite https being specified), hits the 302 redirect to https, but doesn't follow the redirect (just stops).

Logs prior to March 3rd show that PSI was following the 302 redirect to https.

[connorjclark]

please follow #12222

also related is #7326

[privatenumber]

This was marked resolved but the problem still persists for me.

Try: https://benjerrytakeout.square.site/
(Different Square site just to make sure there's no cache hit or whatever)

[connorjclark]

It seems to be working now. Or maybe it is flaky? EDIT: yes, it is flaky.

[privatenumber]

Yes, flaky for me too. Got a success and a failure:

Thanks for re-opening.

[privatenumber]

Seems to be working consistently as of a day or two ago. Yesterday, I was able to make 14 successful PSI audits for Square sites consecutively.

Thank you for investigating, @connorjclark and Lighthouse team!