Support » Plugin: Wordfence Security - Firewall & Malware Scan » PageSpeed Insights – Lighthouse returned error: ERRORED_DOCUMENT_REQUEST

  • Resolved detoris

    (@detoris)


    I did various tests for better values ​​in PageSpeed ​​Insights. I activated nginx cache for web hosting and changed entries for EXPIRES CACHING and Cache-Control headers in the .htaccess file. Then I tested each time with PageSpeed ​​Insights. Unfortunately, PageSpeed ​​Insights no longer works after various tests. The following message appears:

    Lighthouse returned error: ERRORED_DOCUMENT_REQUEST. Lighthouse was unable to reliably load the page you requested. Make sure you are testing the correct URL and that the server is properly responding to all requests. (Status code: 503)

    I deactivated nginx cache again during web hosting and imported a backup of the .htaccess. However, the error message still comes. After much more testing, I’ve found that when I turn off Wordfence, PageSpeed ​​Insights works again as it did before. When activating Wordfence the error is there again. Activating the Wordfence learning mode did not bring any improvement.

    My question now would be the following:
    What has changed in Wordfence and how can I correct it without losing all my settings?

    Thanks a lot

    The page I need help with: [log in to see the link]

Viewing 7 replies - 1 through 7 (of 7 total)
  • Update:
    After I didn’t have a solution yesterday and this morning, I’ve come a little closer to the matter. Apparently Wordfence is building a blocking list for different IPs during the tests. For example because of “non-existent page” or “blocked for Exceeded the maximum number of page not found errors per minute for humans”. That was not the case before.

    Plugin Support WFAdam

    (@wfadam)

    Hello @detoris and thanks for reaching out to us!

    If these Lighthouse requests are being blocked by Wordfence, you should see these requests in the Tools > Live Traffic page. Are you seeing any blocks here? If so, you could whitelist them directly from here.

    Let me know what you find!

    Thanks!

    detoris

    (@detoris)

    Thank you for the feedback.
    Yes, I see the blocking in live traffic. From there, however, I can only execute unblock directly and not include it in a whitelist. As far as I know, I can only manage and feed different entries in the various whitelists in the options.

    Can I feed a whitelist directly from the live traffic page?
    Or in which whitelist would I have to make which entries (Options / Whitelisted IP addresses that bypass all rules?)?

    Thanks

    Plugin Support WFAdam

    (@wfadam)

    You could add all of Lighthouse’s IPs to the Whitelisted IP addresses that bypass all rules if the option to whitelist is not available in the Live Traffic.

    I was browsing Pagespeed support and I read a post about there not being a set group of IPs that they use, so whitelisting these might be hard. Unless you have the information I don’t, in that case, I would love for you to share!

    Thanks again!

    detoris

    (@detoris)

    Summary of my results:
    Due to the different tests by PageSpeed ​​Insights with and without cache etc. PageSpeed ​​Insights wants to have access to many files with a shortened invalid path at once. Due to the number of failed accesses (tried to access a non-existent page) in a very short time, Wordfence blocks the access of PageSpeed ​​Insights according to my narrowly set rule. Hence the above error message.

    After the observations in Live Traffic, I added 3 IPs to the whitelist:
    66.249.93.220
    66.249.93.221
    66.249.93.222

    After that, PageSpeed ​​Insights can run the tests again without any problems. Alternatively, I could have extended the rule, but that would have been too uncertain for me.

    For information, the complete information about the blocking again

    IP: 66.249.93.222 Hostname: google-proxy-66-249-93-222.google.com
    Human/Bot: Bot
    Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4143.7 Safari/537.36 Chrome-Lighthouse

    IP: 66.249.93.221 Hostname: google-proxy-66-249-93-221.google.com
    Human/Bot: Bot
    Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4143.7 Safari/537.36 Chrome-Lighthouse

    IP: 66.249.93.220 Hostname: google-proxy-66-249-93-220.google.com
    Human/Bot: Bot
    Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4143.7 Safari/537.36 Chrome-Lighthouse

    I don’t know why PageSpeed ​​Insights shows this strange behavior and searches for non-existent pages, but I can now manage the behavior in Wordfence.
    So I put this thread on resolved.

    Thanks for your support

    Plugin Support WFAdam

    (@wfadam)

    Thanks @detoris for the information!

    Glad you could find a resolution to this!

    Let us know if you need anything else!

    Thanks again for your support!

    I just submitted my new website in Google Pagespeed Insights for a website speed audit. But it is returning an error as follows:

    Lighthouse returned error: FAILED_DOCUMENT_REQUEST. Lighthouse was unable to reliably load the page you requested. Make sure you are testing the correct URL and that the server is properly responding to all requests. (Details: net::ERR_FAILED).

    The website https://blueheronhealthnewsreviews.com/ absolutely has no issue & load perfectly at present but still returning the above error.

    Anybody having idea to fix the issue?

Viewing 7 replies - 1 through 7 (of 7 total)
  • You must be logged in to reply to this topic.