False positive Pages Not Found

  • Resolved acekin

    (@acekin)


    10 years, 3 months ago

    I have been having this problem for quite some time. On the Pages Not Found tab there are many URLs listed as not found. However, I can link on every one of the links and it takes me to a valid page with no error message. I am sharing a partial screen capture:

    https://copy.com/cxSpOoCPbXA2Lylz

    There are 46 links that all work. The activity certainly looks suspicious, the same IP trying to access comments on a couple of posts. I wonder it the behavior is triggering something that results in the source not getting to the target and thus resulting in a Page Not Found issue.

    Cemal

    https://wordpress.org/plugins/wordfence/

Viewing 2 replies - 1 through 2 (of 2 total)
  • Plugin Author WFMattR

    (@wfmattr)

    10 years, 3 months ago

    Hi Cemal,

    These are a little different from the last ones you mentioned — these in particular are caused by a broken crawler that is requesting the URLs incorrectly.

    When you visit the same pages in a browser, the browser removes the “#comment-nnnn” portion in the request it sends to the site, and it uses that internally to scroll to the correct comment. The crawler causing the 404’s is actually including that part when sending the request to the server — if you visit same URLs using a browser that isn’t logged into your site, you can see how the URL appears without the “#comment” portion in Live Traffic.

    I don’t know anything about that crawler — it might be benign or could be a type that is trying to steal content. If it does that often, it may be worth blocking its IP.

    -Matt R

    Thread Starter acekin

    (@acekin)

    10 years, 3 months ago

    Thanks Matt, I was not particularly concerned with this batch of URLs but I still get …/about/ or …/contact/ links reported as not found. (I am using … for the domain name to prevent additional links to crawl!) It may very well be a problem with my ISP or with the logic of Wordfence. The latter is the main reason for my reporting, I know those pages exist anyway.

    Thanks,

    Cemal

Viewing 2 replies - 1 through 2 (of 2 total)

The topic ‘False positive Pages Not Found’ is closed to new replies.