Can i stop this crawling?

  • sjoerd89

    (@sjoerd89)


    9 years ago

    Hi all,

    i recently renewed my website and went from example.com/nl/…. to just example.com.
    Now when i check my 404 reports i see them full of errors from /nl/ pages wich do not excist anymore. I think 90% of these error searches are from crawlers and i was wondering if there is anyway to stop them from looking here. I feel like this is really hurting my google ranks since it finds so many 404-pages. I tried
    disallow: /nl/
    in my robots.txt but i feel like this is not doing anything.

    Any help would be nice,

    Thanks

Viewing 3 replies - 1 through 3 (of 3 total)
  • Peter Hardy-vanDoorn

    (@petervandoorn)

    9 years ago

    There are plenty of plugins that redirect 404s to actual pages. Have a search in the repository for “404 url redirect” to find one that you like the look of.

    Thread Starter sjoerd89

    (@sjoerd89)

    9 years ago

    If i would redirect every page that would take me weeks of my time, what i am looking for is making crawling bots stop looking in pages that start with /nl/ since they do not excist anymore.

    Sorry if i was not clear

    Liam Stewart

    (@gingeds)

    9 years ago

    You can redirect 404s via .htaccess

    Add the following to your .htaccess file.

    Redirect 301 /nl http://example.net/

Viewing 3 replies - 1 through 3 (of 3 total)

The topic ‘Can i stop this crawling?’ is closed to new replies.