WordPress.org

1. malloyoy
   Member
   Posted 7 years ago #
   

   This problem is really puzzling me. Search engine indexing started out fine but now everytime a search engine finds a new url it will add a random category or post title to the end of the url.

   e.g. - boxofficeblog.com

   /buzz/alien-autopsy/oscars/
   (+http://search.msn.com/msnbot.htm)

   /buzz/alien-autopsy/news/
   (+http://search.msn.com/msnbot.htm)

   /news/daniel-craig-promises-a-more-human-bond/articles/
   (+http://search.msn.com/msnbot.htm)

   It gets worse as a site gets indexed more.

   e.g. - babespy.net (no naked images, honest, I link to them)

   /category/aimee-chuhaloff/ftv-girls/angel-dark/vanessa-private-glamour/mariah-rito/ftv-girls/cassandra-lynn/jesse-capelli/adrianne-curry/cassia-riley/joanna-krupa/scarlett-johansson/zusie-galaxia/joanna-krupa/
   (NimbleCrawler 2.0.0)

   .htaccess - (url structure is /%category%/%posttitle%/)

   <IfModule mod_rewrite.c>
   RewriteEngine On
   RewriteBase /
   RewriteCond %{REQUEST_FILENAME} !-f
   RewriteCond %{REQUEST_FILENAME} !-d
   RewriteRule . /index.php [L]
   </IfModule>

   How can I solve this problem? 3 of my sites have this same problem.

2. malloyoy
   Member
   Posted 7 years ago #
   

   I don't mean to be annonying, I just don't want this to be buried.

   Does anyone have any ideas what the problem could be?

3. Samuel B
   moderator
   Posted 7 years ago #
   

   Your best bet would be to delete your .htaccess file - create another empty one and upload it. Then update your permalinks letting WP write to your .htaccess.

4. malloyoy
   Member
   Posted 7 years ago #
   

   Thanks for the reply, I will try that.

5. malloyoy
   Member
   Posted 7 years ago #
   

   No luck trying this. These URL's aren't on my website at all so where are the crawlers/bots finding them?

6. jpump
   Member
   Posted 7 years ago #
   

   From this tag -

   <input type="hidden" name="redirect_to" value="/category/aimee-chuhaloff/ftv-girls/angel-dark/vanessa-private-glamour/mariah-rito/ftv-girls/cassandra-lynn/jesse-capelli/adrianne-curry/cassia-riley/joanna-krupa/scarlett-johansson/zusie-galaxia/joanna-krupa/"/></div>
   </form>

Topic Closed

This topic has been closed to new replies.