wordpress generates its own virtual robots.txt file
-
Hi Guys! Need your help.
I want my robots.txt file to look like this:1. Robots.txt
I fix it using Yoast plugin as:User-agent: *
Disallow: /comment-page-
Disallow: /feed/
Disallow: /trackback/
Disallow: /license.txt
Disallow: /readme.html
Disallow: /xmlrpc.php
Disallow: /*.php$
Disallow: /?s=*Sitemap: http:// hannasles dot com/sitemap_index.xml
But within a certain period of time it changes automatically to:
User-agent: *
Crawl-delay: 1
Disallow: /wp-content/plugins/
Disallow: /wp-admin/I have found out that WordPress can generates a virtual robots.txt file if the site root does not contain a physical file. But I checked and I have got the robots.txt in the root as a physical file. And I am using Yoast plugin.
Can you give me any suggestions?
thanks!
Hanna
The topic ‘wordpress generates its own virtual robots.txt file’ is closed to new replies.