Plugin Author
AITpro
(@aitpro)
Nope BPS has never done anything with a robots.txt file and most likely we will never add anything to BPS that does anything with a robots.txt file. BPS does not have anything included in BPS that would affect a robots.txt file and never will. A robots.txt file falls under the category of SEO and not Security.
If you have added any additional custom htaccess code to BPS Custom Code then check to make sure it is not causing the problem. You can do the standard BPS troubleshooting steps to confirm, eliminate or isolate a problem: https://wordpress.org/support/topic/bps-troubleshooting-steps?replies=1
More likely causes of the problem would be corrupt cache in a caching plugin that needs to be cleared. Server connectivity problems with your host web server. You can use the Google Webmaster Tools Fetch tool and Robots tester tools to test that your robots.txt file is valid and reachable.
Plugin Author
AITpro
(@aitpro)
I have had two days in last week in which Googlebot says they either can’t reach my robots.txt 100% or last time 33 errors.
Actually the problem cannot be related to BPS in any way because it is an intermittent problem. If you added any additional custom htaccess code to BPS Custom Code and it was causing the problem then the problem would happening consistently/permanently and not intermittently. So most likely your host server was having issues/problems on those 2 days last week or a caching problem with a caching plugin was occurring (very unlikely, but possible).
The only caching code I use is the BPS code. The only other thing that has changed is the BlueHost addition of the SSO plugin to all my sites on 8/2/2015 without my permission. And I upgraded to WP 4.2.4
See
https://wordpress.org/support/topic/sso-plugin-in-wp-contentmu-plugins?replies=2#post-7304897
Every time I use Rexswain or another headerchecker or the Wabmaster tools Robots checker I get 200 returns or OK on webmaster tools.
Plugin Author
AITpro
(@aitpro)
Ok so most likely your host server was having some problems for a couple of days and is no longer having any problems. You can check and verify that by looking at your server log file during the dates when your site/server was having issues/problems. Your PHP error log file can also be used to check and veryify that by looking at the dates when your site/server was having issues/problems.
Plugin Author
AITpro
(@aitpro)
This is a long shot, but possible. I found a bug in the MOJO Marketplace plugin (reported the bug to the plugin author last week here with the cause and solution: https://wordpress.org/support/topic/mu-plugin-in-new-install?replies=1#post-7265002) that affects other plugin’s Cron Jobs. So hypothetically if the MOJO plugin / SSO plugin (same code used in both apparently according to the plugin author) affected another plugin’s Cron Job that was doing something with your robots.txt file then that could have caused the problem. This is very unlikely because the problem would most likely be permanently/consistently happening and not an intermittent problem.
Plugin Author
AITpro
(@aitpro)
Here is an awesome site for checking if your site/server is up around the world from different relays: http://www.host-tracker.com/
Plugin Author
AITpro
(@aitpro)
Assuming all questions have been answered – thread has been resolved.
Thread Start Date: 8-15-2015 to 8-16-2015
Thread Resolved/Current Date: 8-18-2015
Comments: non-BPS plugin issue. Most likely an intermittent server problem.
Yes and thank you for your always awesome help.
