robots.txt ??
-
Hello Jason! I hope you are well.
For sometime now I Slimstat keeps showing robots.txt in the reports instead of the visited pages, is this a bug?
Here’s a screenshot:
https://s1.postimg.org/yl323xp6n/Capture.pngPlease let me know.
Thanks!
-
Hi,
Your web server should be configured via .htaccess to not send those requests to WordPress at all. You may want to ask your service provider to help you with this.
Best,
JasonYou mean my hosting company?
This started showing in the reports all of a sudden, I haven’t changed anything myself.
Yeah, your hosting provider might have changed something in your .htaccess or in the way they process it. Or you might have installed a plugin that modified the way your .htaccess is structured. By default, requests to robots.txt should be handled by the web server, and never passed onto WordPress.
Jason
It’s weird because I have not changed anything and it just started appearing all of a sudden.
Is there anything I can do to resolve this before asking my hosting company, since it’s only showing in Slimstat?Maybe you could take a look at some point?
You can always configure Slimstat ( Settings > Filters ) to ignore those pageviews, if you don’t want to ask your provider. However having WordPress serve all those requests might impact on your website performance, so I would still recommend that you talk to them.
Jason
I’ve wasted all day and I have not figured out what causes this.
Do you think it may be a conflict with “Elementor” page builder that I use?Could you test it out too please?
Hi,
we use similar page builders, and have never noticed any issues of this kind. Again, your provider is your best friend in this case. The issue needs to be addressed in your .htaccess
Best,
JasonI will ask them.
However, I asked you about Elementor, because the issue seems to have disappeared when I deactivated it.Where can we download it? Is it a free plugin?
Yes, it’s a very popular page builder:
Thanks. I will install it on our sandbox and run some tests.
Unfortunately, I was not able to reproduce the issue on our end. Elementor is a pretty cool plugin, though! I didn’t know it existed.
Best,
JasonHey Jason!
Sorry for the late reply.
Elementor is indeed great!I was not able to sort out why the Robots.txt thing is still happening…
I’m still testing out to find out the cause. I think though, that it may have to do with the “Wappalyzer” addon/extension that I have installed on my browsers (Chrome & Firefox), but I’m not sure yet.Could you also take a look if you have the time at some point?
-
This reply was modified 8 years, 8 months ago by
catmaniax.
I’m now 99% certain that it was Wappalyzer that was causing that, in case someone else has the same issue.
-
This reply was modified 8 years, 8 months ago by
The topic ‘robots.txt ??’ is closed to new replies.