Thread Starter
EddieG
(@eddievet)
Hi there
After a thorough investigation i found out that RankMath hasn’t added a robot.txt
Such a file does not exist on my server.
Ever w3 cache is showing an error – FTP credentials don’t allow to write to file /var/www/samplesite.com/public_html/robots.txt
RankMath- Edit Robots.txt gives this message – robots.txt file is not writable.
Trying to edit the file but not there to do anything
Must also note that i have lost more than half of my organic traffic.
Can you please let me know how to fix this error?
Thanks,
Eddie
-
This reply was modified 2 years, 7 months ago by
Steven Stern (sterndata).
-
This reply was modified 2 years, 7 months ago by
EddieG.
-
This reply was modified 2 years, 7 months ago by EddieG.
Hello @eddievet,
Thank you for contacting the support and sorry for any inconvenience that might have been caused due to that.
We allow editing robots.txt and htacess file only when FS_METHOD constant is set to direct.
If you are getting robots.txt file is not writable then that means the FS_METHOD constant is not set to direct or your server itself doesn’t allow editing this file from the backend.
Please get in touch with the hosting provider or you can add a Robots.txt file manually to the root of WordPress installation. It should look like this:
https://rankmath.com/robots.txt
Hope that helps.
Thread Starter
EddieG
(@eddievet)
HI Rank Math
After checked my site samplesite.com/robots.txt i found this txt file
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Sitemap: https://www.samplesite.com/sitemap_index.xml
Is this the way RankMath suppose to write txt file? Could this be the cause of my traffic drop by at least 50%?
Thread Starter
EddieG
(@eddievet)
Hi Rank Math
Adding to the same issue. You said
“If you are getting robots.txt file is not writable then that means the FS_METHOD constant is not set to direct”
So i checked the config file and the fs-method it is set to direct. See below
/** Allow Direct Updating Without FTP */
define(‘FS_METHOD’, ‘direct’);
Now, going a little back to the same issue, which at the same time i received this error from w2tc (see image – https://ibb.co/18vcfV4)
I managed to fix that error by creating a robots.txt file on public_html, and that made w3tc happy but looks like I have another message from Rank Math (see image – https://ibb.co/zJSsGhq)
The first message wants me to make robots.txt writable, which Rank Math never asked for it before. It means that I have to change permission to 777,right? Isn’t that dangerous?
How do i proceed fixing the error?
Thanks
Thanks
-
This reply was modified 2 years, 7 months ago by EddieG.
Hello @eddievet
Not sure what the issue is now. The robots.txt contents you shared looks correct. It is certainly not the cause of the traffic drop.
Rank Math will only edit the virtual robots.txt file that is generated by WordPress on the fly.
If an actual robots.txt file is present then that will be used and you won’t be able to edit it in Rank Math.
Hope that clears the confusion.
Also, here’s a great guide on how to recover from a Google update:
https://rankmath.com/blog/google-algorithm-update-recovery/
Thread Starter
EddieG
(@eddievet)
Hi
“It is certainly not the cause of the traffic drop.”
I know and understand that you’re saying but how do i convince my self that soon after (12 hours later) i created the file the ranking started picking up. It is not to the level i was but surely it is picking up.
What is your best advice, shall i keep the one i’ve created or should i delete it in order for RankMath to work better?
Thanks
Hello @eddievet
Please keep the physical Robots.txt file on the server. That won’t hamper the working of the plugin.
You just need to edit the content of the Robots.txt file directly using FTP.
Hope that helps. If you have any further questions, please feel free to ask. We are here to assist.
Thank you.
