Discourage search engines from indexing this site

enricobt
(@enricobt)

2 years, 3 months ago

Hello to the whole community,

a few weeks ago I found this option “Discourage search engines from indexing this site” in “Settings>Reading” activated by itself because there are no other ADMINs active on the site.
I put the option back to off as it should be to avoid blocking search engines. Then I disabled all automatic updates of the plugins & themes, updating them gradually when needed.
Unfortunately, in spite of this, every quite often (1 time in 2 weeks) I still find the option “ON” and this is causing me several problems with indexing the site, as hundreds of pages have been discarded in Google Search Console for this reason.
Is there a possibility to block this option to OFF (option_name “blog_public” found in the WP options table) so that it can no longer be modified except by PHP admin?
Thank you
Henry

The page I need help with: [log in to see the link]

Viewing 12 replies - 1 through 12 (of 12 total)

Moderator threadi
(@threadi)

2 years, 3 months ago

This is not normal behaviour, it is probably caused by a plugin you are using or your theme. Check the plugins you are using. If any of them are thematically orientated towards SEO, check their settings. You should also check the theme to see if there are any settings in this regard.

You could use customised code to prevent such settings from being changed. But if you really want to prevent indexing in X years, e.g. when setting up a staging system, then you won’t even think about it. Finding the cause would therefore be the better way.
Moderator Yui
(@fierevere)

永子

2 years, 3 months ago
You can use robots.txt directives for achieving the same setting
( X-Robots-Tag: noindex )

put into robots.txt
```
User-agent: *
Disallow: /
```
Thread Starter enricobt
(@enricobt)

2 years, 3 months ago

HI @threadi

thanks for your help.

My SEO plugin is Yoast and changed some configurations on it, but I don’t think that this solve my problem. In few days I will find it out.

As alternative can I use robots.txt in a way that it will always allow search crawlers to index all my site ? this of course, despite the option value, “Discourage search engines from indexing this site”, on

Regards

Enrico
Moderator threadi
(@threadi)

2 years, 3 months ago
Yes, of course you can also control this via robots.txt. Yui has already shown what it should look like if you want to prohibit indexing. If you want to allow it, you can do without any robots.txt or write this in it:
```
User-agent: *
Allow: /
```
More informations: https://developers.google.com/search/docs/crawling-indexing/robots/intro?hl=en
alexikakos
(@alexikakos)

1 year, 10 months ago

Hi all,

did you find any answer about this? what changes this option?

geniel
(@geniel)

1 year, 8 months ago

A few days ago I made the same discovery. My longstanding site was not being indexed. It started July 8 and dwindled down to just a few clicks in a week. the ‘discourage search engines’ option I had no idea existed, had been checked. I checked with my SEO plugin AIO and they assured me no changes like this could be implemented. Since I unchecked it, a few days ago it has remained unchecked, but clicks have not improved.

deena777
(@deena777)

1 year, 7 months ago

Hi Team,

Discourage search engines from indexing this site, option Once de-selected, it is automatically re-selected

please refer to attached screen recorder Video Link :

https://www.awesomescreenshot.com/video/30538653?key=d9c7676d7aa207399fdf4cfefae2806b

waqakram
(@waqakram)

1 year, 7 months ago

This seem to be happening again, I am not sure what caused it on our website, Anyone has a clue.

Moderator threadi
(@threadi)

1 year, 7 months ago

Firstly: it would be nice if everyone could open a separate topic for their own problem in accordance with the forum guidelines. This would allow you to deal with one specific problem on one project in one topic and not 3 or 4 in one topic – this is confusing and not very helpful for those who want to help.

About the problem: any plugin you are using will affect the setting. Deactivate all plugins and then try to change the setting again. If it works normally then one of the plugins was to blame. You can find out which one was at fault by deactivating the plugins individually and testing.

deena777
(@deena777)

1 year, 7 months ago

Hi threadi,

But the source code of this website is working fine after moving it to another server.

Moderator threadi
(@threadi)

1 year, 7 months ago

Unfortunately, I don’t know exactly what you are talking about. Please create a separate topic for your very own problem: https://wordpress.org/support/forum/how-to-and-troubleshooting/#new-topic-0
deena777
(@deena777)

1 year, 7 months ago
Hi threadi

We have created a staging environment with the same code and database in a different server, And I have un-ticked “Discourage search engines from indexing this site” in the staging environment then saved it . it is working fine here. So I think there wouldn’t be a plugin issue in the website.
- This reply was modified 1 year, 7 months ago by deena777.