iThemes Blocked Crawler
-
Sorry for the delay in response.
It looks like your site ‘blocks’ our crawler (and being more technical it blocks our user agent or IPs). When accessing the URL using a browser — we get a valid response.
But when our crawler tries to fetch this it is being blocked (and sends back response code 500, which indicates an error)
In order to resolve this, could you please allow our user agent which is “Mozilla/5.0 (Java) outbrain” .
Maybe you also block our IPs.
These are the address ranges of our data centers, out of which we execute crawling.
4.28.73.128/27 [4.28.73.128 → 4.28.73.159]
74.201.198.64/27 [74.201.198.64 → 74.201.198.95]
204.145.89.32/27 [204.145.89.32 → 204.145.89.63]
64.94.43.96/27 [64.94.43.96 → 64.94.43.127]
74.217.148.96/27 [74.217.148.96 → 74.217.148.127]
4.71.33.160/27 [4.71.33.160 → 4.71.33.191]
50.31.185.32/27 [50.31.185.32 → 50.31.185.63]
64.74.232.32/27 [64.74.232.32 → 64.74.232.63]Please help! Thanks
- The topic ‘iThemes Blocked Crawler’ is closed to new replies.