Hi @nbh707
I hope you’re well today!
Amazon and some other services are detecting these checks as automated/bots and that results in requests being rejected or blocked. I wouldn’t say they are “blacklisting users” but rather that they are throttling or ultimately (if there’s too many requests (checks) made from it) blocking specific IPs or domains. But I’m afraid I can’t tell that for sure.
One thing to check would be to look into HTTP response headers for such checks. If you look at the list of links marked as broken, for each link you can click on “Details” link right below the link status (in “Status” column on the list). This gives a bit more details – in right part of the page you’ll see “Response headers” and “request headers”. Those can tell a bit more on what happened – if it was indeed Amazon that rejected URL check or there was some timeout.
If you could share that information with us for one or two example Amazon links that are reported as broken for you, we could confirm if it’s Amazon issue or “something on the way”.
That said, our developers are looking into it but I don’t have any ETA on a solution as there are quite a lot of other updates and fixes “on the table”. Basically, the plugin is pretty much being rewritten from scratch so it might take some time.
Kind regards,
Adam
Hi @nbh707
I hope you are doing well and safe!
We haven’t heard from you in a while, I’ll mark this thread as resolved.
Feel free to let us know if you have any additional question or problem.
Best Regards
Patrick Freitas
Thread Starter
NBH707
(@nbh707)
Hi Adam-Here’s some code for you. I note they do mention their API here.
ink last checked: June 15, 2021
HTTP code: 503
Response time: 0.022 seconds
Final URL: https://www.amazon.com/BETWEEN-PARENT-CHILD-Solutions-Problems/dp/038045369X?&linkCode=ll1&tag=genes2teens-20&linkId=beb1d2dddc60ea9abbb9c890dfde1fe8&language=en_US&ref_=as_li_ss_tl
Redirect count: 0
Instance count: 1
This link has failed 4 times.
This link has been broken for 14 days.
Log: [Warning] Could't follow the redirect URL (if any) because safemode or open base dir enabled
=== HTTP code : 503 ===
Response headers
================
HTTP/2 503
content-type: text/html
content-length: 2671
server: Server
date: Tue, 15 Jun 2021 08:52:47 GMT
x-amz-rid: FG6Z7JJ817A395A5W1HT
vary: Content-Type,Accept-Encoding,X-Amzn-CDN-Cache,X-Amzn-AX-Treatment,User-Agent
last-modified: Fri, 14 May 2021 19:08:48 GMT
etag: "a6f-5c24ef9383000"
accept-ranges: bytes
strict-transport-security: max-age=47474747; includeSubDomains; preload
permissions-policy: interest-cohort=()
x-cache: Error from cloudfront
via: 1.1 7f4d5d15a00b6ae82bb7aabc4560d3a7.cloudfront.net (CloudFront)
x-amz-cf-pop: IAD79-C1
x-amz-cf-id: Q-jFaBcE4khj9sPxBPYgnK0MVuYT8kBzYNcbELM-4e3deY1jlFyvFg==
Request headers
================
GET /BETWEEN-PARENT-CHILD-Solutions-Problems/dp/038045369X?&linkCode=ll1&tag=genes2teens-20&linkId=beb1d2dddc60ea9abbb9c890dfde1fe8&language=en_US&ref_=as_li_ss_tl HTTP/2
Host: www.amazon.com
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36
accept: */*
referer: https://genes2teens.com
connection: close
range: bytes=0-2048
Response HTML
================
<!--
To discuss automated access to Amazon data please contact api-services-support@amazon.com.
For information about migrating to our APIs refer to our Marketplace APIs at https://developer.amazonservices.com/ref=rm_5_sv, or our Product Advertising API at https://affiliate-program.amazon.com/gp/advertising/api/detail/main.html/ref=rm_5_ac for advertising use cases.
-->
<!doctype html>
<html>
<head>
<meta charset="utf-8">
<meta http-equiv="x-ua-compatible" content="ie=edge">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<title>Sorry! Something went wrong!</title>
<style>
html, body {
padding: 0;
margin: 0
}
img {
border: 0
}
#a {
background: #232f3e;
padding: 11px 11px 11px 192px
}
#b {
position: absolute;
left: 22px;
top: 12px
}
#c {
position: relative;
max-width: 800px;
padding: 0 40px 0 0
}
#e, #f {
height: 35px;
border: 0;
font-size: 1em
}
#e {
width: 100%;
margin: 0;
padding: 0 10px;
border-radius: 4px 0 0 4px
}
#f {
cursor: pointer;
background: #febd69;
font-weight: bold;
border-radius: 0 4px 4px 0;
-webkit-appearance: none;
position: absolute;
top: 0;
right: 0;
padding: 0 12px
}
@media (max-width: 500px) {
#a {
padding: 55px 10px 10px
}
#b {
left: 6px
}
}
#g {
text-align: center;
margin: 30px 0
}
#g img {
max-width: 90%
}
#d {
display: none
}
#d[src] {
display: inline
}
</style>
</head>
<body>
<a href="/ref=cs_503_logo"><img id="b" src="https://images-na.ssl-images-amazon.com/images/G/01/error/logo._TTD_.png" alt="Amazon.com"></a>
<form id="a" accept-charset="utf-8" action="/s" method="GET" role="search">
<div id="c">
<input id="e" name="field-keywords" placeholder="Search">
<input name="ref" type="hidden" value="cs_503_search">
<input id="f" type="submit" value="Go">
Link is broken.
Hi @nbh707
Thanks for sharing the debug.
As I suspected, it’s unfortunately the case of requests blocked by Amazon. As you noticed, they suggest using API and that’s exactly the issue – Amazon and some other service detect such requests (such as e.g. link check) and consider it “automated” and “unauthorized”, thus blocking it.
I’m afraid there’s no simple way around that currently, until some support for Amazon Marketplace API is added to the plugin. We are looking into this but I don’t have an ETA so for now you might want to add those links to exclusions list (on “Settings -> Link Checker -> Which Links to Check”) page.
Kind regards,
Adam
Hi @nbh707
I hope you are doing well.
As Adam told me, the issue is because Amazon is blocking the request, I attempt a regular cURL and got the same 503 status.
The plugin will inform the user and then the user can check if it is a false positive or not.
Our developers do have a plan to improve how the plugin check for Amazon links but there is no ETA.
We are marking this as resolved for now, feel free to ping us any time you need.
Best Regards
Patrick Freitas
