regex for url-encoded characters

Resolved sproutchris
(@sproutchris)

5 months, 1 week ago
I was hoping someone might be able to provide a helping hand with a regex query that includes trailing URL-encoded characters. Google Search Console is reporting to our team a lot of instances like the following:
Using the first example above, I can easily redirect it to

https://www.example.com/news/local/page/2769/

without the trailing URL-encoded characters involved using this regex (note: I don’t need the redundant page URL parameter so I’m clearing it out with the redirect):

^/news/local/page/(.*)/\?page=(.*) -> /news/local/page/$1/

However, I’m not sure how to deal with the undetermined random amount of trailing URL-encoded characters at the end the URL. Any advice here would be highly appreciated!

Viewing 7 replies - 1 through 7 (of 7 total)

Plugin Author John Godley
(@johnny5)

5 months, 1 week ago

The characters at the end are no different from other characters.

Do you have your canonical URL data appearing on the pages correctly? You typically don’t need to redirect random query parameters.

Thread Starter sproutchris
(@sproutchris)

5 months, 1 week ago

Google’s issue with those URL’s are that they’re throwing a 403 with those extra characters on the end so I thought I could implement a redirect to prevent that.

Plugin Author John Godley
(@johnny5)

5 months, 1 week ago

I don’t think the query parameters will cause a 403 error. Is something else involved here? It’s probably better to fix the 403 itself.

Thread Starter sproutchris
(@sproutchris)

5 months ago

I might have to look into an nginx fix to avoid the 403’s instead, although in this case a 403 or some other error might be appropriate since the URL-encoded character %2F is the forward slash character so to me this just looks like a bad URL altogether:

/news/local/page/2769/?page=261///////

There shouldn’t be any slashes after the query parameters so I don’t know why Google Search Console ever picked these URLs up in the first place; I just know I’m trying to correct the issue that it’s reporting.

Thread Starter sproutchris
(@sproutchris)

5 months ago

Another thing I found odd (and again, this might just be something with my nginx configuration) is that three or more URL-encoded slash characters (%2F) throws a 403:

/news/local/page/2769/?page=261%2F%2F%2F

but one or two does not and the active redirect (^/news/local/page/(.*)/\?page=(.*)) applies.

Plugin Author John Godley
(@johnny5)

5 months ago

Yeah, there’s something off with your setup as that shouldn’t cause an error, and doesn’t on any of the sites I’ve tested it with.

Thread Starter sproutchris
(@sproutchris)

5 months ago

I figured it out; it’s getting caught up in the 7G WAF installed on my server. I disabled one of the options and now the 403 is gone. Thank you sincerely for talking it through with me, John.

Viewing 7 replies - 1 through 7 (of 7 total)

You must be logged in to reply to this topic.