Querystring searches hang; no response

Hi Kim,
Just to add: having looked at the code I’d offer the opinion that the issue possibly arises at line 323 in the call to decodeURIComponent(urlparam[1] when the string being searched is of the form “%xxx%” – I believe the leading % is maybe throwing it off, in my example it will be trying to process ‘%Potter%’ and won’t like the %P at the start?
Regards,
Graham

• This reply was modified 3 years, 4 months ago by grl570810.

Hi Kim (& Peter),

Thanks for the response; detailed reply coming up!

Yes, if I adjust the URL manually to replace the % signs with %25 the correct results are returned. But that unfortunately just leads me to the next roadblock.

The form that is completed to run the search is created in Gravity Forms. If I try to adjust the querystring to include the %25 escape character it rejects it with an error “The text you have entered is not valid. For security reasons, some characters are not allowed”: 

Nor can I get any of the equivalent coding such as % to work. I am reaching out to the GF support forums to see if there’s a way around that but my reading so far suggests that I’ll need to code my own validation routine to replace the standard GF one. Not a good thing as I’d then need to hack the validation every time GF updates.

I then compared the code at v4, where this worked, with the current code. The change is minimal; in v4 it read:
data[urlparam[0]] = urlparam[1];    (line 285 of the 4.2.7 version)
in v5 it is:
data[urlparam[0]] = decodeURIComponent(urlparam[1]);   (line 323)
so it’s clearly the introduction of decodeURIComponent to parse the querystring that is the root of the issue. I have confirmed this by editing the current code to put back the v4 line and everything works as expected. But again, this is not a good long term solution as updates will re-break the system.

My guess is that you changed the code for security considerations (given the error that GF throws), however the consequence of it seems to me to be that you can’t do ‘contains’ (i.e. SQL LIKE) searches from a GF form.

Given that GF is such a widely used form plugin (and it’s not unlikely that other form plugins will show the same behaviour (in fact if it’s a security concern they should)), I believe that a solution that allows the search to be generated ‘out-of-the-box’ is needed. I’m no programmer in modern languages (can still smash Algol 68 though!) but I’d suggest pre-parsing the querystring to convert a % to a %25 before feeding it to decodeURIComponent might provide a solution. But I could be way off the mark…

Anyway, I hope you find my observations helpful. I’ll going to run my site with the amended js file for the present, if you do change that code and want me to test a new version pre-release I’ll be happy to do so for you.

Regards,
Graham

Hi Peter,

It’s ironic that the combined efforts of you and the GF developers to maintain a secure environment have resulted in the two plugins becoming incompatible out-of-the-box. I’m waiting on some feedback from the GF people to see if we can devise a way around it at their end. And yes, I realised shortly after I’d posted that my suggestion was dumb; if you are passed a validly escaped percent as %25 it would convert that to %%25 and break.

Failing a hack in GF to pass the string you require, and thinking about it a bit more deeply, IMHO the real root of the issue is that SQL uses % as a special character in a manner different to its use in HTML as an escape character indicator, and since you are dealing in SQL the incompatibility arises. As far as I can determine none of the other RFC 3986 reserved characters would give you any problem in a querystring. In other words % is very much a special case for you. Would it be possible to ‘tokenise’ it in your processing, e.g. specify that % is represented in a querystring as something like {pc} and pre-process that before passing to DecodeURIComponent? Once again this is me speculating rather than having a detailed idea of how you implement that in modern code 🙂

The form is at https://movies.blewis.com.au Note there are four forms on the page, Title Search is the simplest as it only targets a single column, all others search across multiple ones.

I’ve also created you guys an admin login on the site as that was suggested earlier in the thread. I have sent you the login details via your contact form.

Regards,
Graham

Hi Peter & Kim,
Sorry to be slow in replying again, but it actually turned out to be relatively simple to add a custom action in Gravity Forms to change the % in the query string to %25 before passing it to the search page. Just pressure of work prevented me from testing and confirming that until now.
I’ll leave this here in case anyone else hits a similar issue. All I needed was to add this to the functions.php file (which I did, of course, in a child theme)…

// This will replace the % characters in the querystring being passed to WPDA with %25 as it requires

add_filter( 'gform_confirmation', function ( $confirmation, $form, $entry ) {

    $confirmation = str_replace ('%', '%25', $confirmation);
    
    return $confirmation;
}, 11, 3 );

That may not be applicable out the box to all implementations, but was all I needed for my simple example and hopefully will point folks on the right track.
Regards,
Graham