Images not automatically compressed when uploaded through the block editor

Resolved wpwebdevj
(@wpwebdevj)

4 months, 3 weeks ago

It doesn’t matter if I select “Compress new images in the background (Recommended)” or “Compress new images during upload”, my images are not automatically compressed when added from the block editor / gutenberg. If I upload the images from the media library then it works as expected.

Viewing 14 replies - 1 through 14 (of 14 total)

Plugin Author TinyPNG
(@tinypng)

4 months, 2 weeks ago

Hi,

Thanks for reporting!

We’re able to reproduce the issue you report for the “Compress new images during upload” option while adding content in the block editor. For the “Compress new images in the background (Recommended)” option however we are currently unable to reproduce it in the block editor. For us, all images added in the block editor with the “Compress new images in the background (Recommended)” option checked are compressed automatically in WordPress 5.5.

Can you perhaps share more details about your use case? The WordPress/PHP versions you’re using as well as the exact steps needed to reproduce the issue for the “Compress new images in the background (Recommended)” option could be very helpful. You can also opt to contact us with the information at support@tinify.com

In the meantime we will further investigate the issue for the “Compress new images during upload” option in the block editor. Thanks again for the report!

Thread Starter wpwebdevj
(@wpwebdevj)

4 months, 2 weeks ago

Hi,

Thanks for the reply. WordPress version 5.5.3. PHP 7.4.

I should also mention that I have disabled wp-cron and I’m running a real cronjob on my server every 2 hours. Could this be causing the issue? Even if I wait over 2 hours, the images are still not optimized.

Thread Starter wpwebdevj
(@wpwebdevj)

4 months, 2 weeks ago

Also, I just realized that the “Compress new images in the background (Recommended)” actually doesn’t work for me when uploading from the media library. It only works when uploading from the media library AND the setting is “Compress new images during upload”. When uploading from the block editor then nothing works.

Plugin Author TinyPNG
(@tinypng)

4 months, 2 weeks ago

Hi,

Thanks for the updates. That explains why we’re not able to reproduce that part of the issue if “Compress new images in the background (Recommended)” isn’t working in your Media Library either. Disabling wp-cron should not have a negative effect on the plugin as far as I know.

When you have configured the plugin to use the “Compress new images in the background (Recommended)” option, then WordPress will make a HTTP request to itself to compress an image in the background when you add it to your Media Library.

We have seen a few cases where the webserver configuration used for hosting WordPress didn’t allow WordPress to create a HTTP request to itself. This could possibly also be the case for you. The plugin uses the WordPress function “wp_remote_post” to make a HTTP request to your own /wp-admin/admin-ajax.php. If this is blocked by your webserver then the image can’t be automatically compressed.
Unfortunately, you might need some technical know-how to be able to check if this is actually the case for you as well, or help from your hosting support in case you use a third party hosting provider.

Thread Starter wpwebdevj
(@wpwebdevj)

4 months, 2 weeks ago

I created a bunch of example sites and I have finally been able to reproduce this error. If I create a new site without DNS records and edit my Windows hosts file with the IP address of my server in order to access the site, then the “Compress new images in the background (Recommended)” does not work.

If I create a site (same server) with proper DNS records then everything works fine.

Once the plugin is activated on a site without actual DNS records, then the “Compress new images in the background (Recommended)” will never work even if DNS records are added later and the plugin is deactivated and reactivated, API key change, etc.

I think this could be the problem on the site I’m working on now. I must have been using host file records instead of actual DNS records when I first activated the plugin.

Steps to reproduce:

1. Create a new WordPress site on example1.example.com without any DNS records. Edit your client hosts file as follows.

server_ip_address example1.example.com

2. Go to example1.example.com in your browser and complete the WP setup.

3. Add the plugin with default settings, enter API key, then try to upload an image. The image will not get compressed automatically.

Thread Starter wpwebdevj
(@wpwebdevj)

4 months, 2 weeks ago

I was playing around some more and found that the second test site did in fact start working once I made DNS records and cleared my browser cache. I disabled the DNS record for the first test site and accessed it through a hosts record and it stopped working.

So it appears there is a correlation between having a valid public DNS record and whether or not the “Compress new images in the background (Recommended)” works.

Back to my original site, it is still not working even though there is a valid DNS record. However, I found that it does work only if the image I upload results in only 1 additional image being created (thumbnail). If the dimensions are less than 300px by 300px and only 2 files need to be optimized (original and thumbnail), then it works. As soon as a dimension is greater than 300px and 3 files need to be optimized then it won’t do it automatically. I tried changing my settings to only optimize the original image and the thumbnail but it still doesn’t work for images greater than 300px.
Thread Starter wpwebdevj
(@wpwebdevj)

4 months, 2 weeks ago
After more investigating, I found some more information that might be useful. The original site I was having problems with is behind a Google Cloud Load Balancer. When I check the logs during the time I upload a new image, I am getting this error.
```
{
  "insertId": "[withheld]",
  "jsonPayload": {
    "@type": "type.googleapis.com/google.cloud.loadbalancing.type.LoadBalancerLogEntry",
    "statusDetails": "client_disconnected_before_any_response"
  },
  "httpRequest": {
    "requestMethod": "POST",
    "requestUrl": "https://[withheld]/wp-admin/admin-ajax.php",
    "requestSize": "981",
    "userAgent": "WordPress/5.5.3; https:[withheld]",
    "remoteIp": "[withheld]",
    "referer": "https://[withheld]/wp-admin/admin-ajax.php",
    "latency": "0.961313s"
  },
  "resource": {
    "type": "http_load_balancer",
    "labels": {
      "target_proxy_name": "[withheld]",
      "project_id": "[withheld]",
      "url_map_name": "[withheld]",
      "zone": "global",
      "forwarding_rule_name": "[withheld]",
      "backend_service_name": ""
    }
  },
  "timestamp": "2020-12-01T07:34:06.257620Z",
  "severity": "INFO",
  "logName": [withheld],
  "trace": [withheld],
  "receiveTimestamp": "2020-12-01T07:34:08.158481088Z",
  "spanId": "[withheld]"
}
```
No errors in the browser console or on the server logs.

When I upload a smaller image less than 300px and the automatic compression works, this is what my log entry on the load balancer looks like.
```
{
  "insertId": "[withheld]",
  "jsonPayload": {
    "statusDetails": "response_sent_by_backend",
    "@type": "type.googleapis.com/google.cloud.loadbalancing.type.LoadBalancerLogEntry"
  },
  "httpRequest": {
    "requestMethod": "POST",
    "requestUrl": "https:[withheld]/wp-admin/admin-ajax.php",
    "requestSize": "1928",
    "status": 200,
    "responseSize": "896",
    "userAgent": "WordPress/5.5.3; https://[withheld]",
    "remoteIp": "[withheld]",
    "referer": "https://[withheld]/wp-admin/admin-ajax.php",
    "serverIp": "10.168.0.2",
    "latency": "2.723672s"
  },
  "resource": {
    "type": "http_load_balancer",
    "labels": {
      "project_id": "[withheld]",
      "target_proxy_name": "[withheld]",
      "backend_service_name": "[withheld]",
      "zone": "global",
      "url_map_name": "[withheld]",
      "forwarding_rule_name": "[withheld]-forwarding-rule"
    }
  },
  "timestamp": "2020-12-01T07:29:33.981202Z",
  "severity": "INFO",
  "logName": "[withheld]",
  "trace": "[withheld]",
  "receiveTimestamp": "2020-12-01T07:29:37.504418604Z",
  "spanId": "[withheld]"
}
```
Any idea on how to troubleshoot this further?
Thread Starter wpwebdevj
(@wpwebdevj)

4 months, 2 weeks ago

Ok, final update. I changed a line of the source code in your plugin and now everything is working fine.

I changed /src/class-tiny-plugin.php line 298 from:

'timeout' => 0.01

to:

'timeout' => 10

At first I tried setting the timeout to 1. It worked for a smaller image around 20kb size but then failed on a larger 150kb image.

I guess the problem is that the AJAX call is taking longer because it’s going through my load balancer instead of happening directly on my server? Maybe it’s possible to use localhost when making the call instead of the full domain name?

Plugin Author TinyPNG
(@tinypng)

4 months, 2 weeks ago

Hi,

Thanks for all the updates! Good to hear that you managed to figure out what was going on.
By default you will indeed need valid DNS records, otherwise the plugin will not be able to reach your /wp-admin/admin-ajax.php.

If you take a look at the block of code where you changed the timeout (in the async_compress_on_upload function), you will see that if the “WORDPRESS_HOST” environment variable is set then it will use that for the POST request. You could try setting that to see if that works.

Another alternative is having a hostfile entry on your webserver for your domain pointing to localhost. That should avoid your load balancer from being used as well.
Thread Starter wpwebdevj
(@wpwebdevj)

4 months, 2 weeks ago
Thanks for your help and quick response time!

By default you will indeed need valid DNS records, otherwise the plugin will not be able to reach your /wp-admin/admin-ajax.php.

I’m just curious, is there not a better way for a plug-in to call something directly without relying on DNS records and going through CDNs?

If you take a look at the block of code where you changed the timeout (in the async_compress_on_upload function), you will see that if the “WORDPRESS_HOST” environment variable is set then it will use that for the POST request. You could try setting that to see if that works.

Do you mean setting this WORDPRESS_HOST variable in your plug-in code or in wp-config.php?

There is something that I still can’t understand. The example sites I was testing yesterday were on Cloudflare with proxy enabled and the automatic compression was working fine once the DNS records were set without modifying the plug-in code (10ms timeout).

But for the site I’m developing that is behind Google Cloud load balancer with CDN enabled it doesn’t work without increasing the timeout.

On the server hosting the Cloudflare sites, if I ping the site URL from ssh I get an average ping time of 1.02ms. If I do the same on the server hosting the Google LB CDN site I get an average ping of 0.79ms. So the Google edge CDN is actually closer to the origin server than the Cloudflare one.

Is it possible that Cloudflare is processing the request and sending it back to the origin server faster than Google is? Or could there be other factors involved? Maybe Google CDN is sending the request back fast enough but my server isn’t processing the response right away and resets the connection before reading it?
- This reply was modified 4 months, 2 weeks ago by wpwebdevj. Reason: formatting
Plugin Author TinyPNG
(@tinypng)

4 months, 2 weeks ago

Sorry for the delay! Hereby a response to your questions.

I’m just curious, is there not a better way for a plug-in to call something directly without relying on DNS records and going through CDNs?

To process images in the background we need a way that would allow that. One such way is executing a HTTP request in non-blocking mode. This uses build-in default WordPress functionality without any dependencies. There could very well be other options, but they might introduce different problems.
In order to send a POST request from WordPress to WordPress this needs to be done using the actual host, as we can’t rely on a request to localhost to be working everywhere.

Do you mean setting this WORDPRESS_HOST variable in your plug-in code or in wp-config.php?

You could set it as an environment variable on your webserver, see https://en.wikipedia.org/wiki/Environment_variable for general information on environment variables. You could definitely just edit the plugin code to test it out. Do note that if you’d opt to change anything in the plugin files directly, those changes would be overwritten when you’d update or reinstall the plugin.

Is it possible that Cloudflare is processing the request and sending it back to the origin server faster than Google is? Or could there be other factors involved? Maybe Google CDN is sending the request back fast enough but my server isn’t processing the response right away and resets the connection before reading it?

The request that we’re sending is triggering the WordPress plugin to compress an image in a new process. We’re not directly interested in the outcome of that process, so we don’t need to wait for the response of the request. We only need enough time to send a HTTP POST request. Normally, 0.01s should be enough to send a request. However, there could definitely be factors at play that cause sending out a request to take longer than 0.01s. Unfortunately, every server is unique and can pose different difficulties. Since releasing the version of the plugin with this feature this method has worked fine for the majority of users, but it’s definitely not guaranteed that it will work everywhere.

It would definitely be very interesting to get to know where the threshold lies for your use cases in regards to the timeout. The more user feedback we get, the better we can make the plugin.
Please do note that the size of your image should have no effect on whether the wp_remote_post request will succeed as the image data is not sent with that request, only the identifier of the image that needs to be compressed.

I hope that answers the questions you had!

Thread Starter wpwebdevj
(@wpwebdevj)

4 months, 2 weeks ago

It would definitely be very interesting to get to know where the threshold lies for your use cases in regards to the timeout. The more user feedback we get, the better we can make the plugin.
Please do note that the size of your image should have no effect on whether the wp_remote_post request will succeed as the image data is not sent with that request, only the identifier of the image that needs to be compressed.

That is very strange because the auto compression works consistently for very small (less than 300kb) images with the original timeout value of 0.01. Why would that make any difference? Maybe my server is delaying sending out the http request until the image uploading process finishes, by which time the timeout is already passed for images larger than 300px?

By the way, I don’t think 0.01 is a valid value for the timeout. The minimum timeout value for wp_remote_post seems to be 1 second. I think 0.01 is rounded up to 1.

I installed a plugin called Log HTTP Requests to see what happens when I upload a new image.

If the timeout setting in your plugin is 10s, then the Runtime of the admin-ajax http request is around 5.8 seconds. Auto compression works.

If the timeout setting in your plugin is 4s, then the Runtime of the admin-ajax http request is 4.0033 seconds. Auto compression still works.

If the timeout setting in your plugin is 1.5s, then the Runtime of the admin-ajax http request is 1.5028 seconds. Auto compression still works.

If the timeout setting in your plugin is 0.01s, then the Runtime of the admin-ajax http request is 1.0015 seconds. Auto compression doesn’t work on Google load balancer server but works on Cloudflare one.

Any thoughts?

Thread Starter wpwebdevj
(@wpwebdevj)

4 months, 2 weeks ago

Ok, I have finally gotten to the bottom of this.

I created 2 test sites on my same GCE server hosting the site I’m having problems with. One of the sites is proxied through Google load balancer and the other through Cloudflare.

Auto compression was working fine on both sites without changing your plugin code. Then I added SSL and the Cloudflare site still worked (with both full and flexible SSL), but the Google load balancer site didn’t work (only set up full encryption, couldn’t get non-SSL on back end connection to work).

Then I changed the back end protocol on Google load balancer from HTTP/2 to HTTPS and then magically your plugin started working again.

The problem seems to be caused by using HTTP/2 for the back-end connection between Google load balancer and the origin site. Once I changed the setting to HTTPS for the backend connection on the load balancer for my original site, that started working as well (without modifying plugin code).

I’m kind of confused because I thought HTTP/2 was supposed to be faster than HTTP/1 but it seems that is not the case here.

Anyway, I hope some of this info is useful for you.
Plugin Author TinyPNG
(@tinypng)

4 months, 2 weeks ago
Thanks a lot for the extended testing, and sharing your results! It is definitely very useful information.

In regards to your comment about the timeout value. It depends on which library is used for the connection. For cURL the minimum value is indeed 1 second, but for fsockopen you can use lower timeouts. See the documentation in wp-includes/class-requests.php in the WordPress source:
```
* - 'timeout': How long should we wait for a response?
*    Note: for cURL, a minimum of 1 second applies, as DNS resolution
*    operates at second-resolution only.
*    (float, seconds with a millisecond precision, default: 10, example: 0.01)
```

Viewing 14 replies - 1 through 14 (of 14 total)

You must be logged in to reply to this topic.

WordPress.org

Images not automatically compressed when uploaded through the block editor

Views