Last-Modified header is changing frequently

Resolved Stelios
(@invader7)

4 years, 8 months ago
Hello, thanks for the great plugin.

I’m trying to understand if my W3 configuration is causing the below issue or if I have to look somewhere else in my setup.

I have NGINX with PHP FPM, CloudFlare and W3 Total Cache.

Without CloudFlare / W3 I see response times around 800-900ms.
With CloudFlare / W3 I see response times around 100ms.

Sporadically, I see spikes up to 800-900ms and I’m trying to understand what is causing this.

Inspecting the network tab of my browser I see that most of the requests for “/” are 304 Not Modified and I have some 200 which are causing those spikes. Checking the response headers I found that all 304 have the same headers (as expected):

ETag “87e48f504efbae05cb079f3ece95da44”
Last-Modified Sat, 06 Nov 2021 13:32:20 GMT
Expires Sat, 06 Nov 2021 14:32:20 GMT

The subsequent 200 (1st after the last 304) has the following:

ETag “87e48f504efbae05cb079f3ece95da44”
Last-Modified Sat, 06 Nov 2021 13:33:14 GMT
Expires Sat, 06 Nov 2021 14:33:14 GMT

The Last-Modified header is changing and this is happening every 2-3 minutes. Comparing the response payload, I see no difference in the content and I’m wondering why/how the Last-Modified header is changing. I believe that’s the reason for the spikes in my response times.

Do you have any idea if this could be related with W3 ?

Thanks !
- This topic was modified 4 years, 8 months ago by Stelios.
The page I need help with: [log in to see the link]

Viewing 9 replies - 1 through 9 (of 9 total)

Plugin Contributor Marko Vasiljevic
(@vmarko)

4 years, 8 months ago

Hello @invader7

Thank you for reaching out and I am happy to assist you with this.
Can you please share which Page Cache caching method you are using and also, have you set the Cloudflare to respect the existing Browser Cache headers?
Thank you!

Thread Starter Stelios
(@invader7)

4 years, 8 months ago

Thank you for your response Marko. Really appreciated.

I’m using Memcached.

Regarding “have you set the Cloudflare to respect the existing Browser Cache headers?” do you mean creating a Page Rule with “Origin Cache Control” enabled? If this is what you mean, no I don’t use this, I would say I have the default configuration.

My “/” page is always returning CF-Cache-Status: DYNAMIC. Even 304 responses in “/” are returning DYNAMIC https://developers.cloudflare.com/cache/about/default-cache-behavior

I think that the issue is not related with CloudFlare because I’m bypassing CloudFlare and I’m calling my origin server directly with cURL and still the Last-Modified header is changing every 2-3-4 minutes.

Thanks

Plugin Contributor Marko Vasiljevic
(@vmarko)

4 years, 8 months ago

Hello @invader7

Thank you for the information.
Can you please share an example of the page you are referring to and the one that has 304?
THanks!
Thread Starter Stelios
(@invader7)

4 years, 8 months ago
Hi Marko,

Sure, the URL that I’m referring too is the main page of https://www.radioevros.gr/

Below you can see 4 requests within 2-3 minutes, ordered.

Request #2 is different than others, #1, #3 and #4 have the same ETag header but Last-Modified and Expires headers are different.

Requests are bypassing CloudFlare, they are send directly to origin server.

Thanks.

#1 curl https://www.radioevros.gr/ -I
```
HTTP/1.1 200 OK
Date: Thu, 11 Nov 2021 23:49:51 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Vary: Accept-Encoding
Link: <https://www.radioevros.gr/wp-json/>; rel="https://api.w.org/"
Link: <https://www.radioevros.gr/wp-json/wp/v2/pages/36189>; rel="alternate"; type="application/json"
Link: <https://www.radioevros.gr/>; rel=shortlink
Last-Modified: Thu, 11 Nov 2021 23:49:50 GMT
Expires: Fri, 12 Nov 2021 00:49:50 GMT
Pragma: public
Cache-Control: max-age=3599, public
ETag: "c4a3003d0e6daaa00a84117a75379b06"
Link: <https://www.radioevros.gr/wp-json/>; rel="https://api.w.org/"
Link: <https://www.radioevros.gr/wp-json/wp/v2/pages/36189>; rel="alternate"; type="application/json"
Link: <https://www.radioevros.gr/>; rel=shortlink
```
#2 curl https://www.radioevros.gr/ -I
```
HTTP/1.1 200 OK
Date: Thu, 11 Nov 2021 23:50:05 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Vary: Accept-Encoding
Link: <https://www.radioevros.gr/wp-json/>; rel="https://api.w.org/"
Link: <https://www.radioevros.gr/wp-json/wp/v2/pages/36189>; rel="alternate"; type="application/json"
Link: <https://www.radioevros.gr/>; rel=shortlink
```
#3 curl https://www.radioevros.gr/ -I
```
HTTP/1.1 200 OK
Date: Thu, 11 Nov 2021 23:51:44 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Vary: Accept-Encoding
Link: <https://www.radioevros.gr/wp-json/>; rel="https://api.w.org/"
Link: <https://www.radioevros.gr/wp-json/wp/v2/pages/36189>; rel="alternate"; type="application/json"
Link: <https://www.radioevros.gr/>; rel=shortlink
Last-Modified: Thu, 11 Nov 2021 23:51:40 GMT
Expires: Fri, 12 Nov 2021 00:51:40 GMT
Pragma: public
Cache-Control: max-age=3596, public
ETag: "c4a3003d0e6daaa00a84117a75379b06"
Link: <https://www.radioevros.gr/wp-json/>; rel="https://api.w.org/"
Link: <https://www.radioevros.gr/wp-json/wp/v2/pages/36189>; rel="alternate"; type="application/json"
Link: <https://www.radioevros.gr/>; rel=shortlink
```
#4 curl https://www.radioevros.gr/ -I
```
HTTP/1.1 200 OK
Date: Thu, 11 Nov 2021 23:52:31 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Vary: Accept-Encoding
Link: <https://www.radioevros.gr/wp-json/>; rel="https://api.w.org/"
Link: <https://www.radioevros.gr/wp-json/wp/v2/pages/36189>; rel="alternate"; type="application/json"
Link: <https://www.radioevros.gr/>; rel=shortlink
Last-Modified: Thu, 11 Nov 2021 23:52:10 GMT
Expires: Fri, 12 Nov 2021 00:52:10 GMT
Pragma: public
Cache-Control: max-age=3579, public
ETag: "c4a3003d0e6daaa00a84117a75379b06"
Link: <https://www.radioevros.gr/wp-json/>; rel="https://api.w.org/"
Link: <https://www.radioevros.gr/wp-json/wp/v2/pages/36189>; rel="alternate"; type="application/json"
Link: <https://www.radioevros.gr/>; rel=shortlink
```
- This reply was modified 4 years, 8 months ago by Stelios.
- This reply was modified 4 years, 8 months ago by Stelios.
Plugin Contributor Marko Vasiljevic
(@vmarko)

4 years, 8 months ago

Hello @invader7

Thank you for the information.
Can you please disable the Cache-control header in Performance>Browser Cache, save all settings and purge the cache, and see how the last-modified header behaves.
Thanks!
Thread Starter Stelios
(@invader7)

4 years, 8 months ago
Hi Marko,

I disabled “Set cache control header” from the General section in Performance>Browser Cache. I saved and purged the cache. The issue persists.

I believe it could be related to Nginx or Memcached. Testing in my localhost with Set cache control header enabled and Apache instead of Nginx and Disk caching instead of Memcached seems to work as expected (Last-Modified & Expires do not change).

Thank you.
- This reply was modified 4 years, 8 months ago by Stelios.
- This reply was modified 4 years, 8 months ago by Stelios.
Thread Starter Stelios
(@invader7)

4 years, 8 months ago
I found something interesting which I was missing.

Reading your wiki https://github.com/W3EDGE/w3-total-cache/wiki/FAQ:-Installation:-Nginx-Web-Server#configuration I saw that I didn’t have the following line in my nginx config:

include /var/www/vhosts/yoursite.com/nginx.conf;

Adding this I see that the changes are reflected because now I can see the modified Cache Control header which I lowered to 120 seconds, so this works.

Cache-Control max-age=120, public

I believe you should highlight this more in the Browser Cache page just to make sure that people are aware that this is required, this is an administrative task outside wordpress / w3.

The issue still persist even after this change.

It is not clear to me why some responses have obsolete HTTP headers, as you can see below within 5 seconds I get 2 different responses and the 1st is missing Last-Modified and Expires.

Thanks!
```
curl https://www.radioevros.gr/ -I

HTTP/1.1 200 OK
Date: Sat, 13 Nov 2021 13:48:15 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Vary: Accept-Encoding
Link: <https://www.radioevros.gr/wp-json/>; rel="https://api.w.org/"
Link: <https://www.radioevros.gr/wp-json/wp/v2/pages/36189>; rel="alternate"; type="application/json"
Link: <https://www.radioevros.gr/>; rel=shortlink
Referrer-Policy: no-referrer-when-downgrade
```
```
curl https://www.radioevros.gr/ -I

HTTP/1.1 200 OK
Date: Sat, 13 Nov 2021 13:48:21 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Vary: Accept-Encoding
Link: <https://www.radioevros.gr/wp-json/>; rel="https://api.w.org/"
Link: <https://www.radioevros.gr/wp-json/wp/v2/pages/36189>; rel="alternate"; type="application/json"
Link: <https://www.radioevros.gr/>; rel=shortlink
Last-Modified: Sat, 13 Nov 2021 13:48:20 GMT
Expires: Sat, 13 Nov 2021 13:50:20 GMT
Pragma: public
Cache-Control: max-age=120, public
ETag: "2ff2496d2a04cf5adf8ffa0b79ceefa6"
Link: <https://www.radioevros.gr/wp-json/>; rel="https://api.w.org/"
Link: <https://www.radioevros.gr/wp-json/wp/v2/pages/36189>; rel="alternate"; type="application/json"
Link: <https://www.radioevros.gr/>; rel=shortlink
Referrer-Policy: no-referrer-when-downgrade
```
- This reply was modified 4 years, 8 months ago by Stelios.
- This reply was modified 4 years, 8 months ago by Stelios.
- This reply was modified 4 years, 8 months ago by Stelios.
- This reply was modified 4 years, 8 months ago by Stelios.
- This reply was modified 4 years, 8 months ago by Stelios.
Thread Starter Stelios
(@invader7)

4 years, 8 months ago
Hello Marko,

I’m sorry for multiple comments but I’m digging deeper here and I want to log everything in this thread for me and anyone else who might experience something like this in the future.

I enabled Debug logging and I found in the pagecache log file the following entries
```
[Sun, 14 Nov 2021 21:32:43 +0000] [/] [-] start, can_cache: true, reject reason: 
[Sun, 14 Nov 2021 21:32:43 +0000] [/] [-] no cache entry for www.radioevros.gr/ 55275e17cac96015371b0e03b7116732_ssl

[Sun, 14 Nov 2021 21:32:48 +0000] [/] [-] start, can_cache: true, reject reason: 
[Sun, 14 Nov 2021 21:32:48 +0000] [/] [-] no cache entry for www.radioevros.gr/ 55275e17cac96015371b0e03b7116732_ssl

[Sun, 14 Nov 2021 21:32:50 +0000] [/] [-] start, can_cache: true, reject reason: 
[Sun, 14 Nov 2021 21:32:50 +0000] [/] [-] no cache entry for www.radioevros.gr/ 55275e17cac96015371b0e03b7116732_ssl

[Sun, 14 Nov 2021 21:33:10 +0000] [/] [-] start, can_cache: true, reject reason: 
[Sun, 14 Nov 2021 21:33:10 +0000] [/] [-] no cache entry for www.radioevros.gr/ 55275e17cac96015371b0e03b7116732_ssl
```
It seems that the key is not found in my cache engine (Memcached) and as a result a new version is served. Each time the same key is calculated. I will dig deeper to find why this is happening and will update the thread.

Thanks
- This reply was modified 4 years, 8 months ago by Stelios.
Plugin Contributor Marko Vasiljevic
(@vmarko)

4 years, 8 months ago

Hello @invader7

The Last-modified is set by wp/PHP itself for PHP-generated pages and not managed by W3 Total Cache.
One way to change it is to use page cache like disk-enhanced. In that case, it will be set by the webserver to file the last modified date and you’ll get what you want
Thanks!

Viewing 9 replies - 1 through 9 (of 9 total)

The topic ‘Last-Modified header is changing frequently’ is closed to new replies.