Viewing media in grid mode is unresponsive/very slow

tuanmh
(@tuanmh)

9 years, 2 months ago
Hi,

We got a problem in viewing media with grid mode. It takes very long to load all the images and their information, usually around 24 – 30 seconds on Google App Engine and Google Cloud Storage. Disabling this plugin can reduce the time to 2-3 seconds. Debugging showed that the bottleneck is at line 329, modules/uploads.php. Executing this line usually takes 0.5 – 0.8 seconds.
```
$file = get_attached_file( $id );
```
It is quite strange that get_attached_file with the same ID has been called before this line and is very fast. Tracing deep down to the get_attached_file function, which calls wp_upload_dir, in which wp_mkdir_p will be invoked. In wp_mkdir_p, we recognised that is_dir call is actually the root cause, accounting for most of the time taken. It seems this bug has something to do the underlying implementation of is_dir in Php Google App Engine. We thought there must be some caching mechanism behind it.

Have anyone encountered this? Do you have any suggestions or ideas that could be taken? At the moment, we use a hacky solution by adding is_writable check along with is_dir as following. It turned out that is_writable is far more faster than is_dir under Google App Engine.
```
// Workaround for GCS.
// as is_dir consumes too much time in Google App Engine and Cloud Storage
// we check if the directory is writable or not.
if ( file_exists( $target ) )
	return is_writable($target) || @is_dir( $target );
```
It is also a question for WordPress. It seems very inefficient in calling wp_mkdir_p every time we call get_attached_file. is_dir may be very fast in dealing with local environments but working with remote Cloud Storage would incur high latency.

Thanks!
https://wordpress.org/plugins/google-app-engine/

Viewing 6 replies - 1 through 6 (of 6 total)

aidanlane
(@aidanlane)

9 years, 2 months ago

Hi,

Is anyone able to help with this?

Here’s some more debugging, which illustrates the problem:

wp-includes/functions.php:

[moderated – that’s too much code to post in these forums – please read the forum guidelines]

WPyogi
(@wpyogi)

9 years, 2 months ago

@aidanlane – please don’t hijack someone else’s thread? Please read over the forum guidelines and also see the part about posting code – you need to start your own thread and use a pastebin for that much code.

http://codex.wordpress.org/Forum_Welcome#Where_To_Post
aidanlane
(@aidanlane)

9 years, 2 months ago
Apologies, @wpyogi, I should have stated that @tuanmh works for me- that is for the same issue on the same project.

I have reduced the code down…

wp-includes/functions.php:
```
function wp_mkdir_p( $target ) {
	syslog(LOG_INFO, "wp_mkdir_p point A for $target");
	$wrapper = null;

	…

	// Make sure we have an uploads directory.
	syslog(LOG_INFO, "wp_mkdir_p point B for $target");
	if ( file_exists( $target ) ) {
		syslog(LOG_INFO, "wp_mkdir_p point C for $target");
		$is_dir = @is_dir($target);
		syslog(LOG_INFO, "wp_mkdir_p point D for $target");
		return $is_dir;
	}
...
```
Log output:

I 16:18:40.141 2015-02-04 200 142.71 KB 22427ms /wp-admin/admin-ajax.php
… – – [03/Feb/2015:21:18:40 -0800] “POST /wp-admin/admin-ajax.php HTTP/1.1” 200 146134 – “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.94 Safari/537.36” “…” ms=22427 cpu_ms=10850 … app_engine_release=1.9.17
I 16:18:18.576 wp_mkdir_p point A for gs://our-bucket
I 16:18:18.576 wp_mkdir_p point B for gs://our-bucket
I 16:18:19.160 wp_mkdir_p point C for gs://our-bucket
I 16:18:19.160 wp_mkdir_p point D for gs://our-bucket
I 16:18:19.195 wp_mkdir_p point A for gs://our-bucket
I 16:18:19.195 wp_mkdir_p point B for gs://our-bucket
I 16:18:19.195 wp_mkdir_p point C for gs://our-bucket
I 16:18:19.195 wp_mkdir_p point D for gs://our-bucket
I 16:18:19.285 wp_mkdir_p point A for gs://our-bucket
I 16:18:19.285 wp_mkdir_p point B for gs://our-bucket
I 16:18:19.595 wp_mkdir_p point C for gs://our-bucket
I 16:18:19.595 wp_mkdir_p point D for gs://our-bucket
aidanlane
(@aidanlane)

9 years, 2 months ago

Note that the “Add Media” for posts is also effected by this issue.

Plugin Author slangley
(@slangley)

9 years, 2 months ago

We’ve add a per request stat cache for GCS objects that will should resolve this. This will roll out in the 1.9.19 release of app engine.

PHP does include a limited stat cache for when stating objects using stream wrappers, but it only remembers the last item stat’d. In this case as WP decides to stat different objects in a loop that cache is useless.

aidanlane
(@aidanlane)

9 years, 1 month ago

Thanks for that @slangley.