• normadize

    (@normadize)


    I’m not the first one to experience this. I have a normal WP 3.5.1 installation (only a handful of plugins). I have only two languages. This page is also actually very easy on translation without any calls to __(), _(), etc, other than what WP does internally in the loop — actually the page template I have has fixed language titles and meta, and no comments.

    I profiled this page with qTranslate disabled and with qTranslate enabled. The numbers speak for themselves. Yes, that is 5 (five) times slower when qTranslate is enabled!

    qTranslate DISABLED:

    https://www.dropbox.com/s/84zfwnuky8qynim/no-qT-Cumul-Time.png
    https://www.dropbox.com/s/t900cw3zbj6d5g5/No-qT-Own-Time.png
    https://www.dropbox.com/s/uk3miie2tiyfvov/No-qT-Calls.png

    qTranslate ENABLED:

    https://www.dropbox.com/s/0zoatuyzu5682to/qT-Cumul-Time.png
    https://www.dropbox.com/s/q8rwrfs2nsde646/qT-Own-Time.png
    https://www.dropbox.com/s/8o871hedu4zcr31/qT-Calls.png

    The time is in milliseconds … 1 second vs 5 seconds page load time.

    You can see the ridiculously high number of calls to functions such as qtrans_isEnabled(), qtrans_use(), preg_replace(), in_array(), etc. A lot of these are unnecessary, and most are expensive. You don’t need preg_replace() all the time. If you absolutely have to use it and must call it so many thousands of times, using the same regex many times, then make sure you analyze and precompile the regex (use the S modifier) so that it doesn’t have to to compile it on every call. Don’t use preg_replace() when you can use str_replace(). Don’t use recursive functions if you can do it in a for/foreach loop. And so on and so forth.

    There are so many inefficient aspects to the qTranslate code causing it to be so slow that it is becomes a real pain to actually fix … I wanted to but gave up realizing I lack the time. I like qTranslate’s idea of storing all languages in the same post (although it could be improved) but I’m seriously thinking of giving up on it because
    – of how slow it is
    – the lack of support and replies from the author

    Is the author still around? Could he please at least reply to this and maybe state whether he’s going to consider improving the code?

    Cheers.

    p.s. Did I mention the ghastly function names, e.g. qtrans_useCurrentLanguageIfNotFoundUseDefaultLanguage() ?

    http://wordpress.org/extend/plugins/qtranslate/

Viewing 15 replies - 1 through 15 (of 36 total)
  • Thread Starter normadize

    (@normadize)

    I’m going to use this thread to post code optimizations that I do to qTranslate for anyone who is interested.

    Thread Starter normadize

    (@normadize)

    It’s not actually necessary to seek translations for ALL the options in the wp_options table. Most of them are only used internally and do not require translation so the whole process of seeking a translation can be cut down.

    Many WP installations have a huge wp_options table due to a large number of plugins — it doesn’t matter if they are disabled or not, their options are still there in the table. Most of them ‘forget’ their options in the table even if you uninstall them so if you had previously tried a large number of plugins in the past and uninstalled most of them, your website can get increasingly slow if qTranslate is enabled due to the code below. My wp_options table has 491 rows!

    Commenting out the following block in qtranslate_core.php makes the website a hell of a lot faster for me:

    /*
    if(!defined('WP_ADMIN')) {
    	$alloptions = wp_load_alloptions();
    	foreach($alloptions as $option => $value) {
    		add_filter('option_'.$option, 'qtrans_useCurrentLanguageIfNotFoundUseDefaultLanguage',0);
    	}
    }
    */

    With this tweak, the page from the initial post loads in under 1.5 seconds now, down from 5.5 seconds!

    In case you have options that need to be translated, then you can manually hook the qtranslate filter above only for those options, rather than hooking into all options, e.g. I would replace the above code with

    $options = array('options','that','need','translation','here');
    foreach ($options as $opt)
        add_filter('option_'.$option, 'qtrans_useCurrentLanguageIfNotFoundUseDefaultLanguage',0);

    And you can add that into your functions.php.

    Thread Starter normadize

    (@normadize)

    You can also make the qtrans_parseURL() function faster, in qtranslate_utils.php. This function is called quite a few times:

    function qtrans_parseURL($url) {
        $r  = '!(?:(\w+)://)?(?:(\w+)\:(\w+)@)?([^/:]+)?';
        $r .= '(?:\:(\d*))?([^#?]+)?(?:\?([^#]+))?(?:#(.+$))?!i';
    
        preg_match ( $r, $url, $out );
        $result = @array(
            "scheme" => $out[1],
            "host" => $out[4].(($out[5]=='')?'':':'.$out[5]),
            "user" => $out[2],
            "pass" => $out[3],
            "path" => $out[6],
            "query" => $out[7],
            "fragment" => $out[8]
            );
        return $result;
    }

    Even though the above could be sped up by using the S PCRE modifier to analyze the regex for future calls, you can replace the entire function with the much faster:

    function qtrans_parseURL($url) {
            $result = parse_url($url) + array(
                    'scheme' => '',
                    'host' => '',
                    'user' => '',
                    'pass' => '',
                    'path' => '',
                    'query' => '',
                    'fragment' => ''
            );
            isset($result['port'])
                    and $result['host'] .= ':'. $result['port'];
            return $result;
    }

    note the + array union operator which basically implements default array values since parse_url() populates those array fields only if a match was found in the url.

    Thread Starter normadize

    (@normadize)

    Looking through the code, I’m cringing at the sight of stuff like this:

    elseif(preg_match("#^<!--:-->$#ism", $block, $matches)) {
       $current_language = "";
       continue;
    }

    The above is a clear example of what I was talking about. Not only $matches is not used anywhere, so preg_match() shouldn’t have been asked to create and populate a new array, and not only are the s, m and i modifiers not needed, but the entire if clause could have been replaced with just:

    elseif($block == '<!--:-->'))

    There are too many unnecessary calls to preg_match() and preg_replace(), a lot of them shouldn’t even be case insensitive, or multi-line … strpos(), stripos(), str_replace(), str_ireplace() should be used when possible, or plain comparison like above.

    Also, there are too many for/foreach loops triggering subsequent large number of calls to preg_match/preg_replace. It’s usually better, if possible, to try and call preg_* beforehand on the big string, rather than split the string and then call preg_* a lot of times.

    db9429

    (@db9429)

    Hey normadize

    I just wanted to stop by and thank you for this thread. I’m having issues with the plugin too: it’s taking up to 60 seconds to load with qTranslate, and about 4 seconds without.

    Anyway, I’ve bookmarked this page, and will continue to watch.

    Thanks again

    db9429

    Fred

    (@wp_351_user)

    Thank you for your help
    I will post some improvements too.
    Mainly regarding the way to store the tag translation.
    Change 35mo into 48ko…

    Fred

    (@wp_351_user)

    Hi
    I have developed some code to not store the tag translation into the wp_options table option_name qtranslate_term_name field.
    It can be improved or stored into the term_meta table, but for my need and with the time I had, I did that.
    Basically, it turns 35Mo of serialized array into a 48ko table content.
    Here is the related thread http://www.qianqin.de/qtranslate/forum/viewtopic.php?f=3&t=4073
    You can remove the unique Key, I did it because I need one language.

    CREATE TABLE IF NOT EXISTS wp_qtranslate_terms_lang (
      term_id int(11) NOT NULL,
      term_meta varchar(32) NOT NULL,
      term_value varchar(150) NOT NULL,
      UNIQUE KEY term_id (<code>term_id</code>)
    ) ENGINE=MyISAM DEFAULT CHARSET=latin1;

    Please visit the thread to get the files I updated. Some hardcoding but it can serve as a basis to improve the rest.

    Frederic

    qtranslate_core.php
    – qtrans_saveConfig()
    – qtrans_updateTermLibrary()
    – qtrans_useTermLib($obj)

    qtranslate_wphacks.php
    – qtrans_insertTermInput($id,$name,$term,$language)
    – qtrans_insertTermInput2($id,$name,$term,$language)

    Fred

    (@wp_351_user)

    Fred

    (@wp_351_user)

    let me know if it works for you

    // splits text with language tags into array
    function qtrans_split($text, $quicktags = true) {
    	global $q_config;
    
    	//init vars
    	$split_regex = "#(<!--[^-]*-->|\[:[a-z]{2}\])#ism";
    	$current_language = "";
    	$result = array();
    	foreach($q_config['enabled_languages'] as $language) {
    		$result[$language] = "";
    	}
    
    	// split text at all xml comments
    	$blocks = preg_split($split_regex, $text, -1, PREG_SPLIT_NO_EMPTY|PREG_SPLIT_DELIM_CAPTURE);
    	foreach($blocks as $block) {
    		# detect language tags
    		// Frederic
    		// if(preg_match("#^<!--:([a-z]{2})-->$#ism", $block, $matches)) {
    		$matches = array();
    		foreach($q_config['enabled_languages'] as $language) {
    			if($quicktags) { if(strpos($block, "[:".$language."]") !== false) { $matches[1] = $language; } }
    			else {
    				if(strpos($block, "<!--:".$language."-->") !== false) { $matches[1] = $language; }
    			}
    		}
    		if(isset($matches[1]) && $matches[1] != '') {
    			if(qtrans_isEnabled($matches[1])) {
    				$current_language = $matches[1];
    			} else {
    				$current_language = "invalid";
    			}
    			continue;
    		// detect quicktags
    		/*} elseif($quicktags && preg_match("#^\[:([a-z]{2})\]$#ism", $block, $matches)) {
    			if(qtrans_isEnabled($matches[1])) {
    				$current_language = $matches[1];
    			} else {
    				$current_language = "invalid";
    			}
    			continue;
    		// detect ending tags
    		 */
    		// Frederic preg_match("#^<!--:-->$#ism", $block, $matches))
    		} elseif($block == '<!--:-->') {
    			$current_language = "";
    			continue;
    		// detect defective more tag
    		// Frederic
    		// } elseif(preg_match("#^<!--more-->$#ism", $block, $matches)) {
    		} elseif(strpos($block, "<!--more-->") !== false) {
    			foreach($q_config['enabled_languages'] as $language) {
    				$result[$language] .= $block;
    			}
    			continue;
    		}
    		// correctly categorize text block
    		if($current_language == "") {
    			// general block, add to all languages
    			foreach($q_config['enabled_languages'] as $language) {
    				$result[$language] .= $block;
    			}
    		} elseif($current_language != "invalid") {
    			// specific block, only add to active language
    			$result[$current_language] .= $block;
    		}
    	}
    	foreach($result as $lang => $lang_content) {
    		$result[$lang] = preg_replace("#(<!--more-->|<!--nextpage-->)+$#ism","",$lang_content);
    	}
    	return $result;
    }
    Thread Starter normadize

    (@normadize)

    I also optimized qtrans_split() and qtrans_use() myself … but the whole thing needs to be rewritten.

    The plugin author is not replying to anyone anymore. Maybe it’s time to move on.

    Fred

    (@wp_351_user)

    I try to see what I can do.

    replace in previews post

    foreach($q_config['enabled_languages'] as $language) {
    			if(strpos($block, "<!--:".$language."-->") !== false) { $matches[1] = $language; }
    			if($quicktags && strpos($block, "[:".$language."]") !== false) { $matches[1] = $language; }
    		}
    Thread Starter normadize

    (@normadize)

    A first (not-complete) optimization for qtrans_use() which calls itself unnecessarily many times is as follows. Replace the lines:

    global $q_config;
    // return full string if language is not enabled
    if(!qtrans_isEnabled($lang)) return $text;
    if(is_array($text)) {
        // handle arrays recursively
        foreach($text as $key => $t) {
            $text[$key] = qtrans_use($lang,$text[$key],$show_available);
        }
        return $text;
    }
    
    if (is_object($text)||@get_class($text) == '__PHP_Incomplete_Class') {
        foreach(get_object_vars($text) as $key => $t) {
            $text->$key = qtrans_use($lang,$text->$key,$show_available);
        }
        return $text;
    }
    
    // prevent filtering weird data types and save some resources
    if(!is_string($text) || $text == '') {
        return $text;
    }

    with

    if (empty($text) || !is_string($text) || !preg_match($re = '/<!--:[a-z]{2}-->/', $text) || !qtrans_isEnabled($lang))
            return $text;
    
    if (is_array($text) || $text instanceof __PHP_Incomplete_Class) {
        foreach ($text as &$t)
            if ($t && is_string($t) && preg_match($re, $t))
                $t = qtrans_use($lang, $t, $show_available);
        return $text;
    }
    
    global $q_config;

    Note the & in the foreach loop.

    Thread Starter normadize

    (@normadize)

    Actually, most of the above is not needed. In fact, the second if (is_array() ... is not even reached if $text is an array or object, which is Ok in most cases, see below. You could just as well remove it altogether.

    The qtrans_postsFilter() function is hooked into ‘the_posts’ filter. This function is:

    function qtrans_postsFilter($posts) {
    	if(is_array($posts)) {
    		foreach($posts as $post) {
    			$post->post_content = qtrans_useCurrentLanguageIfNotFoundShowAvailable($post->post_content);
    			$post = qtrans_useCurrentLanguageIfNotFoundUseDefaultLanguage($post);
    		}
    	}
    	return $posts;
    }

    Note how it is passing each object $post in a loop to qtrans_useCurrentLanguageIfNotFoundUseDefaultLanguage($post) which in turn calls qtrans_use() on each of the object’s properties. Only the title and content (and maybe guid and pinged) properties require translation (none of the other 20+ do) and for those, qTranslate already hooks into ‘the_content’, ‘the_title’ etc filters. This function can be useful if you call get_posts() in your own code.

    You could remove the ‘the_posts’ hook from qtranslate_hooks.php and if you need to call get_posts() in your own code and want the title and content translated, then you can either call qtrans_postsFilter(), which is very inefficient (and you may not want the default behaviour of not showing the default language for the content if the current language is unavailable), of just translate the title and content in a loop — you can have your own function for that.

    Fred

    (@wp_351_user)

    Thank you normadize.
    I`m working currently on it.
    Patch, hardcode etc, and still slow.
    I continue.
    Also qtranslate slug can maybe be improved.
    I check qtranslate currently.
    Thanks

    Thread Starter normadize

    (@normadize)

    To keep current functionality but with a faster, less-recursive function, replace in qtrans_use() the following:

    global $q_config;
    // return full string if language is not enabled
    if(!qtrans_isEnabled($lang)) return $text;
    if(is_array($text)) {
        // handle arrays recursively
        foreach($text as $key => $t) {
            $text[$key] = qtrans_use($lang,$text[$key],$show_available);
        }
        return $text;
    }
    
    if (is_object($text)||@get_class($text) == '__PHP_Incomplete_Class') {
        foreach(get_object_vars($text) as $key => $t) {
            $text->$key = qtrans_use($lang,$text->$key,$show_available);
        }
        return $text;
    }
    
    // prevent filtering weird data types and save some resources
    if(!is_string($text) || $text == '') {
        return $text;
    }

    with the following

    if (empty($text) || !qtrans_isEnabled($lang))
        return $text;
    
    $re = '/<!--:[a-z]{2}-->/';
    if (is_string($text) && !preg_match($re, $text))
        return $text;
    
    if (is_array($text) || is_object($text)) {
        foreach ($text as &$t)
            if ($t && ((is_string($t) && preg_match($re, $t)) || is_array($t) || is_object($t)))
                $t = qtrans_use($lang, $t, $show_available);
        return $text;
    }
    
    global $q_config;
Viewing 15 replies - 1 through 15 (of 36 total)
  • The topic ‘evidence on how *slow* and inefficient qTranslate is … please do something’ is closed to new replies.