WordPress.org

Ready to get started?Download WordPress

Forums

qTranslate
[closed] evidence on how *slow* and inefficient qTranslate is ... please do something (37 posts)

  1. normadize
    Member
    Posted 1 year ago #

    I'm not the first one to experience this. I have a normal WP 3.5.1 installation (only a handful of plugins). I have only two languages. This page is also actually very easy on translation without any calls to __(), _(), etc, other than what WP does internally in the loop -- actually the page template I have has fixed language titles and meta, and no comments.

    I profiled this page with qTranslate disabled and with qTranslate enabled. The numbers speak for themselves. Yes, that is 5 (five) times slower when qTranslate is enabled!

    qTranslate DISABLED:

    https://www.dropbox.com/s/84zfwnuky8qynim/no-qT-Cumul-Time.png
    https://www.dropbox.com/s/t900cw3zbj6d5g5/No-qT-Own-Time.png
    https://www.dropbox.com/s/uk3miie2tiyfvov/No-qT-Calls.png

    qTranslate ENABLED:

    https://www.dropbox.com/s/0zoatuyzu5682to/qT-Cumul-Time.png
    https://www.dropbox.com/s/q8rwrfs2nsde646/qT-Own-Time.png
    https://www.dropbox.com/s/8o871hedu4zcr31/qT-Calls.png

    The time is in milliseconds ... 1 second vs 5 seconds page load time.

    You can see the ridiculously high number of calls to functions such as qtrans_isEnabled(), qtrans_use(), preg_replace(), in_array(), etc. A lot of these are unnecessary, and most are expensive. You don't need preg_replace() all the time. If you absolutely have to use it and must call it so many thousands of times, using the same regex many times, then make sure you analyze and precompile the regex (use the S modifier) so that it doesn't have to to compile it on every call. Don't use preg_replace() when you can use str_replace(). Don't use recursive functions if you can do it in a for/foreach loop. And so on and so forth.

    There are so many inefficient aspects to the qTranslate code causing it to be so slow that it is becomes a real pain to actually fix ... I wanted to but gave up realizing I lack the time. I like qTranslate's idea of storing all languages in the same post (although it could be improved) but I'm seriously thinking of giving up on it because
    - of how slow it is
    - the lack of support and replies from the author

    Is the author still around? Could he please at least reply to this and maybe state whether he's going to consider improving the code?

    Cheers.

    p.s. Did I mention the ghastly function names, e.g. qtrans_useCurrentLanguageIfNotFoundUseDefaultLanguage() ?

    http://wordpress.org/extend/plugins/qtranslate/

  2. normadize
    Member
    Posted 1 year ago #

    I'm going to use this thread to post code optimizations that I do to qTranslate for anyone who is interested.

  3. normadize
    Member
    Posted 1 year ago #

    It's not actually necessary to seek translations for ALL the options in the wp_options table. Most of them are only used internally and do not require translation so the whole process of seeking a translation can be cut down.

    Many WP installations have a huge wp_options table due to a large number of plugins -- it doesn't matter if they are disabled or not, their options are still there in the table. Most of them 'forget' their options in the table even if you uninstall them so if you had previously tried a large number of plugins in the past and uninstalled most of them, your website can get increasingly slow if qTranslate is enabled due to the code below. My wp_options table has 491 rows!

    Commenting out the following block in qtranslate_core.php makes the website a hell of a lot faster for me:

    /*
    if(!defined('WP_ADMIN')) {
    	$alloptions = wp_load_alloptions();
    	foreach($alloptions as $option => $value) {
    		add_filter('option_'.$option, 'qtrans_useCurrentLanguageIfNotFoundUseDefaultLanguage',0);
    	}
    }
    */

    With this tweak, the page from the initial post loads in under 1.5 seconds now, down from 5.5 seconds!

    In case you have options that need to be translated, then you can manually hook the qtranslate filter above only for those options, rather than hooking into all options, e.g. I would replace the above code with

    $options = array('options','that','need','translation','here');
    foreach ($options as $opt)
        add_filter('option_'.$option, 'qtrans_useCurrentLanguageIfNotFoundUseDefaultLanguage',0);

    And you can add that into your functions.php.

  4. normadize
    Member
    Posted 1 year ago #

    You can also make the qtrans_parseURL() function faster, in qtranslate_utils.php. This function is called quite a few times:

    function qtrans_parseURL($url) {
        $r  = '!(?:(\w+)://)?(?:(\w+)\:(\w+)@)?([^/:]+)?';
        $r .= '(?:\:(\d*))?([^#?]+)?(?:\?([^#]+))?(?:#(.+$))?!i';
    
        preg_match ( $r, $url, $out );
        $result = @array(
            "scheme" => $out[1],
            "host" => $out[4].(($out[5]=='')?'':':'.$out[5]),
            "user" => $out[2],
            "pass" => $out[3],
            "path" => $out[6],
            "query" => $out[7],
            "fragment" => $out[8]
            );
        return $result;
    }

    Even though the above could be sped up by using the S PCRE modifier to analyze the regex for future calls, you can replace the entire function with the much faster:

    function qtrans_parseURL($url) {
            $result = parse_url($url) + array(
                    'scheme' => '',
                    'host' => '',
                    'user' => '',
                    'pass' => '',
                    'path' => '',
                    'query' => '',
                    'fragment' => ''
            );
            isset($result['port'])
                    and $result['host'] .= ':'. $result['port'];
            return $result;
    }

    note the + array union operator which basically implements default array values since parse_url() populates those array fields only if a match was found in the url.

  5. normadize
    Member
    Posted 1 year ago #

    Looking through the code, I'm cringing at the sight of stuff like this:

    elseif(preg_match("#^<!--:-->$#ism", $block, $matches)) {
       $current_language = "";
       continue;
    }

    The above is a clear example of what I was talking about. Not only $matches is not used anywhere, so preg_match() shouldn't have been asked to create and populate a new array, and not only are the s, m and i modifiers not needed, but the entire if clause could have been replaced with just:

    elseif($block == '<!--:-->'))

    There are too many unnecessary calls to preg_match() and preg_replace(), a lot of them shouldn't even be case insensitive, or multi-line ... strpos(), stripos(), str_replace(), str_ireplace() should be used when possible, or plain comparison like above.

    Also, there are too many for/foreach loops triggering subsequent large number of calls to preg_match/preg_replace. It's usually better, if possible, to try and call preg_* beforehand on the big string, rather than split the string and then call preg_* a lot of times.

  6. db9429
    Member
    Posted 1 year ago #

    Hey normadize

    I just wanted to stop by and thank you for this thread. I'm having issues with the plugin too: it's taking up to 60 seconds to load with qTranslate, and about 4 seconds without.

    Anyway, I've bookmarked this page, and will continue to watch.

    Thanks again

    db9429

  7. Fred
    Member
    Posted 1 year ago #

    Thank you for your help
    I will post some improvements too.
    Mainly regarding the way to store the tag translation.
    Change 35mo into 48ko...

  8. Fred
    Member
    Posted 1 year ago #

    Hi
    I have developed some code to not store the tag translation into the wp_options table option_name qtranslate_term_name field.
    It can be improved or stored into the term_meta table, but for my need and with the time I had, I did that.
    Basically, it turns 35Mo of serialized array into a 48ko table content.
    Here is the related thread http://www.qianqin.de/qtranslate/forum/viewtopic.php?f=3&t=4073
    You can remove the unique Key, I did it because I need one language.

    CREATE TABLE IF NOT EXISTS wp_qtranslate_terms_lang (
      term_id int(11) NOT NULL,
      term_meta varchar(32) NOT NULL,
      term_value varchar(150) NOT NULL,
      UNIQUE KEY term_id (<code>term_id</code>)
    ) ENGINE=MyISAM DEFAULT CHARSET=latin1;

    Please visit the thread to get the files I updated. Some hardcoding but it can serve as a basis to improve the rest.

    Frederic

    qtranslate_core.php
    - qtrans_saveConfig()
    - qtrans_updateTermLibrary()
    - qtrans_useTermLib($obj)

    qtranslate_wphacks.php
    - qtrans_insertTermInput($id,$name,$term,$language)
    - qtrans_insertTermInput2($id,$name,$term,$language)

  9. Fred
    Member
    Posted 1 year ago #

  10. Fred
    Member
    Posted 1 year ago #

    let me know if it works for you

    // splits text with language tags into array
    function qtrans_split($text, $quicktags = true) {
    	global $q_config;
    
    	//init vars
    	$split_regex = "#(<!--[^-]*-->|\[:[a-z]{2}\])#ism";
    	$current_language = "";
    	$result = array();
    	foreach($q_config['enabled_languages'] as $language) {
    		$result[$language] = "";
    	}
    
    	// split text at all xml comments
    	$blocks = preg_split($split_regex, $text, -1, PREG_SPLIT_NO_EMPTY|PREG_SPLIT_DELIM_CAPTURE);
    	foreach($blocks as $block) {
    		# detect language tags
    		// Frederic
    		// if(preg_match("#^<!--:([a-z]{2})-->$#ism", $block, $matches)) {
    		$matches = array();
    		foreach($q_config['enabled_languages'] as $language) {
    			if($quicktags) { if(strpos($block, "[:".$language."]") !== false) { $matches[1] = $language; } }
    			else {
    				if(strpos($block, "<!--:".$language."-->") !== false) { $matches[1] = $language; }
    			}
    		}
    		if(isset($matches[1]) && $matches[1] != '') {
    			if(qtrans_isEnabled($matches[1])) {
    				$current_language = $matches[1];
    			} else {
    				$current_language = "invalid";
    			}
    			continue;
    		// detect quicktags
    		/*} elseif($quicktags && preg_match("#^\[:([a-z]{2})\]$#ism", $block, $matches)) {
    			if(qtrans_isEnabled($matches[1])) {
    				$current_language = $matches[1];
    			} else {
    				$current_language = "invalid";
    			}
    			continue;
    		// detect ending tags
    		 */
    		// Frederic preg_match("#^<!--:-->$#ism", $block, $matches))
    		} elseif($block == '<!--:-->') {
    			$current_language = "";
    			continue;
    		// detect defective more tag
    		// Frederic
    		// } elseif(preg_match("#^<!--more-->$#ism", $block, $matches)) {
    		} elseif(strpos($block, "<!--more-->") !== false) {
    			foreach($q_config['enabled_languages'] as $language) {
    				$result[$language] .= $block;
    			}
    			continue;
    		}
    		// correctly categorize text block
    		if($current_language == "") {
    			// general block, add to all languages
    			foreach($q_config['enabled_languages'] as $language) {
    				$result[$language] .= $block;
    			}
    		} elseif($current_language != "invalid") {
    			// specific block, only add to active language
    			$result[$current_language] .= $block;
    		}
    	}
    	foreach($result as $lang => $lang_content) {
    		$result[$lang] = preg_replace("#(<!--more-->|<!--nextpage-->)+$#ism","",$lang_content);
    	}
    	return $result;
    }
  11. normadize
    Member
    Posted 1 year ago #

    I also optimized qtrans_split() and qtrans_use() myself ... but the whole thing needs to be rewritten.

    The plugin author is not replying to anyone anymore. Maybe it's time to move on.

  12. Fred
    Member
    Posted 1 year ago #

    I try to see what I can do.

    replace in previews post

    foreach($q_config['enabled_languages'] as $language) {
    			if(strpos($block, "<!--:".$language."-->") !== false) { $matches[1] = $language; }
    			if($quicktags && strpos($block, "[:".$language."]") !== false) { $matches[1] = $language; }
    		}
  13. normadize
    Member
    Posted 1 year ago #

    A first (not-complete) optimization for qtrans_use() which calls itself unnecessarily many times is as follows. Replace the lines:

    global $q_config;
    // return full string if language is not enabled
    if(!qtrans_isEnabled($lang)) return $text;
    if(is_array($text)) {
        // handle arrays recursively
        foreach($text as $key => $t) {
            $text[$key] = qtrans_use($lang,$text[$key],$show_available);
        }
        return $text;
    }
    
    if (is_object($text)||@get_class($text) == '__PHP_Incomplete_Class') {
        foreach(get_object_vars($text) as $key => $t) {
            $text->$key = qtrans_use($lang,$text->$key,$show_available);
        }
        return $text;
    }
    
    // prevent filtering weird data types and save some resources
    if(!is_string($text) || $text == '') {
        return $text;
    }

    with

    if (empty($text) || !is_string($text) || !preg_match($re = '/<!--:[a-z]{2}-->/', $text) || !qtrans_isEnabled($lang))
            return $text;
    
    if (is_array($text) || $text instanceof __PHP_Incomplete_Class) {
        foreach ($text as &$t)
            if ($t && is_string($t) && preg_match($re, $t))
                $t = qtrans_use($lang, $t, $show_available);
        return $text;
    }
    
    global $q_config;

    Note the & in the foreach loop.

  14. normadize
    Member
    Posted 1 year ago #

    Actually, most of the above is not needed. In fact, the second if (is_array() ... is not even reached if $text is an array or object, which is Ok in most cases, see below. You could just as well remove it altogether.

    The qtrans_postsFilter() function is hooked into 'the_posts' filter. This function is:

    function qtrans_postsFilter($posts) {
    	if(is_array($posts)) {
    		foreach($posts as $post) {
    			$post->post_content = qtrans_useCurrentLanguageIfNotFoundShowAvailable($post->post_content);
    			$post = qtrans_useCurrentLanguageIfNotFoundUseDefaultLanguage($post);
    		}
    	}
    	return $posts;
    }

    Note how it is passing each object $post in a loop to qtrans_useCurrentLanguageIfNotFoundUseDefaultLanguage($post) which in turn calls qtrans_use() on each of the object's properties. Only the title and content (and maybe guid and pinged) properties require translation (none of the other 20+ do) and for those, qTranslate already hooks into 'the_content', 'the_title' etc filters. This function can be useful if you call get_posts() in your own code.

    You could remove the 'the_posts' hook from qtranslate_hooks.php and if you need to call get_posts() in your own code and want the title and content translated, then you can either call qtrans_postsFilter(), which is very inefficient (and you may not want the default behaviour of not showing the default language for the content if the current language is unavailable), of just translate the title and content in a loop -- you can have your own function for that.

  15. Fred
    Member
    Posted 1 year ago #

    Thank you normadize.
    I`m working currently on it.
    Patch, hardcode etc, and still slow.
    I continue.
    Also qtranslate slug can maybe be improved.
    I check qtranslate currently.
    Thanks

  16. normadize
    Member
    Posted 1 year ago #

    To keep current functionality but with a faster, less-recursive function, replace in qtrans_use() the following:

    global $q_config;
    // return full string if language is not enabled
    if(!qtrans_isEnabled($lang)) return $text;
    if(is_array($text)) {
        // handle arrays recursively
        foreach($text as $key => $t) {
            $text[$key] = qtrans_use($lang,$text[$key],$show_available);
        }
        return $text;
    }
    
    if (is_object($text)||@get_class($text) == '__PHP_Incomplete_Class') {
        foreach(get_object_vars($text) as $key => $t) {
            $text->$key = qtrans_use($lang,$text->$key,$show_available);
        }
        return $text;
    }
    
    // prevent filtering weird data types and save some resources
    if(!is_string($text) || $text == '') {
        return $text;
    }

    with the following

    if (empty($text) || !qtrans_isEnabled($lang))
        return $text;
    
    $re = '/<!--:[a-z]{2}-->/';
    if (is_string($text) && !preg_match($re, $text))
        return $text;
    
    if (is_array($text) || is_object($text)) {
        foreach ($text as &$t)
            if ($t && ((is_string($t) && preg_match($re, $t)) || is_array($t) || is_object($t)))
                $t = qtrans_use($lang, $t, $show_available);
        return $text;
    }
    
    global $q_config;
  17. Fred
    Member
    Posted 1 year ago #

    I understand why you do your own plugin, because patches etc don't work really well.
    It improves fur sure, but still take time with this plugin.
    Even qtranslate slug.
    I removed some code, commented some parts, replaced some pre_reg etc, still slow for google
    Continuing...

  18. Fred
    Member
    Posted 1 year ago #

    Do you know a good debugger, easy to install etc?
    Because some tools like xdebug are not recommanded on production servers.

    I try P3 plugin
    http://wordpress.org/extend/plugins/p3-profiler/
    maybe those
    http://wordpress.org/extend/plugins/askapache-debug-viewer/
    http://wordpress.org/extend/plugins/debug-this/

    based on
    http://blog.slo-host.com/tag/qtranslate/

  19. normadize
    Member
    Posted 1 year ago #

    I've never had issues with Xdebug as far as PHP debuggers go.

    I optimized a few parts of qTranslate functions quite heavily and will post my patches soon.

    In fact, I may even release a very small plugin that patches qTranslate on the fly, which people can just install and use together with qTranslate to make it faster. It's part of a bigger plugin I am writing.

    Stay tuned.

  20. Fred
    Member
    Posted 1 year ago #

    Thank you normadize

  21. zeevg
    Member
    Posted 1 year ago #

    Thank you for all these optimizations. qTranslate is driving me nuts as well, but I don't know of any alternative.

    Could I ask, which version are you applying these mods to?

    And would it be too much to ask that you provide a link to your full optimized version?

    Thanks!

  22. normadize
    Member
    Posted 1 year ago #

    I'll try to find time to do this but I'm pretty heavily swamped with work at the moment.

  23. Graucho Marx
    Member
    Posted 1 year ago #

    I'm waiting in line, too.
    I think qTranslate is a powerful tool, but it's a shame the author stopped working on it.
    When I needed this plugin, it was the best of them, do you know if things have changed now?
    Because, as normadice says, maybe it's moment to move on...

    BTW, thanks for your work, guys.

  24. leoloso
    Member
    Posted 1 year ago #

    I'm also waiting for the optimized version... thx normadize for your great work! :)

  25. leoloso
    Member
    Posted 1 year ago #

    the code in qtrans_use doesn't work with shortcodes ([:en]). I changed it like this and now it works:

    $re = '/<!--:[a-z]{2}-->|\[:[a-z]{2}\]/';

  26. leoloso
    Member
    Posted 1 year ago #

    Also, another problem: with the iterator by reference I get an error:
    An iterator cannot be used with foreach by reference

    So I had to change it to

    foreach ($text as $t)
  27. normadize
    Member
    Posted 1 year ago #

    @leoloso: what version of PHP are you using? I never had that issue with PHP 5.3 or PHP 5.4. Arrays or objects can be used with references in foreach() and the iterator var would be a reference to the actual array value or object property (faster for big and nested stuff).

  28. leoloso
    Member
    Posted 1 year ago #

    5.2.17

    are you suggesting then I ask my hosting provider to update PHP?

  29. leoloso
    Member
    Posted 1 year ago #

    I upgraded to PHP 5.3, 5.4 and 5.5, but the iteration by reference never worked (doing a Google Search of the error, most results also say it doesn't work)

  30. normadize
    Member
    Posted 1 year ago #

    foreach loops with references (to modify directly the contents of arrays/objects) is definitely supported and has been supported for the longest time: http://php.net/manual/en/control-structures.foreach.php

    The following tests were done on PHP 5.3.23, but should work just the same for lower versions.

    $a = array ('foo', 'bar');
    foreach ($a as &$value)
        $value .= ' here!';

    Outputs, as expected:

    Array
    (
        [0] => foo here!
        [1] => bar here!
    )

    Same with objects:

    class A {
        public $foo = 'foo';
        public $bar = 'bar';
    }
    $a = new A;
    foreach ($a as &$prop)
        $prop .= ' here!';
    print_r($a);

    Outputs, as expected:

    A Object
    (
        [foo] => foo here!
        [bar] => bar here!
    )

    It seems there's something fishy on your end.

Topic Closed

This topic has been closed to new replies.

About this Plugin

About this Topic