Misleading results

  • arekm

    (@arekm)


    6 months, 2 weeks ago

    This plugin can be useful but it has tons of issues (was looking at database part only):

    • it totally forgets that network latency between php host and mysql host exists throwing everything into “mysql” bag. It works by doing hundreds of queries where latency adds up causing bad results. Real web pages usually don’t do hundreds of requests for single visitor entry. You can have very powerful sql hardware that will show up poorly in this benchmark while being great in real scenarios.
    • It does artificial queries that are rarely done in real word like not using indexes, ordering by rand().
    • Scores are based on results stored on plugin creator server which means that they change (as others are doing own testing and creator is adjusting these) and are not comparable between even own machines if done in some time range. Simple time reports for user own needs (beside “scoring” would be a great addition).
    • “complex test” not only tests joins but also cpu power because it uses a bit of… MD5().
    • it lack documentation of all such nuances

    This plugin has some usage but it won’t give you any real knowledge on your sql server capabilities or which parts of your db setup are weak.

    • This topic was modified 6 months, 2 weeks ago by arekm.
Viewing 1 replies (of 1 total)
  • Plugin Author Anton Aleksandrov

    (@antonaleksandrov)

    5 months, 2 weeks ago

    Thanks for the review and opinion.

    Network latency, indeed, can be an issue. And yes, number of queries, that plugin executes is enormous – just because our servers are too fast and obviously because so far I did not come across a good and heavy and slow SQL query – but I am open for that.

    I wonder what queries would look more realistic? Once again – very open for suggestions.

    And when it comes to scoring – yes, the score number depends on some values, stored on my server, but it does not depend on how many benchmark tests have been performed. I have measured performance of a very fast server and those timings serve as a reference. What ever score you’ve got – does not interfere anyone. And regarding more details – you can see actual time spent for each test on a detailed page. 🙂

    Indeed, figuring out database server bottleneck is not an easy task. Especially if you know, that you have a good hardware – it would then come to the fact of logging all SQL queries and analyzing those. Purpose of this plugin was more to identify and compare different servers, different hosting platforms to see how the perform and if the hardware is “ok” or not – because for those, who are usually on the shared hosting subscriptions – that part is a totally dark box.

    Take care!:)

Viewing 1 replies (of 1 total)
  • You must be logged in to reply to this review.