Support » Plugin: WP-Appbox » Number of issues with Wp-Appbox for a high-traffic site

  • Resolved archon810

    (@archon810)


    Hi everyone,

    Since Playboard is shutting down, we’re evaluating WP Appbox for androidpolice.com, a high-traffic site. I’m the AP founder and a developer, with experience in scaling.

    First of all, I love what you’ve done here – it’s at the top of our list for a reason.

    But some issues have cropped up in our testing that I was hoping to highlight for you. The issues don’t particularly present a scaling problem for us – our hosts are quite beefy (64GB RAM, fast SSDs, 16 CPU), but rather from the DOS/hammer of other sites point of view – specifically the Play Store.

    Since we’ve had some experience designing similar applications, I’ll offer suggestions at the end.

    1. The most serious issue, in my opinion, is the fact that the plug-in does not cache failures. Specifically, if an a plugin is used for an app that isn’t yet (future release but we know the pname), is no longer (more common – was there before but was removed since), or will never be (maybe by accident) in the Play Store, it will not cache this fact and will hit the Play Store every single time.

    This means that if 10,000 people visit the page today, it will hit the Play Store 10,000 times. Now, a server-side caching plugin, like W3-total-cache, does alleviate this a bit, but this logic is very bad IMO. This can result in the Play Store servers / Google firewalls blocking our servers either temporarily or permanently, or adding CAPTCHA checks.

    1a. Follow-up concern – there’s no rate-limiting for Play Store requests, is there? So in theory, a large (possibly for the purpose of a DOS attack) number of requests to our pages hosting such widgets may translate into a DOS attack on the Play Store, and it’d be easy for someone to engineer an attack that would prompt Google to ban our servers.

    Solution: Cache failures (with a separate configurable setting, say defaulting to 5 minutes).

    2. In the absence of cache, data from the Play Store is loaded within the same request, meaning a page first waits for the Play Store data, then returns to the user. If the Play Store is slow or timing out entirely (if their firewall kicked in, for instance), this could affect our whole site and visitors will start getting slow responses.

    3. The refresh button. Personally, I don’t think it’s necessary at all, but perhaps it is for sites that will choose to cache for a very long time – hours. The refresh button on its own would be fine, but it has issues.

    a) On a busy site, many people could be clicking it over and over, thus requesting refreshes much more than necessary.

    b) As an extension, and similar to issue 1. above, it’s possible for an attacker to cause instantaneous cache invalidations, thus amplifying the attack on the Play Store because not only non-existent app widgets would be hitting Google’s servers, but also existing working widgets.

    c) From our analysis, the cache is invalidated immediately upon pressing the refresh button, without a refreshed version of the cache available, which can cause spikes of requests due to race conditions (say, 25 people visit the site while the widget is being regenerated – all of these people will cause Play Store requests to fire because the cache isn’t yet populated)

    Solution: This is what worked well for us.

    a) Instead of using transients and invalidating them right away, use a separate table (which I think you are already doing), which keeps track of the last update time.

    b) When invalidation is needed, populate the fresh data first, then delete the old one. In the meantime, continue serving the old data.

    c) Use WP cron to schedule refreshes. This allows for bundling multiple refresh requests (an anti-DOS tool) into one by not scheduling multiple pings for the same pname. It also resolves 2. where visitors would be slowed down by Play Store requests. And in order for the first visitor to not see an unpopulated widget, you can hook into draft -> publish and fire off all the requests then – basically pre-populate the data before a post goes live.

    Oh, also separate suggestions:
    4. The presence of the refresh button should be configurable (present, not present).
    5. The widget not found text should be configurable (I would definitely want to remove “:( #wpappbox”, for example).

    Would love to hear your thoughts about these.

    Thanks,
    Artem

    • This topic was modified 8 months, 1 week ago by  archon810.
    • This topic was modified 8 months, 1 week ago by  archon810.
Viewing 4 replies - 1 through 4 (of 4 total)
  • Plugin Author Marcelismus

    (@marcelismus)

    Hey archon,

    thank you for your constructive criticism.

    3. The refresh button. Personally, I don’t think it’s necessary at all, but perhaps it is for sites that will choose to cache for a very long time – hours. The refresh button on its own would be fine, but it has issues.

    The refresh button is only visible to users who are logged in, not for every visitor. Isn’t it?

    5. The widget not found text should be configurable (I would definitely want to remove “:( #wpappbox”, for example).

    It is also planned (see bottom).

    a) Instead of using transients and invalidating them right away, use a separate table (which I think you are already doing), which keeps track of the last update time.

    WP-Appbox uses an own table.

    Solution: Cache failures (with a separate configurable setting, say defaulting to 5 minutes).

    b) When invalidation is needed, populate the fresh data first, then delete the old one. In the meantime, continue serving the old data.

    c) Use WP cron to schedule refreshes.

    Short answer: I am working on a new cache system atm, but I need some time. Removed apps should also be available later, image caching is also on my todo. I thinked about crons, but it depends on the (small) revisited cache principle.

    At the moment there isn’t any fast solution, but I think an server-side cache plugin or an CDN should be working. Especially with a high setting for the Appbox cache, so visitors don’t trigger a reload from the Play Store. Yes: if the app isn’t found, it triggers to much, this should be recovered by an cache plugin/CDN, but I have add it to my todo.

    You can contact me by mail (https://tchgdns.de/impressum/), maybe we find a short term workaround.

    – Marcel

    • This reply was modified 8 months, 1 week ago by  Marcelismus.

    Hi Marcel,

    Thanks for your responses.

    Regarding the refresh button, ah, I didn’t notice. What about the cache-clearing request – does it check for admin rights too and only allows clearing cache for admins? Or anyone can craft a simple request to fake the refresh button?

    Looking forward to all your improvements, especially caching the not found apps – that’s sort of the biggest issue for us I think.

    Do you have any idea what the roadmap timeline is like for these improvements?

    Thank you.

    Plugin Author Marcelismus

    (@marcelismus)

    Regarding the refresh button, ah, I didn’t notice. What about the cache-clearing request – does it check for admin rights too and only allows clearing cache for admins? Or anyone can craft a simple request to fake the refresh button?

    If the user isn’t author (or above), the button isn’t visible. If a user get the reload link but isn’t author it won’t do anything.

    Do you have any idea what the roadmap timeline is like for these improvements?

    Do you know Duke Nukem Forever? 😉 When it’s done… ^_^

    I’m happy to report that after working with Marcel for several months, reporting bugs and suggesting improvements, the plugin is ready for prime time on a high-traffic site.

    I just rolled it out live to Android Police: https://plus.google.com/u/0/+ArtemRussakovskii/posts/6eWsU86JJwa.

    Thank you, Marcel.

Viewing 4 replies - 1 through 4 (of 4 total)
  • The topic ‘Number of issues with Wp-Appbox for a high-traffic site’ is closed to new replies.