Maximum number of pages supported without performance impact

tanvirkazmi
(@tanvirkazmi)

11 years ago

Hi friends,

I have a WordPress blog where the number of pages (not posts) has been growing at a rapid pace. As of date, it stands at about 3500 pages. The pages are organized into primarily 3 top level pages, with each of the top level pages having about 1000 child pages each.

I use WP super cache, and till now the performance of loading the pages, as well as the delay when I create a new page or save an existing page has been acceptable.

I would like to know from the experts here if I am already operating at the limits of what current wordpress (3.5) can support, or is there more scope to grow, say from 3.5K pages to 10K, or even say 15K pages? Will wordpress be able to handle that kind of load and not become totally unusable or too slow for activities like editing/saving/creating pages?

I have been evaluating whether I should stick to WordPress or move to another CMS like Drupal/Joomla if this can become a problem, and your help on this would be immensely useful for me in making that decision.

Some numbers for the experts which might help in evaluating:
Loading 1st top level page – 1147 queries (very rarely used).
Loading a child page for above – 68 SQL queries.

Loading 2nd top level page – 1389 queries (very rarely used).
Loading a child page for above – 63 SQL queries.

Loading 3rd top level page – 61 queries
Loading a child page for above – 63 SQL queries.

Note: For the 3rd top level page above, the page hierarchy is more evenly broken up rather than 1000 child pages under 1 page. Maybe that is the reason why SQL processing happens in a more optimized manner for the top level page. Just a guess.

I have been using shared hosting on Hostgator. Would a move to dedicated hosting be advised?

Thanks,
Tanvir

Viewing 10 replies - 1 through 10 (of 10 total)

esmi
(@esmi)

11 years ago

WordPress, in itself, does not impose any limits on how many Pages or Posts you can publish. In theory, you could have millions. In reality, however, a great deal depends upon your current server and its configuration, load & performance. If the site is still growing, then I’d suggest that you need to start looking at a dedicated server now (if you aren’t already on one) so that you prepared for when the site does start to become sluggish.

You might also want to subscribe to the wp-hackers mailing list. That’s where some of the real heavy-weight WP users hang out, so it’s one of the best places to get advice on wrangling large, corporate-level, sites.

Thread Starter tanvirkazmi
(@tanvirkazmi)

11 years ago

Thanks for your comments, esmi. Does this imply that it would be okay to continue to create more pages on self hosted wordpress blog (going upto my target limit of about 15,000 pages) and the slowness issues would be resolved by the hosting? Or are there inherent problems in the WP implementation which will cause problems which may not be resolved by moving to a better server?

I mean there are blogs with huge number of posts, all running wordpress, so apparently WP does a good job with posts (as long as permalinks are used intelligently, year/month/day format being probably the best, or one of), but when we are talking about pages, that I think will be a different ball game altogether.

Would just moving to a better server be the answer. Or would one of the answers be that wordpress itself wont be able to handle this and this problem needs another CMS?

esmi
(@esmi)

11 years ago

Does this imply that it would be okay to continue to create more pages on self hosted wordpress blog (going upto my target limit of about 15,000 pages) and the slowness issues would be resolved by the hosting?

Based on what I’ve read on both the wp-hackers list and here, yes – although I should point out that I’ve not hosted a site of that size myself yet. But if you look at some of the huge sites hosted on wordpress.com’s VIP package (eg CNN. NBC Sports etc), it does suggest that WordPress can be scaled up quite successfully given the right server environment.

Gabe Young
(@gabeyoung)

11 years ago

Some of my sites have many pages and posts. Performance has never been an issue. In the event it does, however, I would simply work with my host provider to select the best solution for me.

However, as you know, performance isn’t solely based on number of pages or posts. Obviously some things are more taxing than others, like non-optimized images.

Chris Olbekson
(@c3mdigital)

11 years ago

I have been using shared hosting on Hostgator. Would a move to dedicated hosting be advised?

You don’t necessarily need a dedicated server but you do need something besides cheap shared hosting. You need an environment that uses either memcached or apc or both so you can take advantage of the built in WordPress object caching. With object caching those queries will probably go from over a thousand to less than 200.

You can get decent VPS hosting for around $40 a month or use one of the managed WordPress hosts like WP Engine.

I mean there are blogs with huge number of posts, all running wordpress, so apparently WP does a good job with posts (as long as permalinks are used intelligently, year/month/day format being probably the best, or one of), but when we are talking about pages, that I think will be a different ball game altogether.

A page is just another post type and new rewrite rules were added back in version 3.3 that fixed the verbose rewrite rules performance hit. So using pages should be just like using posts with /%postname%/ rewrite rules.

Loading 1st top level page – 1147 queries (very rarely used).

This seems very excessive. Are these results the same with all plugins disabled using one of the default themes? If not your code needs to be better optimized.

Mike Bijon
(@mbijon)

11 years ago

If you’re considering the jump to a VPS host, beware that there’s a lot of from-scratch server setup work to be done. If you’re new to it, it can take a full day and you should plan time to do security fixes too.

Unless you have a lot of free time, WP Engine is more expensive but can be a big shortcut to save setup time. Other options that are faster to setup include Dreamhost’s VPS systems & AppFog. Both have one-click WordPress installers that include database setup and guenerally follow good security practices. Neither configure caching out of the box however.

Thread Starter tanvirkazmi
(@tanvirkazmi)

11 years ago

A note to add more perspective on this problem.

My blog monthly views average about 225,000.
By blog database size is 62.5 MB with the wp_posts taking up 40MB.

I am sure the above numbers are pretty ordinary. Given these, what kind of hosting would be advisable? I talked to Hostgator and they straightaway recommended their Basic Dedicated Server ($139/mo). They also advised me against their VPS saying it was not a performance platform. Several months ago, another of their support guy had given me a figure that shared hosting can easily handle 45K daily pageviews. But I guess this is subjective and vary blog to blog.

I am also talking to WordPress oriented hosting services, in particular WPEngine ($249/mo) and Websynthesis ($97/mo). WPEngine looks very costly by comparison but has a neat set of features (staging area, auto caching, backups etc). WebSynthesis is the lowest costing out of these three and supports much more pageviews/day compared to WPEngine (I calculated an equivalent cost of a sixth of WPEngine for same number of pageviews).

Should a move to a WordPress specific host be considered, any experiences from WordPress experts here would be useful. Or just jump to basic dedicated server from Hostgator, analyze it over few months, learn from the experience, and then see how it goes.

Thanks,

Gabe Young
(@gabeyoung)

11 years ago

From what I can tell, I don’t think you need the WP oriented services. They package a lot of things in there and it sounds like you’re perform much of that already.

As for whether or not Hostgator is offering you the best option, if you prefer to stay put, I agree with you that choosing their recommended dedicated server option and monitoring is a good way to go. Keep in mind that they may have you sign up for more than just a month-to-month.

If you are not married to your current host (and it doesn’t seem like you are), another alternative is to call other hosting companies and see what they would recommend based on your numbers. You’re moving into the $1000-2000 annual range so shopping around might not be a bad idea.

Moderator Ipstenu (Mika Epstein)
(@ipstenu)

🏳️‍🌈 Advisor and Activist

11 years ago

My blog monthly views average about 225,000.
By blog database size is 62.5 MB with the wp_posts taking up 40MB.

The pageviews can be handled on a 2G VPS, if correctly tuned and protected with the right server caching.

If your DB is that large, I would actually make sure it was optimized, and delete old post revisions.
DrowSserp
(@drowsserp)

10 years, 1 month ago
Old post but in case this helps others..

The largest WordPress site I have on Hostgators shared hosting with their ‘Business Plan’ gets around 75,000 page views per day from around 7,000 visitors/day. So around 7,000 people viewing 10 pages each in around 5 minutes per visit. (aw-stats)

I am pretty much maxed at that. By maxed I mean if I get traffic spikes above that I run over the shared resource limits and they will occasionally shut me down if those spikes last too long. By shutting me down it means they disable the site until I assure them I’ve resolved the issues (optimize the site to be able to handle it by deleting plugins, rewriting the theme etc.)

I ‘might’ be able to get it up to 100,000 page views per day by rewriting the theme to eliminate the use of a few plugins that it still relies on but that handful of plugins that are left have pretty minimal impact so it will remain to be seen.

I have another WordPress site on Hostgator that can handle around 30k visits per day but pageviews are only about 40-45k. It uses just 1 plugin, Advanced Custom Fields, and it seems to run as cool as a cucumber but we’ll have to see how well it does once it grows to match the 75k wall I’ve encountered.

About the site
- It is image heavy
- Has around 1500 posts
- DB is around 40mb (occasionally optimized)
- I use WP Super Cache with Hostgators recomended settings
- I have the free account with Cloudflare (a semi-cdn, caches lots of site files like your css etc.)
- Burns about 5gb bandwidth per day (much more if not using Cloudflare)
You can further optimize by using WP Super Caches static page options (which i’ve yet to test out) but if it gets to that point it’s time to look into a dedicated and save yourself the headache of a website that could be temporarily suspended at any random time. I keep this one site on Hostgator as an experiment for a project I’m working on and it’s been great learning as it forces me to optimize themes to run smoother with limited server resources.