WordPress › Support » Permalinks doesn't scale

waterwheel
Member
Posted 2 years ago #

As noted here:
http://comox.textdrive.com/pipermail/wp-testers/2009-January/011116.html

Using permalinks in wordpress doesn't scale. Works OK if you are using a dozen pages, but dies a slobbery death if you have thousands of pages.

...which I have (thousands of pages, and looking to add more).

After some research it looks like I may be able to turn off permalinks for most of my thousands of pages, but I'm migrating my site over to wordpress and my base pages do need to keep pretty URL's. My about page needs to be /about, /contactus and so on.

Does anyone have any thoughts on solutions? Either having permalinks on all pages across a wp install with thousands of pages, or having permalinks only for some pages? I don't think 301 individual pages in my .htaccess is a proper solution.
@mercime
Member
Posted 2 years ago #

I would use WordPress Multi-User as CMS in your case and not the single WP. Spead your Pages in Sections via sub-blogs of the installation, plus use pretty permalinks all through the whole installation. WPMU as CMS is outstanding :-)

One sub-blog = One section. You can have up to 200 sub-blogs or a little more before you have to think about multi-database solution.

You could use one theme throughout your installation with only conditional tags for main site's home page.

Shane G.
Member
Posted 2 years ago #

Hi,

You can set your desired permalink from wordpress admin area and add this code in htaccess:

# BEGIN WordPress

<IfModule mod_rewrite.c>
ErrorDocument 404 /index.php?error=404
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>

# END WordPress

Now, have a check with old and new posts for all the pages and posts.

Thanks,

Shane G.

waterwheel
Member
Posted 2 years ago #

Shane, perhaps I've misunderstood, but I think that's just the regular permalink .htaccess structure. And the problem is that works fine for 100 pages, but does not work fine for 1000 pages.

Mercime, that's a solution but very manual intensive. I'm looking to do this across a number of sites, some of those sites will have 10K to 50k pages. And I need to automate it, otherwise it becomes unmanageable. We're already touching these pages way more than we should - it's hugely labor intensive.

I was thinking that I might just do this. Write a script that dumps all the pageid's out to a database, onetime. then write a 404 error script that takes the pretty page address and maps it to the wordpress page id. In otherwords, bypass the wordpress page redirect. I've used this on other non-wordpress sites and it works fine. Not sure why wordpress doesn't do something like this - I'm sure it's technical - but I'm a bit astonished that a platform as big as this has a feature so widely used, that doesn't work for more than a couple hundred pages.

In any event - is there any other solution other than the cludge I've suggested?
waterwheel
Member
Posted 2 years ago #

Resolved via kludge :).

We're going to do this:
on a 404 we're going to call a script. If the page pretty-page-url is called it's going to look in the wp_posts table for a page where the title field is "pretty page url' (though case insensitive). It will then grab the page id, and with that information it will display index.php?page_id=ID.

In otherwords, we're going to do the mapping ourselves. I appreciate there are reasons wordpress does this the way they do - but frankly, it's wrong. Rationalize however you like, in the end permalinks don't work for large numbers of pages. It should be seamless, it's not - it crashes and burns. (and the answer is not 'modify user behavior :) ).
furca
Member
Posted 1 year ago #

Hi,

PLEASE send me an email to andy@furca.com with more details on your fix.

I would greatly appreciate it!
Darrell Schauss
Member
Posted 1 year ago #

I don't have an answer for this problem but I just wanted to leave a note. Maybe its pages versus posts, but I helped transfer 3000 posts from joomla to wordpress. The permalink structure is post title only. The site has 4500 articles now. Doesn't seem to be any issues with posts.
furca
Member
Posted 1 year ago #

Still looking for a way to do the 404 call-script as described above.
Gary Bairead
Member
Posted 1 year ago #

If you read earlier in the thread, it's only certian permalink structures which have scalability problem once you start using large numbers of pages - such as having /%postname%/ or /%category%/ at the beginning of the rewrite structure.

More info - http://ottopress.com/2010/category-in-permalinks-considered-harmful/
furca
Member
Posted 1 year ago #

But it is my feeling that %pagename% is basically the same thing.
James
Happiness Engineer
Posted 1 year ago #

%pagename% is not a valid permalink tag. Post names and page names are actually the same thing when permalinks are concerned, so %postname% should be what you want.
furca
Member
Posted 1 year ago #

Mac,

Thanks for pointing that out. So, when I have 3,000 pages it generates this error "Fatal error: Maximum execution time of 30 seconds exceeded in /home/furca/public_html/wp-includes/rewrite.php on line 851"

That line contains:

if ( ! $this->use_verbose_page_rules ) {
$this->add_rewrite_tag('%pagename%', "(.+?)", 'pagename=');
$rewrite_rules = array_merge($rewrite_rules, $this->generate_rewrite_rules($page_structure, EP_PAGES));
return $rewrite_rules;
}

------------------------

So I think it has to do with %pagename% / permalink
James
Happiness Engineer
Posted 1 year ago #

That could certainly be the problem. Try %postname% instead.
furca
Member
Posted 1 year ago #

Mac,

The %pagename% is the default in that file. I switched it to postname and it generated some errors. Like, it works good now (with the default setting of %pagename% there) since I deleted 2,000 pages so i'm down to 1,000. But when I go back and add 2,000 pages it will break again. I need to re-route using the 404 method like he mentioned, or manually map out the pages to their URLs. Any ideas how to manually map?
James
Happiness Engineer
Posted 1 year ago #

%pagename% is the default in what file? It must not be a core WordPress file since %pagename% is an invalid permalink tag.

I understand what you're after, but I think you're trying to fix something that's broken by working around it rather than fixing the problem. There must be at least a hundred WordPress blogs out there with over 3,000 posts. Of the few that I read on a daily basis, none use any fancy scripts or mapping techniques.
furca
Member
Posted 1 year ago #

Mac,

That's correct. The core file references this in my line 851 of rewrite.php This is the default code and the error line that the web is saying has an issue/timeout.

"if ( ! $this->use_verbose_page_rules ) {
$this->add_rewrite_tag('%pagename%', "(.+?)", 'pagename=');
$rewrite_rules = array_merge($rewrite_rules, $this->generate_rewrite_rules($page_structure, EP_PAGES));
return $rewrite_rules;
}"

You noted that some sites have over 3000 posts and I am not sure if there is a difference but these are strictly pages. And this error has been documented on various websites (after a few hours of google research). The sites you are using may use a different structure, thus deviating from this error, but I need my pages to be located at site.com/page-name/.

Solutions appreciated.

Cheers,
furca
Member
Posted 1 year ago #

"There's no particularly good all-around solution, unless you want to
precalculate every possible URL and store them in a separate table
with that URL as the index. Then it's one query for all cases, but
maintaining that table becomes difficult and problematic. It also
becomes static, without the ability to adapt to new cases. Perhaps
this is the way to go for the future, perhaps a plugin could make this
sort of optimization by overriding the rewrite system. But it seems
like more trouble than it's worth, really."

Anyone with any ideas?
furca
Member
Posted 1 year ago #

I AM SO DUMB!

SOLUTION: Set the permalink to /%year%/%postname%/ then it WORKS.

REASON WHY IT WORKS: I wasn't posting articles anyway and pages are defaultly set to display based on page name anyway!

AWESOME.

Topic Closed

This topic has been closed to new replies.

WordPress.org

Forums

Permalinks doesn't scale (18 posts)

Topic Closed

About this Topic

Tags

Code is Poetry