Remove Tag Base Fatal Error.

Hello, I have 34,549 Tags, 8,118 Categories [Note:The Number will increase in the very recent future]. I have used wordpress seo plugin to remove category base now i want to remove tag base. Whenever i goes to active Wp no tag base, I get this error,

Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 88 bytes) in /home6/anterio1/public_html/mysite/wp-includes/cache.php on line 430

I have edited wp.config -> define(‘WP_MEMORY_LIMIT’, ‘512M’); [Note: I also tried with higher values]

Please Help Me.
Important Notice: I wish i could remove the tag base and category base from the very core of wordpress and make it optional.