Posted 5 months ago #
Hi, Tobias:
I have built a website that currently has about 250 Tables (each about 10-50kb in size, avg. 20-25kb). Is there a physical, or desirable, limit at which TablePress, or WordPress for that matter, will either be unable to accept more, or will show serious performance issues? I plan to add another 500, and possibly another 300-500 every quarter.
Thanks for your response,
Ganaxi
Hi,
thanks for your question.
I can't give guarantees, obviously, but with the new data format and storage location that TablePress uses ("Custom Post Types" in WordPress), I don't think that the number of tables will result in performance problems. After all, each table is - very similar to a post or page - just an entry in the "wp_posts" table. And there are WordPress sites with thousands of posts and pages, so I don't think that this will be an issue here.
At some point, the "All Tables" screen might be slow in loading, but I don't have enough experience from feedback (or "measurement data") to give details or approximations here.
Regards,
Tobias
Posted 5 months ago #
Hi Tobias,
Thanks for your earlier response.
Is there any limit to file size. I have been trying to upload a CSV delimited (from Excel) file (size 234kb) with 8 columns and 3300 rows, and it does not save (the page becomes unresponsive, with the option given to kill pages or wait), and then when I click on what appears it saved, it does not open it.
Thanks,
Ganaxi
Hi,
the limit here is not really the file size, but the browser memory. See, 8 columns and 3300 rows mean that there at least 8*3300=26400 text input fields on the page. Unless a computer is really fast and has a lot of memory, this is simply going to overstrain the browser, as you experienced.
So, importing this is possible, but editing the table in TablePress is not really a good idea. Unfortunately, I don't have a good solution to this, sorry :-(
Regards,
Tobias
Posted 5 months ago #
Addendum to above Question: It does save, but even on a 30 Mbps connection, it takes many minutes.
Hi,
yes, as mentioned it's more the browser, the memory, and the processing time of the input field. The time to transmit the data (and thus the connection speed) are not relevant here.
It's just slow for the browser to manage and walk through all the input fields, collect their content, throw it all together in a "package" and then finally send that off.
Regards,
Tobias
Posted 5 months ago #
Hi Tobias,
I figured how to do this in TablePress (I had one even larger table, that just crashed the browser everytime).
I hid the Table Content tab while uploading and saving a smaller table. Then, when I uploaded the subject table, since the Table Content tab was collapsed, the browser did not have to render the data, and that narrowed down the upload time to a few seconds for even the largest tables.
A problem I am continuing to have with some of the larger Tables, is that after it saves, if I change the ID# to fit my custom nomenclature, it refuses to change it -- no matter how many times I try (it reverts back to the original ID#). I am hoping that this is something that you have a fix for.
Thanks again,
Ganaxi
Hi,
that sounds like a good workaround, indeed.
I'm not really sure about that ID-changing issue. Are you seeing the confirmation message about having to change all Shortcodes?
A work around for this might be to create a new table, change the ID to what you want, and then import your (large) data set, choosing to replace the existing table (which already has the desired ID).
Regards,
Tobias
Posted 5 months ago #
Hi,
The ID-changing issue happens only for the large tables (more than, say, 1000 rows, and 8 columns). Thanks for the workaround, about creating a new table... I will try that. If you come up with a more elegant solution, though, please post it.
Hi,
I don't see how this could only affect large tables, but probably it also is related to the JavaScript being executed only very slowly.
Of course, if I find something, I'll keep you posted with updates!
Best wishes,
Tobias
Posted 5 months ago #
Hi,
FYI - Found a better workaround. Quirky, but it works.
If you click on the Table Name field after entering the new (alternate) ID, for some reason, it saves the new ID (does not revert back to the old one). And it does that every time.
Thanks,
Ganaxi
Hi,
ah, so before, you were changing the ID and directly clicking "Save changes"? That won't work indeed, you'll have to click somewhere else first.
I'm trying to find a better way for this though, which will then be part of the next TablePress version.
Regards,
Tobias
Posted 5 months ago #
Hi, Tobias:
Now I am getting the following error when I try to import more Tables.
=======
Server error
The website encountered an error while retrieving http://www.gurufundpicks.com/wp-admin/admin.php?page=tablepress_import. It may be down for maintenance or configured incorrectly.
Here are some suggestions:
Reload this webpage later.
HTTP Error 500 (Internal Server Error): An unexpected condition was encountered while the server was attempting to fulfill the request.
======
Can you tell if this is this a TablePress issue, or something that my host should have to fix.
Thanks,
Ganaxi
Hi,
unfortunately, I don't know why this happens. It might be a result of the processing that takes too long.
You should try to get more details about the error from the server error log. Either you can access this yourself, or you'll have to ask your host for it.
Regards,
Tobias
Posted 5 months ago #
Hi Tobias,
Just got feedback from my hosting provider. They helped me fix the problem temporarily, but below is their feedback to help you further enhance the plugin.
Basically, I understood that the problem maybe that the plugin loads all of the tables into memory on the back-end, as opposed to loading just one table or just the names, so when there are a lot of tables (in my case, over 500 ranging in size 3kb to 300kb each, the resulting file is too large for the server memory even when set to 0.25GB, which I understood is very large for just one plugin.
=======
For the developer of the Tables plugin, please let him know that the $table variable in model-table.php is using json_decode (line 151), which, for having a lot of tables, will exhaust the memory limit on the server as the number of tables increases. Additionally, the "View Tables" tab (default) of the plugin should probably only list the names of the tables, not the data, making this json_decode call unnecessary.
=======
The temporary fix they have given me includes commenting out line 151 while importing, and then reverting back to the old model-table.php file, after import is complete.
Thanks,
Ganaxi
Posted 5 months ago #
Hi Tobias,
the workaround mentioned above did not work. So basically, I am in a bind now, as I would like to continue using the plugin, but I have a website to deploy, and I can't do it using the plugin until I can add more tables. Please help.
Thanks,
Ganaxi
Hi,
you seem to have a very cooperative webhost! Their investigations are really nice and helpful! I haven't seen a webhost who even investigates the source code of a plugin that seems to cause such plugins before, so that's very cool. Please send them a thank you from me! :-)
Now, unfortunately I can't give estimates on whether, or how, I can use this information to make the plugin better usable with many tables. I have some ideas, but I need some time to think about, to implement, and to test them. It definitely looks promising though.
So, unfortunately, until I have a solution to this that I can add to the plugin, I'm out of ideas on how I could help you right now :-( Sorry.
Regards,
Tobias
Posted 5 months ago #
Yes, they are great!
I know you mentioned that you can't give a time estimate, and while I appreciate the candor, I will explain to you my situation. I can wait till sometime midday tomorrow, but after that, if there is no solution that is workable, I have to look at options. A couple that you can shed light on, perhaps.
Can I use both WPTable-Reloaded and TablePress on the same installation, or will they conflict. My idea is since I am exceeding the memory limits for one plugin, to distribute the tables between two, TablePress and WP-Tables Reloaded.
If that's possible (i.e., there is no fatal flaw there in your estimate, i.e., a conflict or that the memory would still be a problem), are you able to recommend perhaps another Tables plugin that I can use, while you fix the issue, so that I can get my site deployed.
Thanks for all your prompt responses, Tobias, and the great plugin. I think I may just be pushing the limits here with the memory needed, compared to the average user - hopefully the problems I am experiencing help you ID issues early enough to make an even better plugin.
Ganaxi
Hi,
I can understand your situation, but have to take the defensive route here: I won't be able to have a solution in place by midday tomorrow, and maybe not even until after the weekend. Sorry.
Exploring other options is the best idea for now, for your situation.
Using WP-Table Reloaded and TablePress together is possible, but needs some extra work. The first reason is that both plugins use the same WordPress Shortcode [table]. This can easily be circumvented, if you change the WP-Table Reloaded Shortcode with the help of the instructions at http://tobias.baethge.com/2010/01/extension-3-changing-the-table-shortcode/ You could change it to [wptrtable] for example.
The second reason is the CSS styling, which is slightly different in both plugins. Here, I recommend to deactivate the Default and Custom CSS in WP-Table Reloaded entirely, and to use the TablePress CSS for the tables from both plugins. This will also guarantee a uniform styling of your tables. To achieve that, just paste the lines
add_filter( 'wp_table_reloaded_table_css_class', 'wptr_add_tablepress_classes', 10, 2 );
function wptr_add_tablepress_classes( $classes, $table_id ) {
$classes[] = 'tablepress';
$classes[] = 'tablepress-id-' . $table_id;
return $classes;
}to your functions.php (or actually to the Extension that changes the Shortcode, to keep everything together).
After that, all tables will have the TablePress styling.
Note that this is not really a long-term solution. as WP-Table Reloaded will be discontinued soo. Unless I'm able to find a solution to the actual issue in TablePress, you might need to look into a custom PHP/mySQL database solution. Unfortunately, I also don't know other table plugins that could handle your amount of tables (which is indeed more than the average user has).
Regards,
Tobias
Posted 5 months ago #
Hi Tobias,
I will give that a shot. Thanks.
For what its worth, the host had another suggestion to pass on to you, as follows:
"One other thing to pass on to the developer would be to not store the data in the wp_options table."
Thanks,
Ganaxi
Hi,
thanks for that note as well!
I'm not really sure why they come to that conclusion though. TablePress only stores a small list with the table IDs, and their post IDs in the wp_options table. This really is not a lot of data, and actually much less than for example WP-Table Reloaded stores there... (Basically, this was one of the reason why I decided to completely rewrite the plugin.)
Regards,
Tobias
Posted 5 months ago #
Hi, Tobias:
I am not a techie, but from what I understand in laymen terms, what they told me is that I am having the memory problem of exceeding 0.25 GB - because the plugin on the back-end loads up the entire data-set for all the tables - as opposed to the just the table you are uploading, or just the names of the tables when you ask to list the Tables. That makes sense to me, because on their own, each of my Tables are very small, on the order of 3kb to 300kb -- with most near the lower end of that range. But right now, when I try to upload any - it refuses to do so, indicating that I am exceeding that 0.25 GB in the error log.
The host is WP-Engine, and you are welcome to contact them, if you need more details. The person is Topher that I had communicated this with. http://www.wpengine.com
Generally, they are open to and welcome communications with plugin developers, to make them function better in their environment. If you would like to contact them, and need further info, you can reach out to me via this message or the personal e-mail you have from our earlier communication.
Thanks,
Manish
Hi,
yes, I can understand the point about not loading all the tables in one batch. That makes sense, and I'll investigate ideas to change this.
I was just confused, because the suggestion with the "wp_options" doesn't really fit into that description.
And thanks for their contact details! If I need more information, I'll ask them directly! Thanks!
Best wishes,
Tobias
Posted 5 months ago #
Hi Tobias!!
Your plugin (TablePress) provides much-requested functionality in the WordPress ecosystem.
The comment above (regarding not storing plugin data in wp_options) is more for a convenience for your users; it would allow much easier selective backup (and insertion) of new display tables.
Contact me off-thread if you'd like more information about our platform. Topher@wpengine.com.
Hi,
ah, ok. Thanks for that feedback!
The data in wp_options is really only a list, that maps table IDs to post IDs. It's not that important for the export/backup. The easiest way for that is to use the TablePress Bulk Export feature, with JSON as the format, as that will also include all table options and not just the data.
Best wishes,
Tobias
Posted 5 months ago #
Hi,
I implemented the solution as you recommended. Thanks. So, I now have both TablePress and WP-Tables running, and I have been able to upload my Tables.
Everything is working on the front-end, but when I just now went into TablePress from the back-end, it is giving a server error, as follows:
"""HTTP Error 500 (Internal Server Error): An unexpected condition was encountered while the server was attempting to fulfill the request."""
Going into the server log, it is giving the same memory error as before, so that I can't access from the back-end, as follows:
[Thu Dec 13 07:38:07 2012] [error] [client xx.yy.zz.aaa] PHP Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 32768 bytes) in /nas/wp/www/cluster-1346/xyz/wp-content/plugins/tablepress/models/model-table.php on line 161
I need to edit a couple of the Tables in TablePress. How can I override the error so that I can edit the tables on the back-end.
Thanks,
Ganaxi
Hi,
this is not a really an error due to a bug in the code, and it can not just be "overridden". It is an error due to limited ressources (in this case the memory that PHP is allowed to use). So, to fix this, you'll need to increase the available memory limit (which probably is not possible), or reduce the amount of memory that is actually used. The easiest way for that is to deactivate a few "big" other plugins while editing tables in TablePress. It might even be enough to temporarily deactivate WP-Table Reloaded, and the re-activate it after your done editing tables in TablePress. I know, this is an ugly solution, but it is the only thing that will work, until I can implement a long-term solution that does not load all tables at once.
Regards,
Tobias
Hi again,
wait, there might be another way: The main problem is the "All Tables" list, from what I can see. So, you can try to navigate around that, by using the "Edit" link below the table (on the frontend). That will take you directly to the table's "Edit" screen, and it bypasses the problematic "All tables" list.
Regards,
Tobias
Posted 5 months ago #
Thanks,
That works great!
Hi,
very nice! :-) Thanks for the confirmation!
Best wishes,
Tobias
You must log in to post.
