Table edit load/scroll time for 1250 rows x 12 colums

  • Resolved rgeddes

    (@rgeddes)


    11 years, 2 months ago

    Hello,

    I hope I did not miss a post relating to this topic.

    I imported csv data (1250 rows x 12 columns) and when I select a table to edit, the web page becomes unresponsive for 30 secs, and even after the table has loaded, scrolling down the table using chrome’s scroll bar is very slow. (My computer: AMD 64 Athlon X2 2.8GHz dual core.)

    I created an application to pull the data from this table and display it in a table… nothing fancy, just a display table using the following function:

    $table = TablePress::$controller->model_table->load( 9 );

    This data I load into a JS variable, and subsequently generate a table. In my case I hide the table and once it is filled I display the rows.

    It takes < 1 sec to get the table data from the server into the JS variable, and about 3 secs to display the whole table.

    I ran the Tablepress edit page with Chrome’s ‘Timeline’ profiler turned on, and I saw many layout events with a note from Chrome saying:

    “Forced synchronous layout is a possible performance bottleneck.”

    After plugging this note into google, I found this article about triggering layout events in the browser:

    How (not) to trigger a layout in WebKit

    I think Tablepress is making too many demands on the DOM in a short period of time, and of course, the DOM is *the* performance weak point.

    Question: In the Tablepress edit mode, do you draw the table with server side code (php) or do you draw the table with JS? I’d be interested in looking at this code more closely. Can you point me to that section of the Tablepress code?

    Thanks
    Richard

    http://wordpress.org/extend/plugins/tablepress/

Viewing 13 replies - 1 through 13 (of 13 total)
  • Plugin Author TobiasBg

    (@tobiasbg)

    11 years, 2 months ago

    Hi Richard,

    thanks for your post, and the research you have done so far!

    Yes, for large tables, the “Edit” screen performance is horribly slow, and I really would like to see improvements on that! Unfortunately, my knowledge on JS DOM performance and profiling is limited, so I’d love to get help on this! If you have suggestions on how to make this better in the code, please let me know!

    The relevant code for the “Edit” screen is basically the file /views/view-edit.php for the PHP part, and /admin/edit.js for the (quite extensive) JS part. The initial screen is rendered in PHP (i.e. there’s a loop over the table array that prints the HTML code for a textarea for each cell). Then there’s the JS that works with these textareas and is responsible for updating it, when rows/columns are added/deleted/inserted/etc.

    Regards,
    Tobias

    Thread Starter rgeddes

    (@rgeddes)

    11 years, 1 month ago

    Tobias,

    I ran some tests on the page load and it looks ok as far as stuffing the DOM… the code basically spits out the table as textarea elements in the table… these inputs seem to be expensive to the cpu. I tried replacing the textarea elements with input elements, but it does not change the latency. Getting rid of the input type elements was the only thing to speed up the editing process.. of coarse, there’s a conundrum there… editing without input elements.

    I looked at what phpmyadmin does, and they have two things going that make editing a large table less painful:

    1. They use pagination and by default process about 30 rows at a time.
    2. Instead of using many input elements, they use one, and dynamically move it over the table cell being edited… when the textarea loses focus, the value beneath the textarea is updated, the the data is probably sent back to the db (via ajax?), and the textarea element is hidden, and waiting to be moved over the next cell to be edited.

    Both approaches are worthwhile, but pagination would probably be easier since you already have pagination code to display the table to the user.

    I renamed the edit.js file edit.min.js to see it in the chrome debugger, and some of the features stopped working. Is edit.js the latest source? I’d like to see the latest code to maybe adapt pagination to the editor.

    Regards
    R

    Plugin Author TobiasBg

    (@tobiasbg)

    11 years, 1 month ago

    Hi,

    thanks for your further investigations on this! That’s great information!

    Yes, input elements and textareas are probably not that much of a difference. I’ve been thinking about using something like a <div> with the contenteditable attribute activated, but that technology doesn’t seem to be used that much, so there’s not a lot of information about it :-/

    A solution with pagination is actually tricky, but could maybe work. The problem is that saving just parts of a table is not supported, as the table rows can not be addressed independently, as they can in PHPMyAdmin (because they have a unique ID in the DB table). In TablePress, the entire table is stored as one (JSON encoded) string.
    Pagination could however be used to only show parts of the table, while the rest is not in the DOM. I’m not sure though if that is possible with textareas, as those can’t really have a changed value when they are not in the DOM. Re-using the pagination from the frontend is probably tricky as well, as that comes from the external DataTables JavaScript library and might already be too sophisticated.

    Having just one textarea that is moved also sounds nice. That would require some more JS, to make this behavior as “invisible” as possible, also in regard of using keyboard navigation, like “Tab” and “Shift-Tab”.

    I’m not sure why you are having problems with renaming edit.js to edit.min.js. Those have the same content, just that the min.js version ran through Google’s Closure Compiler at http://closure-compiler.appspot.com/home for minification.
    The easier way (instead of renaming files) should be to set the SCRIPT_DEBUG constant in your wp-config.php, see http://codex.wordpress.org/Debugging_in_WordPress#SCRIPT_DEBUG

    Regards,
    Tobias

    Thread Starter rgeddes

    (@rgeddes)

    11 years, 1 month ago

    Tobias,

    Thanks for your input… found all the relevant code.

    The thing about the edit.js vs edit.min.js was my fault.. I commented out tp.init() in edit.js to see if that made any difference.. forgot to undo that change… all is normal now.

    Thanks for pointing me to the DEBUG feature… unfortunately, when I turned it on, all sorts of messages were displayed…

    Anyway, I have an implementation of the single textarea input table editor… with tab/shift-tab and it wraps when it reaches the end of a row when tabbing, or the beginning of a row when shift-tabbing.

    It takes about 8 secs to load the table, thereafter the latency is much less, especially when navigating with tab/shift-tab.

    I also converted the sort by column, but it’s a little inconsistent… with numbers, it seems to do ok… with mixed numbers and characters, I noticed some minor mis-ordering.

    I’d like to send you a copy of this implementation and get your opinion… and suggestions. How would I do that?

    Regards,
    R

    Plugin Author TobiasBg

    (@tobiasbg)

    11 years, 1 month ago

    Hi,

    wow, that sounds good, I’d really like to see and test this!

    If you are using some sort of version control, like Git or Subversion, for development, please send me a “patch” via email (which you can find in the main plugin file “tablepress.php”). If you just used the plugin ZIP file and made your modifications, just send me the changed files or simply the entire (zipped) plugin folder that contains the modifications.

    About the messages that were displayed with SCRIPT_DEBUG enabled: Were those errors/warnings in the TablePress JS files? Or in other plugins’ files?

    Regards,
    Tobias

    Thread Starter rgeddes

    (@rgeddes)

    11 years, 1 month ago

    Tobias,

    I’ll zip the tablepress directory..

    BTW, where can I send it…?

    The DEBUG message was from several plugins… I never even navigated to a Tablepress related url. I”m working on a project that has many plugins.

    Regards,
    R

    Plugin Author TobiasBg

    (@tobiasbg)

    11 years, 1 month ago

    Hi,

    ah, yeah, my email address 🙂 You can find that in the main plugin file “tablepress.php”.

    About the debug messages: Did you set WP_DEBUG or SCRIPT_DEBUG?

    Regards,
    Tobias

    BeHeard

    (@beheard)

    11 years, 1 month ago

    Hope you dont mind me adding here.
    Really would love to be able to use on large tables, If this gets sorted would love the update…
    Have a problem in so far as we appear not to be able to upload the csv with the large files.
    Have tested with 1250 rows and 15 columns shows upload but then gives server error (no error logs on host or domain server)
    Those of us not so clever remain EVER GRATEFUL to you wonderful coders..

    Plugin Author TobiasBg

    (@tobiasbg)

    11 years, 1 month ago

    Hi,

    thanks for your post, and sorry for the trouble.

    The issue you are describing is unrelated to the problem/idea, that we are discussing in this thread.

    My assumption is that the available PHP memory limit is reached in your case. To be sure, we’d need to take a look at the error logs, so it’s unfortunate, that these don’t exist 🙁
    You could still try to increase the memory limit, by increasing the value of the “WP_MEMORY_LIMIT” constant, by following the instructions at http://codex.wordpress.org/Editing_wp-config.php#Increasing_memory_allocated_to_PHP

    Regards,
    Tobias

    BeHeard

    (@beheard)

    11 years, 1 month ago

    Thanks so much
    Sorry yes 2 issues.
    The load for large files.
    And the memory upload for the large flies.
    PHP limit is 96 – will look over the link provided..

    the only error appeared after 100% upload with chrome showing
    HTTP Error 500 (Internal Server Error): An unexpected condition was encountered while the server was attempting to fulfil the request.

    Host checked no errors, no server load when the upload completed etc.
    Domain no error log for the action only the chrome.

    Thankyou so very much again.

    Plugin Author TobiasBg

    (@tobiasbg)

    11 years, 1 month ago

    Hi,

    ok, thanks for that information!

    So, yes, these are two different problems then.
    For the loading of the “Edit” screen, I’m investigating better ways, and rgeddes work on this is greatly appreciated and very helpful. I’m hoping to use this to make editing large tables easier and faster in the future.

    About the other error: The 500 error by Chrome means that an error happened on the server. It should be logged in the server logged. The best bet is the memory limit, so I suggest to try that. Otherwise, I’m out of ideas, unfortunately, especially if there’s really no info in the log files.

    Regards,
    Tobias

    BeHeard

    (@beheard)

    11 years, 1 month ago

    Thanks there was also the chrome error.
    The website encountered an error while retrieving http://sitenem.info/wp-admin/admin-post.php. It may be down for maintenance or configured incorrectly.

    Plugin Author TobiasBg

    (@tobiasbg)

    11 years, 1 month ago

    Hi,

    no, this is not a Chrome error. It is a server error that was just shown to you by Chrome. There should be an entry about this in the server logs.

    Regards,
    Tobias

Viewing 13 replies - 1 through 13 (of 13 total)
  • The topic ‘Table edit load/scroll time for 1250 rows x 12 colums’ is closed to new replies.