WordPress.org

Ready to get started?Download WordPress

Forums

Really Simple CSV Importer
server overload when importing (5 posts)

  1. hippypink
    Member
    Posted 7 months ago #

    I am only importing about 1,000 at a time. Each listing has about 30 fields, with 7 custom taxonomies (in some cases, 5 or 6 terms per taxonomy).

    My hosting company says its generating 300,000,000 (300 million) reads, and 22 database connections, so they disabled my site for taking over the server--its not even live yet.

    Im using IE
    ---------------
    Table rows read hit 301582810 > 300000000
    Current load: 23:40:45
    ---------------------

    P.S. Have you thought about adding ability to add some sort of wait command, or batching, to be able to upload large uploads on shared hosts?

    http://wordpress.org/plugins/really-simple-csv-importer/

  2. hippypink
    Member
    Posted 7 months ago #

    They also sent this statement that was running at the time:

    STATE: Creating tmp table
    TIME: 0
    COMMAND: Query
    INFO: SELECT t.*, tt.* FROM wp_terms AS t INNER JOIN wp_term_taxonomy AS tt ON tt.term_id = t.term_id INNER JOIN wp_term_relationships AS tr ON tr.term_taxonomy_id = tt.term_taxonomy_id WHERE tt.taxonomy IN (example') AND tr.object_id IN (34057) ORDER BY t.name ASC

  3. Takuro Hishikawa
    Member
    Plugin Author

    Posted 7 months ago #

    Interesting, why such a lot of read queries..I'd like to do test case of importing many taxonomy. But, I think importing 1000 posts in single run is quite a heavy job for shared hosts. You should divide your csv file (eg. each file has 100 posts). I don't have any plans to add ability to queue import job, because this plugin extends built-in importer of WordPress which has not queueing functions.

  4. hippypink
    Member
    Posted 7 months ago #

    This particular database is 11,000, so dividing into 100 blocks prob going to take awhile :)

    Interesting about the queuing stuff. Maybe I will develop one someday. I have another project that needs several hundred thousand posts imported :(

    Glad to see you are supporting your plugin well!

  5. Takuro Hishikawa
    Member
    Plugin Author

    Posted 7 months ago #

    Thanks a lot. In my project, I had been imported several thousands posts, but in that time, our WordPress were hosted on AWS and changed larger instance while importing. I think wp_get_object_terms function is too heavy, so I will try to reduce using it. Perhaps we need more lightweight API ;-)

Reply

You must log in to post.

About this Plugin

About this Topic

Tags

No tags yet.