WordPress.org

Ready to get started?Download WordPress

Forums

Add Multiple Users for WordPress
CSV/TXT Upload - email address in place of username (15 posts)

  1. bamajr
    Member
    Posted 2 years ago #

    Your plugin shows real promise, however I'm having quite a time making use of it. I have a few WordPress websites and an old email marketing system. I want to do away with the separate email marketing system and handle all communications through WordPress.

    I have an email list with a few hundred thousand users. The problem is the email marketing system didn't use username/password combinations it used email/password combinations.

    My website already supports email/password logins, but I can't use your plugin as it doesn't allow email addresses in the username field. I know the option for Email List will create usernames based off the emails address, but I'd rather import the email address as both the username and email address as I also have firstname/lastname data to import.

    Is their a simple fix to this?

    http://wordpress.org/extend/plugins/add-multiple-users/

  2. happynuclear
    Member
    Plugin Author

    Posted 2 years ago #

    Hi Bamajr,

    Thanks for the question - will be responding to your other post in a moment.

    I was just playing with the adding of an email address in the username field and it seems to accept email addresses as usernames both in the manual input and CSV section.

    Is the issue that you have the CSV data and want to use the same column of info, the email address, as both the username and the email and the Custom Column Order function does not have the facility for that? Or is it actually throwing you an error when you try to put email addresses in the username field?

    Cheers, look forward to your response!

  3. bamajr
    Member
    Posted 2 years ago #

    @happynuclear - Thank you for such quick responses.

    Like I said before, I have a list with a few hundred thousand users from an old system. I split the list up into smaller lists, with 50,000 lines per list. The list is already scrubbed and I put the list in the column order your plugin recommends.

    In the admin panel, I go to Users > Add Multiple Users > CSV/TXT Upload

    My CSV file is in the order: username, password, email, role, firstname, lastname, website

    When I try to upload the .TXT file, I get an error:

    Error: Not a valid file type! Only .csv and .txt files may be uploaded.

    When I convert the file to a comma delimited .CSV file, I do not get this error. I get taken to a page that says:

    The following user information has been extracted from your uploaded file.

    Please review your extracted CSV data in the text field below and, if necessary, customize the column order appropriate to your CSV data structure (see Custom Column Order information below).

    ...and my data is listed in the scroll-able box.

    However, when I click the button "Create User Information" I get taken to a page that says:

    Please review the user information below to correct any errors or to add additional information for each user and click the Add All Users button to process your new user accounts.

    ...and only one line is listed, and it is blank. There is two buttons that say "Add All Users" and "Add Extra Row" but none of my data has been populated.

    I've tried reducing the file to 25,000 lines and have the same issue.

    I continued testing by reducing the file to 10,000 lines, I get a table of populated data on the following screen, but it never finishes loading before the browser crashes (tried on both Firefox and Chrome).

    A 10,000 line .CSV file is only 905.8Kb so I'm not sure why it causes such problems. To me, importing anything less than 10,000 lines at a time, is useless.

    More testing effort was put in by reducing the file to 5,000 lines, and still, it never finishes loading before the browser crashes (tried on both Firefox and Chrome).

    A 5,000 line .CSV file is only 452.7Kb yet still there are problems.

    I finally got a different result, when I reduced the file again, to 2,500 lines, which is only 225.9Kb, the browser (both of them) stopped crashing, but both were extremely slow and sluggish. It would take hours to get to the bottom of a 2500 line list, via scrolling.

    When I click the "Add All Users" button, I get a screen that says "New User User Accounts Processed" at the top and "New User Account Registration Details:" at the bottom, but none of the user accounts were added.

    Only with a list of 1000 lines, do the browsers operate efficiently (or as expected). Yet, still, when I click the "Add All Users" button, I get a screen that says "New User User Accounts Processed" at the top and "New User Account Registration Details:" at the bottom, but none of the user accounts were added. (FYI: There is something wrong with the bottom "Add All Users" button, only the top one seems to work correctly).

    With list which is just under 1000 lines, it acts like it is working and goes through all the process of working, but never actually adds the users.

    I don't know what is going on, but I've tried this on two separate, clean installs of WordPress, and I'm getting the same/similar results. It just isn't working correctly with lists of 1000 lines or larger, and isn't worth the effort for lists less than 10,000 lines.

  4. happynuclear
    Member
    Plugin Author

    Posted 2 years ago #

    Hi Bamajr,

    Sorry for the slow response - it seems the forum ate my last post...

    I have been testing it out and my server, with a php memory limit of 64M (with script limit of 32M) seems to be able to do up to 550 registrations at once without running into the issues you list above. If I attempt to do anything over that I get the same thing as you - it says it has processed but nothing actually happens.

    I think without really being able to jack up the memory limit on the server its going to make it very difficult to do 10,000 at a time. I've requested an increase to test it out and make sure its the memory limit breaking it, but am yet to hear back from the host.

    There is another alternative, if you're running php 5.3 - a 'garbage collection' command can be invoked to clear out the memory. My server is running 5.2 so I can't test it out, but if you are running 5.3 I could create a modified version of the plugin and send it to you to test out.

    Sorry its being such a pain to use - honestly I didn't expect anyone to want to add so many accounts to a WP site at once, but obviously I underestimated that and do apologise. Hopefully we can find a solution.

  5. happynuclear
    Member
    Plugin Author

    Posted 2 years ago #

    After testing quite a bit it appears each loop to add a user is just doing way too much, so I'm gonna just rewrite the whole CSV thing for the next version, with the eliminate duplicate lines function built in and an option to skip the verification screen for larger files. I think that's going to be the only way to make it work with such high volumes of users.

  6. happynuclear
    Member
    Plugin Author

    Posted 2 years ago #

    Alright, I cracked it. Yay.

    Am doing some further testing and code clean up then will update to 1.2.2 version soon.

    After you upload your file, after the file contents have been added to the scroll box, there will be another button to skip the form and just straight add the users. This is a similar but different function that bypasses the creating of form lines on the page. As it progresses through each line of the CSV, it will find matching entries with usernames/emails and ignore them, giving a simple error saying that user or email already exists, and move on to the next line.

    Because it takes a while (I just added 10000 users to my blog!) it does require this line:

    set_time_limit(0);

    added to the wp-config.php file above the 'that's it, happy blogging' line - this is to stop WordPress from cancelling the function loop after its default 30 secs. It did take a good five mins of the browser 'thinking' before the adding of 10,000 was complete, but it worked nonetheless.

  7. happynuclear
    Member
    Plugin Author

    Posted 2 years ago #

    Ok, version 1.2.2 should be up in a sec. Please give that a try and let me know how it works for you!

    Cheers.

  8. bamajr
    Member
    Posted 2 years ago #

    @happynuclear - Its great to see such attention given to a support issue. Thanks! Hopefully I'm not the only one who will use your plugin to import large lists, but if I am, its nice to know it can handle it.

    In any event, I wanted you to know my server/php config, in case that helps your development in the future.

    Linux 2.6.32-5-amd64
    PHP Version 5.3.3-7+squeeze3 w/:
      max_execution_time: 120 & 120
      max_file_uploads: 20 & 20
      max_input_nesting_level: 64 & 64
      max_input_time: 120 & 120
      memory_limit: 128M & 128M
      open_basedir: no value & no value
      output_buffering: 4096 & 4096
      output_handler: no value & no value
      post_max_size: 8M & 8M
      precision: 14 & 14
      realpath_cache_size: 16K & 16K
      realpath_cache_ttl: 120 & 120

    Now I'll go and try your new release and see what kind of result I get ;-)

  9. happynuclear
    Member
    Plugin Author

    Posted 2 years ago #

    You're welcome, I hope it works out! Not much of a bulk function if it doesn't work in bulk!

    Thanks for the php info too, that definitely will come in handy.

    Make sure you do the set_time_limit thing before you try adding heaps of users - when I added 10,000 it took a while to power through it!

  10. bamajr
    Member
    Posted 2 years ago #

    Ouch...

    First try to add 9,999 users and I get a "500 Internal Server Error"

    Internal Server Error
    
    The server encountered an internal error or misconfiguration and was unable to complete your request.
    
    Please contact the server administrator, [no address given] and inform them of the time the error occurred, and anything you might have done that may have caused the error.
    
    More information about this error may be available in the server error log.
    
    Apache/2.2.16 (Debian) Server at nacits.com Port 80

    Upon further investigation, it loaded 1604 new users before crashing, which is way more than it did yesterday, but still doesn't seem to want to do 10,000 at a time.

  11. bamajr
    Member
    Posted 2 years ago #

    On the Second attempt I tried to add the difference of the original 9,999 users and the 1604 which actually loaded (8395 users).

    Again I get a "500 Internal Server Error"

    Internal Server Error
    
    The server encountered an internal error or misconfiguration and was unable to complete your request.
    
    Please contact the server administrator, [no address given] and inform them of the time the error occurred, and anything you might have done that may have caused the error.
    
    More information about this error may be available in the server error log.
    
    Apache/2.2.16 (Debian) Server at nacits.com Port 80

    Upon further investigation, it loaded 1487 new users before crashing. Again, this is way better than yesterday, but doesn't look like a viable option for 10,000 users at a time.

    I know I can import a .CSV file through phpMyAdmin, so maybe that is the better option.

    @happynuclear - anything else you can think of to make this work in your plugin?

  12. bamajr
    Member
    Posted 2 years ago #

    In an effort to be thorough, I tried a 3rd time. This time I tried to add only 1499 users.

    Again I get a "500 Internal Server Error"

    Internal Server Error
    
    The server encountered an internal error or misconfiguration and was unable to complete your request.
    
    Please contact the server administrator, [no address given] and inform them of the time the error occurred, and anything you might have done that may have caused the error.
    
    More information about this error may be available in the server error log.
    
    Apache/2.2.16 (Debian) Server at nacits.com Port 80

    This attempt loaded 1325 new users before crashing.

  13. bamajr
    Member
    Posted 2 years ago #

    In my 4th and final attempt, I used a list of 999 users.

    At this point, I get the success messages for each of the 999 users (even the dupes). So, it looks like this plugin and they way it uses PHP is going to be limited for most people at 1000 users at a time or less.

    If you looked at the PHP info I posted above, my PHP limits are way above average.

    I know I didn't post the hardware specs of this machine, as I'm positive they are not even close to being part of the issue.

    Dual Quad-Core 2800 MHz Processors
    16GB DDR2 RAM
    4 1.5TB Hard Drives

    During testing, only the one website, with WordPress and this plugin are installed/running, so it isn't putting too big of a load on the processors and it isn't consuming too much ram.

    The .CSV file size is no where near the WordPress/PHP 8MB file size limit.

    In any event, I'm still giving this plugin a 5-Star Recommendation and a "Works" vote, as it was never intended to support such large lists.

    @happynuclear - If you need anything else from me, this is pretty much what I do all day (write HTML/CSS, evaluate code, test/re-test & modify).

    I manage my own hosting environment and I do all my own server administration and network admin. The hosting environment is a Cloudmin/Virtualmin/Webmin powered LAMP Stack. This gives me the ability to quickly load any number of web systems, in a matter of seconds.

    If you want to test this further, just let me know.

  14. happynuclear
    Member
    Plugin Author

    Posted 2 years ago #

    Hi Bamajr,

    Sorry for the delay in response - problem with being Australian is I'm asleep while the rest of the world is working!

    So sorry it hasn't worked - I didn't encounter any such error when I tested up to 10,000 - and yes your system should definitely be able to handle it, so I guess my php adding process is probably way too bloated to be feasible at such high volumes. The 500 error is really bad - dammit.

    I'm gonna spend a little time doing some rewrites and hopefully nail down the problem. It'll probably take a little time but I'll get it there.

    Thank you very much for taking the time to test it out thoroughly, its a great help in getting it working properly, and yes if you're interested in testing the next update before it gets released I'd definitely take you up on that!

  15. happynuclear
    Member
    Plugin Author

    Posted 2 years ago #

    Ok another test time, if you're willing!

    In the Skip Form function I have replaced the wp_create_user and update_user_meta functions with the insert_user function and a couple of direct additions to user meta data for multisite.

    I am hoping this will create less load on the function - it worked for me adding 10,000 users, but then, the last one did as well and did not work for you.

    The real issue with going a step further than this is that I don't want to bypass WordPress functions like 'insert_user' and go straight to database inserting just with sql. I want WordPress to handle it, however, if it has to be done, it has to be done!

    If you'd like to test it out, visit this page:

    http://www.happynuclear.com/sandbox/amu/add-multiple-users-for-wordpress.php

    And below the Download link is a link to the beta test version. Just grab this file, replace multiadd.php in the plugin folder with this new version and give it a shot!

Topic Closed

This topic has been closed to new replies.

About this Plugin

About this Topic