• I’ve used RSCI successfully a few times, but this last attempt is the first time I’ve tried to import an image for each post, and it hasn’t worked so far. Here’s what I’ve done so far:

    1) Tried to import 300+ posts with 8 fields, two of them ACF fields. After waiting a long time I checked how many posts had been generated and saw there were 71 new posts, and that there were corresponding images for each of them, except the last image was in the media library, but was blank. I was using Safari on the Mac, although I don’t think that’s a factor. I know that it was finished because the time of the last post was hours earlier – all the posts that been generated in a short time period, and then hours went by before I went to check why it wasn’t doing anything.

    I noticed as well that the custom fields had not been imported. I thought perhaps it was because I had created new categories and forgotten to add them to the ACF field group, although that seemed strange (i.e. the fields exist even if they’re not shown, so why wouldn’t they get imported?). I added the new categories to the field group.

    2) I split my CSV into 50-line files (and copied over the header line for each one obviously). I tried importing and this time it stopped at 41 posts. Same thing with the most recent image being blank.

    The two custom fields still had not been imported. I finally realized the problem – one of the fields was a Select field and I had changed the values (i.e. I was importing values that did not exist in the Select options). I switched the field to a text field so it wouldn’t be an issue.

    3) I moved over to Chrome just in case that mattered. I split the CSV into 25-line chunks and imported the first file. This time I got a 504 Gateway Time-out. I had checked how many posts were generated a minute earlier and it was up to 21, so I expected it to finish about then anyways. What’s really odd is that when I went to check how many posts there were, there were 41 – and yet there were only 25 in the CSV. Did it somehow run a repeat of the previous import? No idea. Doesn’t seem it could be a caching issue since the name of the file was different.

    The custom fields did get imported this time. The odd thing is that if one custom field can’t be imported, why did both fields not get imported in previous attempts?

    4) Thinking perhaps it might be connected to the fact I use Cloudflare, I switch Cloudflare to Development Mode and tried exactly the same thing as before (25 rows in Chrome). This time it seemed to import 50 posts, but never timed out. After enough time went by I stopped the import and looked at the 50 new posts. What I noticed was that it was not 50 unique posts, but 25 posts twice. The second set of 25 was not in exactly the same order, although close.

    Not sure what to learn from this round, although there were 50 complete images in the Media Library, but they were not attached to any posts yet. Not sure what the order that the plugin works in, but I’m guessing the images are imported and only later assigned to the appropriate posts.

    5) Disabled all the caches in W3, and moved over to Firefox just for good measure (I know the browser likely had no effect, but I figured it couldn’t hurt). This time I got an Error 520 in Cloudflare saying the site was offline. I don’t know exactly when that happened, as I wasn’t watching it. I opened the site again and it worked fine. I checked out the posts and there were indeed 25 new ones this time. Everything seems to have worked correctly, even though there was the error at the end. Still, it’s only 25 rows, and I have a few hundred more to do.

    https://wordpress.org/plugins/really-simple-csv-importer/

Viewing 13 replies - 1 through 13 (of 13 total)
  • Thread Starter philipt18

    (@philipt18)

    6) Moved back to Safari and tried importing another 25. Same Error 520 from Cloudflare, but this time I got 50 again. Duplicate records and unattached images.

    Also, just for the record I delete all the imported posts and images in between attempts (except for the 25 that actually worked in step 5).

    Any ideas what is going on?

    Thread Starter philipt18

    (@philipt18)

    7) Moved back to Firefox. Tried importing the second 25 again. Cloudflare Error 520 again. This time I was there when it happened and the first few times I tried loading the site, it didn’t work. Makes me think the site crashed and was rebooting.

    Loaded the site finally, and found 25 posts, 25 images attached, and oddly the CSV file as an unattached media file. Otherwise things seemed okay. No idea why Firefox seems to be working better than the other browsers, even if it’s not perfect.

    Thread Starter philipt18

    (@philipt18)

    Weird, I went back to delete the CSV file from the Media Library and it was not there anymore.

    Thread Starter philipt18

    (@philipt18)

    8) Tried a CSV with 50 records in Firefox. Generated 56 posts, but it’s not 50 + 6 duplicates, as every record I checked seems to have a duplicate. It’s hard to figure out what is going on. Were 28 records imported and then something went haywire and imported those same records a second time in different order? This is so weird.

    Okay, this is interesting. There are two copies of the CSV in the unattached section of the Media Library. Their IDs are 21 apart. There were 56 images, all attached, and two of them, for the same imported item (but different posts), were blank.

    Thread Starter philipt18

    (@philipt18)

    9) Tried a CSV with 25 records in firefox. Generated 50 posts and 50 attached images. No CSV files left unattached like last time. Perhaps they were left last time because the process didn’t finish and this time it did so the CSV were deleted.

    Thread Starter philipt18

    (@philipt18)

    I realized that in attempts 8 and 9 the Cloudflare development mode had expired. There’s also some kind of caching mode on my host that I didn’t remember to turn off. Not sure what exactly it does, but it has a specific mode for WordPress caching, which I turned off.

    10) Ran the third 25-row csv in firefox again, and while I still got the Error 520 at the end, everything seemed to be imported properly.

    11) Ran another 25-row csv file in the same way, but this time got 50 posts and images. That’s annoying since there was no difference between these two attempts.

    Plugin Author Takuro Hishikawa

    (@hissy)

    did you check the PHP max execution time value?

    Thread Starter philipt18

    (@philipt18)

    No, I didn’t. How would that cause duplicate CSV files and imports? It’s currently set to 120 seconds. I’ll set it to 240 and see what happens, but any other ideas of what might cause this to occur?

    Thread Starter philipt18

    (@philipt18)

    That helped. I ran a csv with 50 items and it worked. I then ran another 50, and I saw a small problem. 9 out of the 50 posts didn’t get their thumbnails imported. It turns out I had the wrong filenames in the csv for those posts, so it’s my fault, but I bring it up because there were no error messages when the csv was processed. If a thumbnail can’t be found it would be very helpful to see some kind of error message.

    Unfortunately, when I tried to import another 50 after I fixed the 9 posts without thumbnails, I ran into the duplicate csv problem again and got about 90 posts before I cancelled it.

    Thread Starter philipt18

    (@philipt18)

    Tried again and at one point there were actually 3 CSV files. It actually generated triple (150) posts this time instead of just double.

    Increased max execution time to 1200. Ran a CSV with 50 posts and it worked. I guess that was the problem. It still gives me an Error 520 at the end of the process, however. Not sure what the effect of leaving such as long execution time in my php settings will do, although I guess I can shorten it after I finish importing my posts.

    Thread Starter philipt18

    (@philipt18)

    Hmmm. Got triple posts on the next attempt. Would appreciate additional suggestions.

    Thread Starter philipt18

    (@philipt18)

    Okay, it’s just a theory, but I noticed in the last CSV file that some of the locations of the thumbnails were wrong. Basically I changed some filenames and didn’t update the CSV. Is it possible that RSCI is doing too much work when it can’t find an image, and that is triggering the problem here? Unfortunately, in the past one of the attempts worked even when some of the files were missing, so this isn’t necessarily the answer.

    Even so, perhaps it might be useful to do a check for the existence of the image file before trying to import it, if you don’t already. I’ll do some more testing to see what I can determine.

    Thread Starter philipt18

    (@philipt18)

    I think it’s more than a theory, but it’s a bit more complicated. I got things working, but the import page was always timed out. Finally, the last import which had less than 20 records, finished properly and showed the results of the import. I think there are two things going on here:

    1) When there are missing images, perhaps it takes more than one, for some reason a second (or even third) copy of CSV is loaded and duplicate posts imported.

    2) When the entire process takes too long, the import page times out, but that doesn’t seem to effect the import process.

    I don’t know where the cutoff is for the timeout, probably somewhere between 25 and 50 imports that have thumbnails. It could also have to do with the size of the images used as thumbnails. I’m using ~1200 pixel wide images that I extracted from PDFs using Imagemagick. The files tend to be between 250-500KB.

    I also have no idea what triggers the duplicate CSV files. It’s possible that it has to do with the size of the thumbnail files or some other random factor. Although I could guess where it happened (by the sequence of ID numbers of created files) I didn’t actually pay much attention to that when I was trying to get things working, and didn’t record which files came before and after the creation of duplicate CSV files.

    That’s about all I can say right now as the posts I was working on are now all imported. I’ll likely need to do more in the future, but not too soon. I hope my observations help pinpoint what is going wrong, and will help improve the plugin in the future.

Viewing 13 replies - 1 through 13 (of 13 total)
  • The topic ‘Import stops in the middle – some troubleshooting’ is closed to new replies.