CSV import problems

Well, I am through a lot of tests, still no luck.

1. I streamlined the header line and removed a unused column
2. I changed the delimiter from semicolon to comma
3. I added double quotes as delimiter
4. I tried with ‘Auto’ settings but also with dedicated specification
5. I removed the email addresses
6. Finally I removed even any spaces 🙂

Sometimes it works fine a few times in a row, then it doesn’t work again.
Now I’m a bit clueless.

regards, Peter

Hi Roland,
I have installed a transient plug-in to see anything that I wanted never to see 🙂
After removing all non-persistent transients I imported again, with no success.
BTW, I changed all my test changes back to the prevoius state as mentioned in my first posting.

So here again is my test record, settings are Auto/Auto, duplicate record preference=dont import the record, duplicate record check field=email

hull_number;ship_name;year_built;location___city;country;owner;description;data_source;status;email
1751;Rigoletto;1973;Down Cruising Club;United Kingdom;Michael McDermott;RIGOLETTO (formerly Elkedor) moved from Fareham to Strangford;VEGADB;imported;mjmcd1@btinternet.com

Now two things are interesting.

1.
The import imported all columns except “hull number” and “email”, and in the list view there is no “private ID”. But when I dive into the edit view, there is a “private ID” – which is already in the database and used by another older record.

(cant’t attach screenshots here, but I have some)

2.
After the failed import there is a transient called ‘participant_cache_4’.
This is the content (I formatted it a bit to be more readable, its one long string)

a:1:{i:446;O:8:"stdClass":30:{
s:2:"id";s:3:"446";
s:10:"private_id";N;
s:7:"country";s:14:"United Kingdom";
s:5:"email";N;
s:12:"mailing_list";s:2:"No";
s:5:"photo";N;
s:7:"website";N;
s:9:"interests";N;
s:8:"approved";s:2:"no";
s:13:"date_recorded";s:19:"2022-03-07 08:55:57";
s:12:"date_updated";s:19:"2022-03-07 08:55:57";
s:13:"last_accessed";N;
s:16:"last_update_user";N;
s:11:"hull_number";N;
s:5:"owner";s:17:"Michael McDermott";
s:9:"ship_name";s:9:"Rigoletto";
s:15:"location___city";s:18:"Down Cruising Club";
s:15:"location___area";N;
s:11:"data_source";s:6:"VEGADB";
s:6:"status";s:8:"imported";
s:10:"year_built";s:4:"1973";
s:14:"more_equipment";N;
s:13:"delete_record";N;
s:6:"engine";N;
s:11:"description";s:61:"RIGOLETTO (formerly Elkedor) moved from Fareham to Strangford";
s:11:"hide_record";s:3:"Yes";
s:14:"previous_names";N;
s:12:"current_flag";N;
s:14:"owner_imported";N;
s:14:"email_imported";N;
}}

If I see it right, it says the following:
private id, email and hull number are treated as ‘not in CSV list’, which of course is not true.
The last two colums, owner_imported and email_imported, are no longer in the database, I added it for testing and removed it afterwards. They will surely be in the column table but are not active anymore.

This points into a direction where caching comes into mind?

I hope my findings are of some help, if needed you can have access to my site, it’s just a testbed for the time being.
Another thing I will try is to set up a quick test environment on my personal linux server where I have more control than on my provider’s server.

best regards, Peter

Hi Roland, my problem still persists, but as I mentioned before I was able to delete the corrupt entries and try the import again until all columns are imported correctly.

I just updated to 2.0.8, but I have still a second error that is more than ugly:
The imported private IDs are all the same (in my case “RPNE2”).
I use the addition to expand the private ID to 9 characters, so normal registration IDs look like “IS5T1U0F4”.

Even if I say that private ID is the duplicate record check field and the preference is not to import these records, the import creates duplicates, all with private ID “RPNE2” 🙁

Do you have any idea where I could look after this?

I am very sorry by not being clear with my description 🙁

I do not import private IDs, I expected that they were generated as it is the case when I register a single entry using the registration form.
In my CSV file are just the columns I mentioned before, i.e hull_number, owner_name, year_built, email_address and so on.

A few weeks ago these private IDs have been generated as expected, but now they all have the same value. When I find the time I will set up another test case on my own server so I can better look into it and maybe extend the debug output a bit.

Hi Roland,

I have found the following workaround(s):
I set up a new temporary column for the hull number (not read-only, of course), so now it will be imported in every row. After the import I simply copy the values over using SQL.
That’s works around problem #1.

For the private ID I generate one into my CSV file, because it gets generated itself using Java this is very simple. On that the duplicate check and skip works well.
That works around problem #2.

I will try your suggestion by removing the default value, this maybe would be easier than my solution.

On the other hand I have set up a debugging environment using Netbeans with xdebug, but I have to find time to dig into it. It will take some time, sorry.

brgds, Peter