222-record report missing very last value

jsafire
(@jsafire)

10 years, 5 months ago
Hi –

My latest report I created outputs Lastname,Firstname. All looks great – except, in the CSV, TSV, and JSON downloads, the Firstname in the last record is missing all but the first letter. It does show up in the XML download.

222 records

The same happens in my Firstname,Lastname,Email report.
This is repeatable every time.
Any ideas?
```
SELECT pm1.meta_value AS Lastname , pm2.meta_value AS Firstname
FROM wp_k8hk02qrxa_posts
LEFT JOIN wp_k8hk02qrxa_postmeta AS pm1 ON ID = pm1.post_id
LEFT JOIN wp_k8hk02qrxa_postmeta AS pm2 ON ID = pm2.post_id
WHERE (post_type = 'members'
AND post_status = 'publish'
AND pm1.meta_key = 'wpcf-last-name'
AND pm2.meta_key ='wpcf-first-name')
ORDER BY Lastname
```
Thx,
Jeff

https://wordpress.org/plugins/exports-and-reports/

Viewing 15 replies - 1 through 15 (of 20 total)

1 2 →

Plugin Author Scott Kingsley Clark
(@sc0ttkclark)

10 years, 5 months ago

Oh that’s a very interesting issue I’ve never seen before. I’ll have to look into this and see if I can confirm this is happening. Is this the same with the default post export+report that comes with the plugin? Is this every report? Or just this query?
Thread Starter jsafire
(@jsafire)

10 years, 5 months ago
Hi Scott –
This is occuring on my 4 of 4 reports. I tried simplifying the SQL but it still lost the last term. It also has the same problem with your Examples Post Report. I have posted the results of that CSV file below.

I wonder if this is a server or PHP setup problem. Likely not a memory limit problem with such a small (Lastname,Firstname) query but, perhaps something else?
The site is running on ngnx at getflywheel.com host.
```
"Post ID ","Post Title","Post's Slug","Post Date","Post Content"
13647,"Auto Draft",,2015/12/08,
13667,"Auto Draft",,2015/12/10,
6506,"Effect of RoHS on the US Medical Device Industry",effect-of-rohs-on-the-us-medical-device-industry-2,2014/04/18,
3881,"FCC signals intent to allow Internet ""fast lanes""",fcc-signals-intent-to-allow-internet-fast-lanes,2014/04/24,
6401,"Highlight of Super Mobility Week in Las Vegas",highlight-of-super-mobility-week-in-las-vegas,2014/09/17,
6525,"Elon Musk’s Next Mission: Internet Satellites",elon-musks-next-mission-internet-satellites,2014/11/12,
6600,"C++ 14 has been finalized",c-14-has-been-finalized,2014/11/21,
6628,"Intelligent Hospitals are Harnessing High Tech Developments",copy-of-intelligent-hospitals-are-harnessing-high-tech-developments,2014/11/24,
6655,"Wave of Server Breakin Attempts from Hong Kong",wave-of-server-breakin-attempts-from-hong-kong,2014/12/09,
6738,"The Top Technology Failures of 2014",the-top-technology-failures-of-2014,2015/01/06,
9918,"Hollywood Hands Engineer an Oscar",hollywood-hands-engineer-an-oscar,2015/02/14,
11128,"For New Consultants: How to get the most out of an IEEE-CNSV meeting",for-new-consultants-how-to-get-the-most-out-of-an-ieee-cnsv-meeting,2015/04/13,
11306,"IEEE Region 6 Recognizes CNSV as Outstanding Chapter",ieee-region-6-recognizes-cnsv-as-outstanding-chapter,2015/04/17,
11367,"Mountain View-Based Google Launches New Wireless Service -- Project Fi",mountain-view-based-google-launches-new-wireless-service-project-fi,2015/04/22,
11373,"White Paper: Marketing Your Engineering Consulting Services Using Craigslist",white-paper-marketing-your-engineering-consulting-services-using-craigslist,2015/04/22,
11606,"Tips for Blogging on the New CNSV Website",tips-for-blogging-on-the-new-cnsv-website,2015/05/11,
11793,"What is different about Big Data?",what-is-different-about-big-data-2,2015/06/01,
11636,"How To Add the New CNSV Logo to Your Site",how-to-add-the-new-cnsv-logo-to-your-site,2015/06/02,
11857,"Google using 60 GHz sensors to control smart devices",google-using-60-ghz-sensors-to-control-smart-devices,2015/06/11,
11863,"Three things I learned about the history of Silicon Valley at the CNSV Dinner",three-things-i-learned-about-the-history-of-silicon-valley-at-the-cnsv-dinner,2015/06/12,
12399,"Disney bringing awesome robots to Disneyland!",disney-bringing-awesome-robots-to-disneyland,2015/07/03,
12535,"Google Drive plugin for Microsoft Office lets you ‘use the apps you’re already comfortable with’",google-drive-plugin-for-microsoft-office-lets-you-use-the-apps-youre-already-comfortable-with,2015/07/21,
12560,"Some Tips For Updating Your LinkedIn and CNSV Profile",some-tips-for-updating-your-linkedin-and-cnsv-profile,2015/07/23,
12551,"Recent video ""Wearable Devices - Just a Fad?"" by member, Walt Maclay",recent-video-wearable-devices-just-a-fad,2015/07/28,
12633,"Electrical Engineering Employment Down 29% from 2006 to 2014",electrical-engineering-employment-down-29-from-2006-to-2014,2015/08/04,
12734,"CNSV's Brian Berg Talks With Flash Memory Inventors at Flash Memory Summit About Bob Norman's Contributions",cnsvs-brian-berg-talks-with-flash-memory-inventors-at-flash-memory-summit-about-bob-normans-contributions,2015/08/19,
12731,"The FFT is Fifty",the-fft-is-fifty,2015/08/24,
13111,"A Mission Controller Reviews “The Martian”",a-mission-controller-reviews-the-martian,2015/10/02,
13183,"IEEE Members Vote Karen Bartleson 2016 IEEE President-Elect",ieee-members-vote-karen-bartleson-2016-ieee-president-elect,2015/10/09,
13283,"FAQ: Some Attorney Recommendations for Consultant",faq-some-attorney-recommendations-for-consultant,2015/10/23,
13614,"Apple's iPhone, iPad Secret Sauce - Semiconductors",apples-iphone-ipad-secret-sauce-semiconductors,2015/1
```
As you can see, the post_date in the last post is incomplete. It should be 2015/12/07 as it displays appropriately in the formatted on-screen output.

I will try on a different server.

Thanks,
Jeff
vkotek
(@vkotek)

10 years, 4 months ago

Hello, I am experiencing the same issue; a few characters missing at the end of the CSV export. I am dependent on this feature, so a fix would be really appreciated! (it is a fantastic plugin!)

PS: I also have two empty rows in each CSV export, I’m not sure if this is intentional or perhaps related.

Thank you,
Vojta

Plugin Author Scott Kingsley Clark
(@sc0ttkclark)

10 years, 4 months ago

What about TSV exports, same issue?

Also try Pipe / XML / JSON too, just to see if we’re looking at a bigger bug.

vkotek
(@vkotek)

10 years, 4 months ago

It is happening across all the export formats for me.

The reason XML contains all the values is because it does not trim end last few values, but characters of the actual file. So for XML, the last line is “</it” instead of “</item>”.

The JSON API isnt working for me either, loads a blank screen, it could be because of the missing characters too.
Plugin Author Scott Kingsley Clark
(@sc0ttkclark)

10 years, 4 months ago
Wow so that’s weird!

For writing files, I’m using fopen / fwrite / fputcsv / fclose depending on what file format.

I know that there’s a WP_Filesystem class, not sure it covers fputcsv though which is probably why I kept it as it was.

When you download the file from FTP, is it the same issue?

When downloading through the plugin, we attempt to stream the file out via:
```
// required for IE, otherwise Content-disposition is ignored
	if ( ini_get( 'zlib.output_compression' ) ) {
        ini_set('zlib.output_compression','Off');
	}

    header("Pragma: public"); // required
    header("Expires: 0");
    header("Cache-Control: must-revalidate, post-check=0, pre-check=0");
    header("Cache-Control: private",false); // required for certain browsers
    header("Content-Type: application/force-download");
    header("Content-Disposition: attachment; filename=\"".basename($file)."\";" );
    header("Content-Transfer-Encoding: binary");
    header("Content-Length: ".filesize($file));
    flush();
    readfile("$file");
    exit();
```
vkotek
(@vkotek)

10 years, 4 months ago

Your suspicion was right, it seems that the FTP files in the /exports folder are in fact complete!

I’m using Chrome but get the same issue with IE. The server is Win Server 2008.

Are you not able to replicate this error?

Plugin Author Scott Kingsley Clark
(@sc0ttkclark)

10 years, 4 months ago

I wonder if you’re running out of memory possibly, for whatever reason.

Try increasing it:

https://codex.wordpress.org/Editing_wp-config.php#Increasing_memory_allocated_to_PHP

vkotek
(@vkotek)

10 years, 4 months ago

Increasing the limit did not help.

It seems that there are 3 characters missing at the end of the file regardless of the export size and format.

I’m a total php noob but maybe the filesize() passed in the headers is lower then it should be? I would not be surprised if this is the fault of the IIS and Win server, I’ve had so many problems with it at every step.

Plugin Author Scott Kingsley Clark
(@sc0ttkclark)

10 years, 4 months ago

Do you have non-english characters in the exports?
Plugin Author Scott Kingsley Clark
(@sc0ttkclark)

10 years, 4 months ago
You also might consider adding this for a temporary workaround —

Add to /wp-content/plugins/exports-and-reports/wp-admin-ui/Admin.class.php after line 20:
```
wp_redirect( str_replace( WP_ADMIN_UI_EXPORT_DIR, WP_ADMIN_UI_EXPORT_URL, $file ) );
    die();
```
vkotek
(@vkotek)

10 years, 4 months ago

Yes, there are Czech characters (latin-extended)

PS: just tried an export that includes only English characters and still missing the 3 characters at the end.
vkotek
(@vkotek)

10 years, 4 months ago
I added the line as shown below but then the popup with the download just remains opened and now download happens.
```
if ( ! isset( $_GET['export'] ) || empty( $_GET['export'] ) || ! file_exists( $file ) ) {
		wp_die( 'File not found.' );
	}

	wp_redirect( str_replace( WP_ADMIN_UI_EXPORT_DIR, WP_ADMIN_UI_EXPORT_URL, $file ) );
    die();

    // required for IE, otherwise Content-disposition is ignored
	if ( ini_get( 'zlib.output_compression' ) ) {
        ini_set('zlib.output_compression','Off');
	}
```
Thread Starter jsafire
(@jsafire)

10 years, 4 months ago

I still see this problem on my 4 reports as well as your Example Posts Report – and I have only English characters. My site is running on Linux/nginx – quite different from vkotek’s setup with the same problem. I don’t think memory is an issue as I have other scripts outputting a lot more data.

vkotek: I am displaying all of these files (for this debugging) in a programmer’s editor (bbedit) so all characters are displayed and I can see exactly what is missing.

I have at least one report export file that is missing 4 characters at the end.

I confirmed via ftp that the file is actually all there and complete.
Is it possible to use one of the direct URLs shown on Exports and Reports Settings page as a temp workaround and give my “vice-admin” an ftp://… link?

Jeff

Plugin Author Scott Kingsley Clark
(@sc0ttkclark)

10 years, 4 months ago

That should work as a workaround.

I’m still digging through the possible issues here, I haven’t landed on any solutions yet but it appears so far it’s something with reading the file and sending it to force-download in the browser.

Viewing 15 replies - 1 through 15 (of 20 total)

1 2 →

The topic ‘222-record report missing very last value’ is closed to new replies.