• Resolved garrettm30

    (@garrettm30)


    I’m still working through my migration plan from the previous bloated plugin to your nice solution, and I found an issue with large download files. I have one file that is about 215MB. (Because of the file size, it had to be manually uploaded via FTP.)

    I am able to get that file set up as a download in Simple Download Counter, but when a user clicks on it, the downloaded file has the correct name but is empty, and in the PHP error log, there is this error:

    PHP Fatal error:  Allowed memory size of 209715200 bytes exhausted (tried to allocate 169885696 bytes) in /local/path/mydomain.test/wp-content/plugins/simple-download-counter/inc/functions-core.php on line 387

    I also get this error at line 342, which makes sense, because I tried to do this both as a local and a remote download with the same trouble each time.

    The only idea I have for files such as this is simply to link directly to the resource without going through Simple Download Counter. Do you have another solution of something to do on my end?

    Or perhaps there is some improvement that could be made to the plugin? I don’t know if this is any real help, but I looked up the PHP docs for the readfile() function used at the two lines where I had the error, and I found this note that may be a clue:

    readfile() will not present any memory issues, even when sending large files, on its own. If you encounter an out of memory error ensure that output buffering is off with ob_get_level().

    https://www.php.net/manual/en/function.readfile.php

    I followed the suggestion to test with ob_get_level() to see if there is any output buffering, and it returned 2. So just as a test I threw in a quick while (ob_get_level()) ob_end_clean(); just above the readfile() and tried again. Then the large download seemed to work exactly as I hoped.

    Of course, I don’t presume to know what implications that might have for the rest of your code, but I hope it might be a good start, and maybe it is a good memory optimization even for files that can otherwise fit in allocated memory, because it apparently is reading in the whole file to memory before sending out headers.

    (Note: at present this is only on a local dev site. Using current WordPress 6.3.2 and Simple Download Counter 1.7)

Viewing 13 replies - 1 through 13 (of 13 total)
  • Plugin Author Jeff Starr

    (@specialk)

    This is interesting, will investigate and try uploading and downloading with large files. If there is anything that can be done, it should be available in the next update. In the meantime (just to be sure) can you let me know where (file and line #) you are finding readfile(), in case there is more than one instance used by the plugin. Thank you for the detailed report and your ideas @garrettm30.

    Thread Starter garrettm30

    (@garrettm30)

    No problem. I found it twice in inc/functions-core.php, once for locally hosted files at 342 and once for remotely hosted files at 387. I did not test the possible solution for remote files.

    Thank you for looking into this, especially considering this is a free plugin. That’s why I try to go as far as I can on my own before reporting.

    Plugin Author Jeff Starr

    (@specialk)

    Okay thanks @garrettm30, will check it out. In the meantime feel free to post again with any further information, questions, etc. Glad to help anytime.

    Thread Starter garrettm30

    (@garrettm30)

    I am really encouraged about using your plugin based on the way you have been responding, and it makes me want to do what I can to help (including a 5 star review once I get it in use on a live site).

    To contributed a little more information: I have now tested the suggested quick fix on remote downloads, and it worked there as well. Then I decided to test my theory about memory optimization even for smaller files by measuring memory on either side of the readfile() call, like this:

    $before_readfile = memory_get_usage();
    readfile($file_path);
    error_log(
      ( memory_get_usage() - $before_readfile ) . ' bytes additional memory'
    );

    I tested on a few files, and found that the memory usage is very close to the file size. For example, with that code applied to remote downloads, I downloaded a 4.5 MB file and got the result “4505624 bytes additional memory.” Then I tested adding the previously suggested line to empty output buffering just before the code above and got ‘0 bytes additional memory.’ I conclude that readfile() does read the whole file into memory in the current code, but this can be entirely solved by emptying output buffering just before reading the file. I don’t know if this is the best solution, but maybe there is no harm since you call exit; immediately afterward both times you read the file.

    So this could help on files that are not themselves too large but on environments where WordPress is already low on memory. Then certain files might put it over the threshold while others seem to work.

    Plugin Author Jeff Starr

    (@specialk)

    Nice work, I think you’re on to the solution, from what I’ve read around online so far. I’m thinking just something like this should do the trick for both instances of readfile():

    while (ob_get_level()) ob_end_clean();
    readfile($file_path);

    Of course will put it through the tests before implementing.

    • This reply was modified 7 months, 3 weeks ago by Jeff Starr.
    • This reply was modified 7 months, 3 weeks ago by Jeff Starr.

    Hello,
    I have nothing specific to add ( like PHP error log, because I *never* thought to check it ) just that I have the same problems with files of 207MB but I have no problem with a file of 107MB. Some people couldn’t download or get an empty file.
    I’ll patiently wait to see if a future update takes larger file into account.

    In the meantime, thank you very much for such a simple and straightforward plugin. It is perfect.

    Plugin Author Jeff Starr

    (@specialk)

    You are very welcome, @k1hippie. And thank YOU for leaving such a great review for the plugin. It really helps and is very much appreciated.

    I have a large series of pdf, epub, mp3, mp4 and zip files containing collections of the latter for downloading, the largest having almost 1 GB of data. The largest file I could get through your plugin has 131 MB. The next largest that failed has 156 MB (both are mp4 files). So the limit of what goes must be somewhere between the two sizes.

    The file that gets downloaded on fail has some 222 bytes and states, in a case where the file size to download is 938 MB (bloody hell, I now):

    “Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 966053888 bytes) in […]/wp-content/plugins/simple-download-counter/inc/functions-core.php on line 342”

    268435456 bytes means 256 MB, but I get the error already with a file of 156 MB as mentioned. So it allocates more than the file size itself. (at least 100 MB, it seems).

    Moreover, I have the free version of the plugin “Folders” by Premio installed (https://premio.io/downloads/folders/). It gives me an error message “failed to initialize plugin: sdc_download.” So there seems to be some conflict, although I have not noticed any issues so far. I will report that to them as well.

    Thread Starter garrettm30

    (@garrettm30)

    @germarrudolf Thanks for joining in on this discussion.

    268435456 bytes means 256 MB, but I get the error already with a file of 156 MB as mentioned. So it allocates more than the file size itself. (at least 100 MB, it seems).

    The 256 MB refers to the total memory allocated to WordPress. The memory is exhausted when a large file is read into memory in addition to all the other memory already in use by WordPress and all of its plugins. In other words, a lot of that memory was already allocated by your WordPress install before the file began to be read into memory. The 156 MB file was enough to put it over the limit.

    Moreover, I have the free version of the plugin “Folders” by Premio installed (https://premio.io/downloads/folders/). It gives me an error message “failed to initialize plugin: sdc_download.” So there seems to be some conflict, although I have not noticed any issues so far. I will report that to them as well.

    I can confirm that I encounter that error as well. I hadn’t yet decided to report it because in my case it happens on the product translation screen, and that depends on the two rather complex plugins WooCommerce and WPML (the latter of which is not free), and I judged it would be somewhat cumbersome to troubleshoot for a problem that so far (in my case) is just an oddity on the backend with no actual issue.

    However, I do not believe that it is related to the issue of memory exhaustion for large files. I say that because earlier in this thread we tracked down the root of the memory issue and found a solution, which I have implemented on my end as an interim solution while waiting on the update for an official solution. Since I can now download large files with no memory impact but I still see the error of “failed to initialize plugin,” I conclude it is unrelated.

    @specialk Since this unrelated problem is happening to others, maybe it is worth looking into. If I can help you to look into this or to provide you some more info, I will be glad to do so.

    Plugin Author Jeff Starr

    (@specialk)

    Thanks for the detailed information, definitely will be looking closer at this soon, as we head toward WP v6.5.

    Plugin Author Jeff Starr

    (@specialk)

    Just to follow up with this, the suggested fix has been implemented in the next version (1.8) of Simple Download Counter. Please re-test and let me know the results, thank you @garrettm30, @k1hippie, and @germarrudolf for your help and feedback with this issue.

    Thread Starter garrettm30

    (@garrettm30)

    Hi. As I mentioned in the other thread, I am only now just returning to this. I have tested version 1.8 with the large download that failed earlier, and I have found that the issue I reported is resolved. Thank you.

    The other issue that @germarrudolf mentioned regarding the message “Failed to initialize plugin: sdc_download” is not resolved 1.8. That is an unrelated issue, and since I have seen it reported a few other times, I will comment further in one of the other threads specifically about that issue.

    Plugin Author Jeff Starr

    (@specialk)

    Thanks for the report, @garrettm30. I am here to help however possible with the “Failed to initialize plugin” or any other issue. Let me know how I can help, thank you.

Viewing 13 replies - 1 through 13 (of 13 total)
  • You must be logged in to reply to this topic.