• Resolved Matt

    (@reallymattgray)


    Admittedly, it’s 302,599 files (because there is no option to exclude directories… which are the other sites… that have their own WF installs already).

    I have even increased the requested memory to 450 MB, but none of it works.

    The scan always hangs up after the 302,599 and after scanning the contents of another 101 files. (cf. log results below). I eventually have to kill the scan because it hangs up for hours.

    The only option to exclude directories is from modification alerts. Yes, I’ve added the 119 website directory names, comma separated, to the exclusion line, but it makes no difference.

    The site scans clean by Securi SiteCheck, and anti-malware. But nothing form WordFence.

    Any more ideas?

    [Mar 21 00:13:54:1490055234.158552:1:info] Scan kill request received.
    [Mar 20 20:52:22:1490043142.314259:2:info] Scanned contents of 101 additional files at 1.10 per second
    [Mar 20 20:52:20:1490043140.927923:2:info] Scanned contents of 98 additional files at 1.09 per second
    [Mar 20 20:52:15:1490043135.513711:2:info] Scanned contents of 97 additional files at 1.14 per second
    [Mar 20 20:52:06:1490043126.283872:2:info] Scanned contents of 96 additional files at 1.27 per second
    [Mar 20 20:52:01:1490043121.013406:2:info] Scanned contents of 95 additional files at 1.35 per second
    [Mar 20 20:51:56:1490043116.504042:2:info] Scanned contents of 94 additional files at 1.43 per second
    [Mar 20 20:51:52:1490043112.114759:2:info] Scanned contents of 93 additional files at 1.51 per second
    [Mar 20 20:51:46:1490043106.085352:2:info] Scanned contents of 92 additional files at 1.66 per second
    [Mar 20 20:51:42:1490043102.298750:2:info] Scanned contents of 91 additional files at 1.76 per second
    [Mar 20 20:51:38:1490043098.972896:2:info] Scanned contents of 90 additional files at 1.86 per second
    [Mar 20 20:51:35:1490043095.177136:2:info] Scanned contents of 89 additional files at 2.00 per second
    [Mar 20 20:51:31:1490043091.000530:2:info] Scanned contents of 88 additional files at 2.18 per second
    [Mar 20 20:51:25:1490043085.662343:2:info] Scanned contents of 87 additional files at 2.48 per second
    [Mar 20 20:51:21:1490043081.471309:2:info] Scanned contents of 86 additional files at 2.79 per second
    [Mar 20 20:51:18:1490043078.355760:2:info] Scanned contents of 85 additional files at 3.07 per second
    [Mar 20 20:51:16:1490043076.094631:2:info] Scanned contents of 84 additional files at 3.30 per second
    [Mar 20 20:51:12:1490043072.612678:2:info] Scanned contents of 83 additional files at 3.78 per second
    [Mar 20 20:51:05:1490043065.349223:2:info] Scanned contents of 82 additional files at 5.57 per second
    [Mar 20 20:51:00:1490043060.777514:2:info] Scanned contents of 81 additional files at 7.99 per second
    [Mar 20 20:50:57:1490043057.321843:2:info] Scanned contents of 80 additional files at 11.97 per second
    [Mar 20 20:50:54:1490043054.451100:2:info] Scanned contents of 79 additional files at 20.72 per second
    [Mar 20 20:50:52:1490043052.682149:2:info] Scanned contents of 50 additional files at 24.47 per second
    [Mar 20 20:50:51:1490043051.656608:2:info] Scanned contents of 21 additional files at 20.63 per second
    [Mar 20 20:50:50:1490043050.636141:2:info] Starting scan of file contents
    [Mar 20 20:50:49:1490043049.399005:2:info] Analyzed 305229 files containing 4.53 GB of data.
    [Mar 20 20:50:49:1490043049.090364:2:info] Analyzed 305200 files containing 4.53 GB of data so far
    [Mar 20 20:50:48:1490043048.048830:2:info] Analyzed 305100 files containing 4.53 GB of data so far

Viewing 10 replies - 1 through 10 (of 10 total)
  • My idea is that the Wordfence scan is an over rated feature. Firstly, it should only be done during low server demand. In my case, I do it once a week, at night during weekend. I also uncheck nearly all the options. Scanning my tens of thousands of images, for example, simply does not work. Likewise, scanning the huge amount of comments. In other words, try unchecking nearly all the scan options, run during low server demand, and see if you can get it working. MTN

    @reallymattgray,

    In order to identify the cause of this behavior I would recommend you enable the debug mode which will provide more information in the “Scan Detailed Activity“.

    1. Kill any scan currently running

    2. Go to your Wordfence Options page
    – Set “Maximum execution time for each scan stage” to 20
    – Save the Options page

    3. Go to the Wordfence Tools page, click the Diagnostics tab

    4. At the bottom of the page, in the Debugging Options section
    – Check the box to “Enable debugging mode
    – Save the Diagnostics page.

    5. Start a new scan

    6. Copy and paste here the last 20 lines of the activity log so we can further investigate.

    In the meantime I would recommend this article from our documentation for more information on all scan options and how to keep the scan duration under control without compromising the security of your website.

    Thread Starter Matt

    (@reallymattgray)

    Thank you. The scan took nearly 3 hours, and I had to refresh tokens at least once, but here are the last 20 lines before the scan ended due to a fork:

    Fri, 24 Mar 17 15:53:25 +0000::1490370805.6202:4:info::Scan process ended after forking.
    Fri, 24 Mar 17 15:53:25 +0000::1490370805.5470:4:info::Scanning contents: 2006_02.htm (Size:380696B Mem:44.2M)
    Fri, 24 Mar 17 15:53:25 +0000::1490370805.5390:2:info::Scanned contents of 10 additional files at 0.20 per second
    Fri, 24 Mar 17 15:53:23 +0000::1490370803.3742:4:info::Scanning contents: 2005_12.htm (Size:201290B Mem:44.2M)
    Fri, 24 Mar 17 15:53:23 +0000::1490370803.3590:2:info::Scanned contents of 9 additional files at 0.19 per second
    Fri, 24 Mar 17 15:53:19 +0000::1490370799.0850:4:info::Scanning contents: 2005_11.htm (Size:413933B Mem:44.2M)
    Fri, 24 Mar 17 15:53:19 +0000::1490370799.0767:2:info::Scanned contents of 8 additional files at 0.19 per second
    Fri, 24 Mar 17 15:53:15 +0000::1490370795.8289:4:info::Scanning contents: 2005_10.htm (Size:302820B Mem:44.2M)
    Fri, 24 Mar 17 15:53:15 +0000::1490370795.7900:2:info::Scanned contents of 7 additional files at 0.18 per second
    Fri, 24 Mar 17 15:53:13 +0000::1490370793.5074:4:info::Scanning contents: 2005_09.htm (Size:230342B Mem:44.2M)
    Fri, 24 Mar 17 15:53:13 +0000::1490370793.4563:2:info::Scanned contents of 6 additional files at 0.16 per second
    Fri, 24 Mar 17 15:53:12 +0000::1490370792.7674:4:info::Scan process ended after forking.
    Fri, 24 Mar 17 15:53:09 +0000::1490370789.9757:4:info::Scanning contents: 2005_08.htm (Size:352136B Mem:44.2M)
    Fri, 24 Mar 17 15:53:09 +0000::1490370789.9637:2:info::Scanned contents of 5 additional files at 0.15 per second
    Fri, 24 Mar 17 15:53:09 +0000::1490370789.2253:4:info::Resuming malware scan at rule G2020/rules#178.
    Fri, 24 Mar 17 15:53:09 +0000::1490370789.2083:4:info::Scanning contents: 2005_07.htm (Size:418950B Mem:43.0M)
    Fri, 24 Mar 17 15:53:09 +0000::1490370789.0495:4:info::Got a true deserialized value back from ‘wfsd_engine’ with type: object
    Fri, 24 Mar 17 15:53:08 +0000::1490370788.9998:4:info::Setting up scanRunning and starting scan
    Fri, 24 Mar 17 15:53:08 +0000::1490370788.9990:4:info::Setting up error handling environment
    Fri, 24 Mar 17 15:53:08 +0000::1490370788.9967:4:info::Requesting max memory
    Fri, 24 Mar 17 15:53:08 +0000::1490370788.9959:4:info::Done become admin
    Fri, 24 Mar 17 15:53:08 +0000::1490370788.9945:4:info::Scan authentication complete.

    Hi @reallymattgray,

    The low scanning speed rate (Scanned contents of 10 additional files at 0.20 per second) indicates that the issue could be related to insufficient resources at the time of the scan.

    In order to confirm this, could you temporarily deactivate other plugins and try launching a new scan?
    You might also want to check with your hosting provider if there are any performance issues on the server.

    I would also recommend you exclude the WordPress uploads directory from the scan. Just add the following line in the “Exclude files from scan that match these wildcard patterns” field:
    */uploads/*

    Regarding the other WordPress sites being scanned, you can modify that behavior by ensuring that the “Scan files outside your WordPress installation” feature (Wordfence → Scan → Options) is disabled.

    Thread Starter Matt

    (@reallymattgray)

    Good idea. So, I turned off every plugin except for WF, and the same problem reoccurs. Keep in mind I increased the max memory usage to 450 MB.

    Here is the activity log for the latest relevant info:
    Tue, 28 Mar 17 07:28:47 +0000::1490686127.8099:1:info::Scheduled Wordfence scan starting at Tuesday 28th of March 2017 07:28:47 AM
    Tue, 28 Mar 17 02:10:16 +0000::1490667016.0250:4:info::Calling Wordfence API v2.23:https://noc1.wordfence.com/v2.23/?v=4.7.3&s=http%3A%2F%2Fmqnf.com&k=1b691394878e93f22f6f4021b734ac87bbb20799067b09ec1d758e0d61602f61262f99f42662905874542395d3fd90f4f3598e3a01ca0b76221f9229ac56d9eee87fa494149245c31f1c670f11b83d01&openssl=9469999&phpv=5.2.17&betaFeed=0&cacheType=disabled&action=send_net_404s
    Mon, 27 Mar 17 23:32:00 +0000::1490657520.9146:4:info::Scanning contents: 2005_12.htm (Size:201290B Mem:37.8M)
    Mon, 27 Mar 17 23:32:00 +0000::1490657520.8981:2:info::Scanned contents of 8 additional files at 0.20 per second
    Mon, 27 Mar 17 23:31:59 +0000::1490657519.5178:4:info::Scan process ended after forking.
    Mon, 27 Mar 17 23:31:57 +0000::1490657517.7884:4:info::Scanning contents: 2005_10.htm (Size:302820B Mem:37.8M)
    Mon, 27 Mar 17 23:31:57 +0000::1490657517.7803:2:info::Scanned contents of 7 additional files at 0.19 per second
    Mon, 27 Mar 17 23:31:55 +0000::1490657515.5070:4:info::Scanning contents: 2005_09.htm (Size:230342B Mem:37.8M)
    Mon, 27 Mar 17 23:31:55 +0000::1490657515.5014:2:info::Scanned contents of 6 additional files at 0.17 per second
    Mon, 27 Mar 17 23:31:52 +0000::1490657512.0611:4:info::Scanning contents: 2005_08.htm (Size:352136B Mem:37.8M)
    Mon, 27 Mar 17 23:31:52 +0000::1490657512.0512:2:info::Scanned contents of 5 additional files at 0.16 per second
    Mon, 27 Mar 17 23:31:47 +0000::1490657507.9362:4:info::Scanning contents: 2005_07.htm (Size:418950B Mem:37.8M)
    Mon, 27 Mar 17 23:31:47 +0000::1490657507.9315:2:info::Scanned contents of 4 additional files at 0.15 per second
    Mon, 27 Mar 17 23:31:47 +0000::1490657507.2513:4:info::Scan process ended after forking.
    Mon, 27 Mar 17 23:31:43 +0000::1490657503.2811:4:info::Scanning contents: 2005_06.htm (Size:472225B Mem:37.8M)
    Mon, 27 Mar 17 23:31:43 +0000::1490657503.2754:2:info::Scanned contents of 3 additional files at 0.13 per second
    Mon, 27 Mar 17 23:31:42 +0000::1490657502.8516:4:info::Resuming malware scan at rule G2020/rules#722.
    Mon, 27 Mar 17 23:31:42 +0000::1490657502.8227:4:info::Scanning contents: 2005_05.htm (Size:326964B Mem:36.5M)
    Mon, 27 Mar 17 23:31:42 +0000::1490657502.6169:4:info::Got a true deserialized value back from ‘wfsd_engine’ with type: object
    Mon, 27 Mar 17 23:31:42 +0000::1490657502.5546:4:info::Setting up scanRunning and starting scan
    Mon, 27 Mar 17 23:31:42 +0000::1490657502.5540:4:info::Setting up error handling environment
    Mon, 27 Mar 17 23:31:42 +0000::1490657502.5508:4:info::Requesting max memory
    Mon, 27 Mar 17 23:31:42 +0000::1490657502.5499:4:info::Done become admin
    Mon, 27 Mar 17 23:31:42 +0000::1490657502.5486:4:info::Scan authentication complete.
    Mon, 27 Mar 17 23:31:42 +0000::1490657502.5457:4:info::Scan will run as admin user ‘matt@95864’ with ID ‘1’ sourced from: singlesite get_users() function
    Mon, 27 Mar 17 23:31:42 +0000::1490657502.5431:4:info::Becoming admin for scan
    Mon, 27 Mar 17 23:31:42 +0000::1490657502.5417:4:info::Checking saved cronkey against cronkey param
    Mon, 27 Mar 17 23:31:42 +0000::1490657502.5410:4:info::Exploding stored cronkey
    Mon, 27 Mar 17 23:31:42 +0000::1490657502.5404:4:info::Fetching stored cronkey for comparison.
    Mon, 27 Mar 17 23:31:42 +0000::1490657502.5397:4:info::Checking cronkey
    Mon, 27 Mar 17 23:31:42 +0000::1490657502.5387:4:info::Scan engine received request.

    Thread Starter Matt

    (@reallymattgray)

    Noteworthy issue for the scan line:
    Mon, 27 Mar 17 23:32:00 +0000::1490657520.9146:4:info::Scanning contents: 2005_12.htm (Size:201290B Mem:37.8M)
    the file “2005.12.htm” is one of the files listed in the
    “Exclude files from scan that match these wildcard patterns”.

    Am I required to put an asterisk (*) in front of the file name for the exclusion rule to work?

    wfyann

    (@wfyann)

    @reallymattgray,

    Sorry about the delayed response.

    Yes, indeed you need put an asterisk (*) in front of the file name.

    Also just to check if this could be related to your environment, could you please check the Wordfence System Info page:

    • Go to the Wordfence Tools page
    • Click the Diagnostics tab
    • In the Other Tests section (near the bottom of the page), click the link that reads “Click to view your system’s configuration in a new window“. This will open the Wordfence System Info page.

    And paste here the values associated with the following parameters:

    • Server API
    • Loaded Configuration File
    • PHP Version
    • cURL support
    • cURL Information
    Thread Starter Matt

    (@reallymattgray)

    Thank you.

    PHP Version 5.2.17
    Server API CGI/FastCGI
    Loaded Configuration File /web/conf/php5.ini
    cURL support enabled
    cURL Information libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5

    • This reply was modified 7 years ago by Matt.
    Thread Starter Matt

    (@reallymattgray)

    curl

    cURL support enabled
    cURL Information libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5

    wfyann

    (@wfyann)

    Hi Matt,

    At this stage what I would recommend is to upgrade PHP to version 5.6 or newer.

    For reference purposes, you can see here the requirements for WordPress itself.

    Please let us know if the issue persists after the upgrade.

Viewing 10 replies - 1 through 10 (of 10 total)
  • The topic ‘Scan hangs for hours, always stopping in same place’ is closed to new replies.