I'd really love it is there was a way to capture bandwidth usage by my visitors. In particular the search bots. But overall too... (Say, for instance, finding out that real visitors are using about 20MB/day, but 'bots are using 5GB/day, etc.)
It would be cool if the plugin could capture whether page views were "complete" (all elements downloaded) or just 'pings' (for instance requests denied by Wordfence due to a crawler exceeding its crawl quota in a given timeframe and being denied page views for a while; that is request received but page not sent). And likewise if a page was loaded how much bandwidth was used. That is how many KB, MB, GB were downloaded "per page" or "per session."
Basically, this month and last month our bandwidth usage has gone through the roof, even though our "visitors" haven't increased significantly. But total page views went from about 800/day up to like 7000/day.
*I think* it's because search engine bots have been relentlessly crawling our site, which seems to be borne out by the breakdown of total page view stats. But I want to be able to see how much bandwidth those page views are taking up "each" and "per session" or "per day."
In part I want to see whether the measures I've taken with Wordfence to throttle crawler bot page views have resulted in lowering our total daily bandwidth usage. But I can't do that without some method of tracking how much bandwidth has been used.
Total "page requests" doesn't quite give that level of detail... It only tells me how many requests were sent, not how many pages were successfully served or how many KB / MB / GB were actually downloaded.
I'd love to see those metrics. Pretty sure that would make the SlimStats / Wordfence combo a kind two-pronged "killer app" for monitoring / controlling bandwidth usage, especially for heading off overly-aggressive search engine crawlers.
To illustrate part of the problem, there has actually been an inverse response from the search engines after I throttled their access with Wordfence, in that it looks like due to their inability to load the pages they're actually responded by *increasing* their page requests (rather than realizing they're being throttled and reducing their request frequency). Requests have now gone from about 9000/day to about 25,000/day (times of increases coinciding with my implementation of throttling).
Now, what I want to know is whether despite page views going up exponentially, total bandwidth usage has maybe actually gone down due to the throttling (IE, just because a page or document is requested doesn't necessarily mean it actually gets served). Make sense?
A bandwidth usage metric would resolve this quandary one way or the other, so I could see whether A) requests went up and bandwidth usage also went up or B) requests went up but bandwidth usage dropped off or C) page requests went up, but bandwidth usage somehow remained steady.
Is this possible? To monitor actual page / file serving, and log the size of said pages / files in terms of real bandwidth usage? And associate those serves with specific users / sessions / IPs?