It takes a surprising amount of work to make WordPress secure and search-engine-friendly. And that's only if you're aware that you might need or want to do something like that, which the average WordPress user is not.
The most obvious of issues here is the fact that most people do not install WordPress very securely because they don't know how or that it's even an option. Simply using admin as the main username, not adding salts and leaving the database table prefix set as wp_ are common and no good.
These shouldn't be optional. At the install screen, it should be mandatory that both a username and wp_ be set to something unique. Salts should automatically be generated and injected upon install.
Most people have no idea that hundreds of URLs from their wp-admin and wp-includes folders end up indexed in Google and other search engines.
This should never happen. WordPress definitely needs to drop a robots.txt file that blocks access to all behind-the-scenes files and perhaps a default .htaccess file to actually enforce it.
All folders should at the very least have indexes to block directory browsing or the equivalent option set in .htaccess.
Since there's already an option of whether to share your site with search engines or not in the first place, it only seems reasonable to have it done properly should you choose to share your site.
I know there's documentation on this:
But, the problem is that nine out of ten people that use WordPress don't know any better to even look into such efforts in the first place. They have no idea. The basics should already be covered for them.
This would not take any work at all, it's just a matter of whether you choose to include a default robots.txt file similar to the following or not:
User-agent: * Disallow: /wp-admin Disallow: /wp-includes Disallow: /wp-content/plugins Disallow: /wp-content/cache Disallow: /wp-content/themes Disallow: /wp-includes/js Disallow: /trackback Disallow: /category/*/* Disallow: */trackback Disallow: /*?* Disallow: /*? Disallow: /*~* Disallow: /*~