I have a basic idea what to put in my robots.txt file for WordPress, but I'd like to hear what others have to say. After doing a search online for a good "WordPress robots.txt file", it seems no one can agree on what to put in there.
Some say to disallow wp-content, while others don't.
Some say to disallow /categories while others don't.
Anyone one care to chime in on what the best robots.txt file might be for WordPress?