Thank you for the explanation. After installing wp 2.1.3, I selected the block search engines because of the tremendous bandwidth problem I had last month. This option seems to have stopped GoogleBot and InktomiSlurp from crawling my site and causing my bandwidth to be exceeded. I’ll keep monitoring it.
Quick question – If my blog is installed on a sub-domain of my core domain and I elect to “block search engines but allow normal visitors,” will search engines still spider my core domain? I only want to stop robots from indexing my blog.