Wordpress how to prevent category and archive crawl and index
I've noticed in Google's webmasters tools that I have two records showing where I have unexpected duplicated content.
Its apparently happening because Google has crawled and indexed my开发者_如何学编程 categories and archives although I have no visible links for either (that I'm aware of).
I'd like to prevent these items from being crawled and indexed, but how?
Here are the two records that Google's webmaster tools are showing....
/2009/10/ /category/test/
One way to control spider access is of course to manually create (or modify) a robots.txt file.
However, for Wordpress, it might make more sense to use a plugin, such as Google Sitemap Generator or the more SEO-geared All in One SEO Pack
You could add an if statement to the header.php file
<?php
if(is_archive) {
?>
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
<?php } ?>
Google should respect that. The is_archive conditional covers categories as well - http://codex.wordpress.org/Conditional_Tags#Any_Archive_Page
精彩评论