bots and 301 redirect

I have changed the structure of the URLs of my site more than 6 months ago. I detect the use of legacy URLs and redirect to the new URL with a 301 status code. I verified with flidder that the status code is correctly returned upon the request. But bots (yahoo slurps, googlebot, etc.) are still hitting the old URLs. Is there something I am miss开发者_Python百科ing?

No, just it takes a very, very long time for crawlers to get the message. I have bots crawling addresses that have not existed since 2005 - when folk harp on with addresses being permanent, they really are.

Additionally, depending on how your URL's are structured, you can disallow the old addresses with robots.txt

Try this and this will only redirect to the bots.

if (preg_match("#(google|slurp@inktomi|yahoo! slurp|msnbot)#si", $_SERVER['HTTP_USER_AGENT'])) {
    header("HTTP/1.1 301 Moved Permanently");
    header("Location: http://www.your-main-site.com/");
    exit;
}

If external sites have linked to your old pages and those links are still accessible for bots, the bots will keep coming and try to access the content.

mentioned you site address here:

http://www.your-main-site.com/

Thats we use to transfer the domain and sometime for blackhat seo.