Xml Sitemaps for large websites
How do large websites deal with thousands of old urls that a开发者_C百科re deleted everyday? particularly when the website redirects to a customized error page on hitting a broken url instead of throwing a 404 error page. (I've read search engine crawlers do not consider urls that throw 404 error page)
If a page is deleted, I'd strongly recommend that you make your "customized error page" actually return a "410 Gone" HTTP header.
精彩评论