开发者

is it possible to stop search engines indexing pages with certain querystring parameter

I am using the asp.net DataPager and noticed that search engines wouldn't be able to crawl the links created, so I added a querystring parameter and now see that it could crawl the links.

The problem I now have is that it will now eventially end up with multiple urls for the same page.

Is it possible to stop search engines indexing all pages with the query string parameter such as:

?pg=

so the search engine will follow these links but not index:

http://domain.co.uk/news/07-04-开发者_开发百科2011/bank-of-england-base-rate?pg=3
http://domain.co.uk/news/11-03-2011/client-feedback-from-employee-relocation?pg=6

but would index:

http://domain.co.uk/news/07-04-2011/bank-of-england-base-rate
http://domain.co.uk/news/11-03-2011/client-feedback-from-employee-relocation

Thanks for any help

J.


I'm pretty sure you can achieve that with a robots.txt

With google you can specify parameters you want to exclude with the webmaster tools.

You could also trap the useragent and simply crash the page when it's a bot ...

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜