Google blocks automated requests after 60 to 70 successful requests
I开发者_如何转开发 am developing in window service where I try fetch the page rank from http://toolbarqueries.google.com/search?q=info:(domainName)
.
The problem is when I run the thread then after 60 to 70 request google block our requests and the following error is returned:
The remote server returned an error: (503) Server Unavailable.
How do I solve that?
There are several things you could try, to avoid googles bot detection, which blocks you:
- Sleep in between the requests for at least a second.
- Vary your user agent string (just a guess)
- Vary your IP address (-> use proxy servers to route your requests)
精彩评论