How can I read a website as a different IP address?
I am using Nokogiri to screen scrape a few websi开发者_如何学JAVAtes. My website is hosted on US servers, and so when it fetches the website, the website responses as if the user was a US user. I need the website to responds as if I was an Australian user, even though the server is located in the US. When running locally it works fine because locally it responds as if it was Australian.
How can I read the site using Nokogiri as if I was from another country?
You have to run your requests through a proxy in Australia.
This doesn't have anything to do with Nokogiri - it applies no matter how you're trying to scrape a page. HTTP travels over TCP, which is a bi-directional protocol so you can't spoof its IP address. If you tried to spoof the IP address of a TCP packet, you would never get your response back.
You can configure Tor to always use exit nodes from a specific country. Please, however, do not use this method if this will put the Tor network under some serious strain (i.e. fetching the pages continually); in this case please consider buying an (Austrlia-based) annonimizing service (or simply a proxy).
精彩评论