Crawling itunes.apple.com
I am trying to crawl the apple itunes websi开发者_如何学JAVAte. I am getting output in binary format. For example
curl -A "mozilla/5.0" http://itunes.apple.com/us/app/the-far-islands-by-john-buchan/id327765949?mt=8
returns binary.
Can anybody please tell me what i am missing?
Thanks
You're getting binary back because the page you cited isn't returning HTML/XML, it's returning an Apple WebObject. From wget
:
wget http://itunes.apple.com/us/app/the-far-islands-by-john-buchan/id327765949?mt=8
--2010-08-03 12:38:14-- http://itunes.apple.com/us/app/the-far-islands-by-john-buchan/id327765949?mt=8
Resolving itunes.apple.com... 17.250.237.16
Connecting to itunes.apple.com|17.250.237.16|:80... connected.
HTTP request sent, awaiting response... 200 Apple WebObjects
Length: 22900 (22K) [text/html]
Saving to: `id327765949?mt=8'
100%[======================================>] 22,900 --.-K/s in 0.05s
2010-08-03 12:38:14 (440 KB/s) - `id327765949?mt=8' saved [22900/22900]
See the good old Wikipedia for more info, but if you want to crawl it, you may need to use something that simulates a browser and thus can interpret it - maybe watir would work.
精彩评论