web scraping without refreshing webpage
I would like to extract the data from this website: http://webcharts.fxserver.com/pairs/activePairs.php?profile=CMSForexWebQuotes1-D
I obtained the url with firebug and i did extract the data from the htmlcode with urlopen and regexes. but I have to do an urlopen each second to refresh the data inside the html code
What I am searching is a way to extract the data in a single query, without refreshing the webpage and without doing an urlopen each second...
It seems like it is written in ajax but I am not quite sure.
Is there any way to extract data without sending a query each second, (and without using cURL?)
PS: I have read several subjects like this on SO and none of them has been really solved, especially without cURL
thanks for help
EDIT With urlopen, I obtain a "frozen" string containing the html code at a fixed time. I need to do an urlopen each second 开发者_如何学Pythonto refresh the htmlcode and the data I am after
The data actually comes from this page:
http://webcharts.fxserver.com/pairs/activePairsFeed.php?day=percent&week=percent&unit=&protocol=2&version=1314105399&pairs%5B%5D=EUR%2FUSD&pairs%5B%5D=GBP%2FUSD&pairs%5B%5D=USD%2FJPY&pairs%5B%5D=GBP%2FJPY&pairs%5B%5D=GBP%2FCHF&pairs%5B%5D=USD%2FCAD&pairs%5B%5D=EUR%2FJPY&pairs%5B%5D=USD%2FCHF&pairs%5B%5D=NZD%2FUSD&pairs%5B%5D=AUD%2FUSD&pairs%5B%5D=CAD%2FJPY&pairs%5B%5D=EUR%2FCHF&pairs%5B%5D=EUR%2FAUD&pairs%5B%5D=AUD%2FJPY&pairs%5B%5D=NZD%2FJPY&pairs%5B%5D=EUR%2FGBP&pairs%5B%5D=CHF%2FJPY&pairs%5B%5D=EUR%2FCAD&pairs%5B%5D=AUD%2FCAD
Note that since the buy and sell price for each currency constantly changes, there is no way around having to refetch the page to get the new prices. See also Pull technology and polling.
精彩评论