Scraping site that requires Javascript enabled with Mechanize + BeautifulSoup (Python)
So.. i got this site I am tryign to scrape, but as I understand lack of support of mechanize for .js, and a stuborn site th开发者_运维知识库at requires javascript enabled browser is not a good mix...
I am looking for ideas, on how to do this...
URL : https://members.iracing.com/membersite/login.jsp
Depending on what you need to do, you could use webkit to parse the page, which will allow you to get the final html after the javascript has been executed. You could then use any decent html parser, beautifulsoup for example, to do the rest.
With JavaScript I use Chickenfoot for simple websites and Webkit for more complex.
精彩评论