Scrape a page with AJAX requests

I'm usin开发者_运维百科g Node.js with jsDOM to scrape web pages and execute JavaScript in them. But if a page contains AJAX requests, they are not executed.

Is there an option to execute AJAX calls inside the scraped pages?

you need to emulate XMLHTTPRequests. Here's an abstraction. They don't work out of the box in node.js.

You will have to figure out how to get it to talk to jsdom. (Hopefully just appending a xmlhttprequest constructor to the window will fix it)