开发者

Using multiple spiders at in the project in Scrapy

I wanna know if it is possible to us开发者_JAVA技巧e multiple spiders within the same project together. Actually I need 2 spiders. The first one gathers the links on which the second spider should scrape. They both work on the same website, so the domain is similar.Is it possible? If yes can you give me an example? Thanks


Maybe this is what you're looking for:

def parse(self, response):
    # parse the links (aka your first spider)
    for link in hxs('//XPATH'):
        yield Request(link.extract(), callback=self.parse_link)

def parse_link(self, response):
    # continue parsing (aka your second spider)

Hope this help you :)

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜