开发者

Can I use Scrapy functions inside Django classes

I have build a website in Djang开发者_运维知识库o. I need to use the web crawling features. So I installed Scrapy. Scrapy is working, as stated in their tutorial, by using

scrapy startproject dmoz

But that works on a diff folder with its own files.

Is it possible to use Scrapy functions in Django classes and put data in Django variables?


Crawling is a long, time consuming task. It doesn't mix well with short HTTP requests. Therefore, you should use a user interface (presumably written in Django) to fire off Scrapy crawls, and a shared storage to monitor the state of the crawl.

But you wouldn't want to crawl during the lifetime of a HTTP requests. That's not how Scrapy is meant to be used.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜