Scrapy domain_name for spider
From the Scrapy tutorial:
domain_name: identifies the Spider. It must be unique, that is, you can’t set the same domain name for different Spiders.
开发者_Python百科Does this mean that domain_name
must be a valid domain name, like
domain_name = 'example.com'
Or can I name
domain_name = 'ex1'
The problem is I had a spider that worked with domain name
domain_name = 'whitecase.com'
Now I created a new spider as an instance of CrawlSpider and named it
domain_name = 'wc2'
but I am getting the error "could not find spider for domain "wc2""
To answer my own question:
domain_name = 'wc2'
works.
As you have found out yourself, domain_name
shouldn't be real. It's used when calling the spider to crawl.
精彩评论