Scraping data straight to Cassandra database
Can anyone help me with Cassandra databa开发者_如何学运维se? Actually, I want to through the scraped data directly to Cassandra. Currently, I am using scrapy for parsing data from different websites.
Thanks-in advance.
ssich
In Scrapy, once you have scraped the data you can run it through an item pipeline to store it.
The documentation is easy to understand: http://doc.scrapy.org/en/latest/topics/item-pipeline.html
Just use the python driver for Cassandra within you custom item pipeline to store the results.
Edit: Updated the dead link with working one.
精彩评论