Best Way to Store Data from Large Web Crawl [closed]
I am using a python web crawler for various social networking sites and am trying to determine the best way to store large amounts of data (mostly xml/text data) that I screen scrape. Could you suggest any databases that would be appropriate and easily accessible. Something that works well with python would be nice. Additionally, I would want to go back and parse the data at a later date.
精彩评论