Most efficient data storage

I need to transform/convert/process TransXchange开发者_如何学编程 data dump to reduce the size of data as some of the xml files can be up to 400 MB. I have the following options:

• Sqlite database
• CSV files
• Binary Serialization
• ?

What is the best method of reducing the file sizes so that they would be feasible for use in a Windows Phone 7 application?

EDIT: I am going to create journey planning application that will allow users to specify source and destination. The application will then present the available services. In down under we have spotty mobile broadband coverage therefore I am aiming to have offline application.

This analysis is superb for showing you the timing of serialisation: http://www.eugenedotnet.com/2010/12/windows-phone-7-serialization-comparison/

For size... It's quite easy to guess that binary is smaller than sqlite (or Sterling) which in turn is smaller than CSV

However, if you are looking at processing 400MB of data on the phone... then I'd say you are doing the wrong thing - farm the processing out to a server (to the cloud?) and just view the summary results on the phone - think "thin client".

(Off to wash my mouth out now after all those jargon words!)

The main question is what are you going to do with that data.

If you just need to store the data and files are fine then binary serialization + compression (zlib, lzo...) will work best.

CSV won't do you any good.. will probably occupy more than the XML.

Database (for example, Sqlite) is the most expensive it terms of storage but you can manage and search the data more easily.