How to manage large set of data on a mobile device

I am currently implementing a Japanese dictionary and would like some ideas on how to find entries in a fast and efficient manner. The dictionary entries themselves are loaded and stored in a class which houses the words in Japanese (potentially with multiple spellings), their phonetic pronunciation(s) and definitions in English as three Lists.

I originally had a tree structure with each node representing a character in a word. It was very fast but unfortunately used a lot of memory. The definition data itself is around 35meg, and the tree structure itself was around 130meg. Additionally, it also only allowed for looking up from Japanese -> English. Memory is a concern as apart from the need for additional indexes it is intended that a Windows Phone 7 version of the application will exist. The dictionary contains a 150,000 entries.

To sum up, this is what I need to do: Store a list of definitions Have three indexes into this list for 开发者_如何学JAVAJapanese, phonetic and English meanings. Have lookups be fast (ideally less than 1 second) Use as little memory as possible (the memory usage limit on Windows Phone 7 is 90meg)

Any ideas?

You should consider using a mobile database, or web service, to deal with that much data on a mobile device. The kind of performance, features and management you're looking for are very difficult to get right on any platform but you don't have to. Use a db with proper indexes and tables, local or remote.

SQL CE is not available for WP7 but there are other options, commercial and open source. Also, if you're expecting to be connected frequently and are OK with a little bit of latency, do consider using a web service. You can use any database of choice on a central server presented to you via the web service. Your bottleneck would be communications but you would offload a significant amount of processing from a constrained device.

• The simplest solution is: client side only accept the input, then send a request to server side, and the server side return the output to the client side.
• Another way is make the application offline:
  1. The list of Japanese words should be sorted in lexicographic order, so does the other two lists.
  2. As Japanese have 50 letters, then the list can be split into 50 * 50 parts according to the first two letters of a word. So the key point is that: prepare an 50 * 50 array, recording the offsets corresponding to the start position of the first two letters in the file. If use 32bit integer to store offsets, it will cost 10KB.
  3. For each Japanese word, store the offsets in the other two lists. For convenience, the two lists can be combined together. If use 32bit integer to store offsets, and you have 150000 entries, it will increase the data size by 600KB.
  4. Keep the array of offsets in memory, when input a Japanese word, first find out the offset according to the first two letters, then read all the Japanese words starting with the first two letters. Cause you have 150000 entries, so in average the number of words starting with the same two letters is 60. And a memory page is 4KB, enough for 60 words, so only 1 IO is needed. After find the offsets in the other two lists, read the corresponding contents. So, only 2 IO is needed in a search operation. And the calculation cost is small too, the memory cost is also very low.