Loading specific section of a compressed file stream

We have a simple binary file format for caching data in our application (C# .NET Windows App). The format is basically a short that indicates the object type followed by a guid (string) for the object id then any object specific data (strings ints whatever). We want to be able to store many objects in the same file (> 10000) but in certain situations only load on demand. The solut开发者_JAVA百科ion we have is to keep an index of the object locations within the file - so when we start writing a new object we record the position in the file stream the object starts. When we want to load this object, we use this indexed location to load the relavent data. This works fine.

However, if we want to compress the file, will this method still be possible? I'm not too hot on how compression works and specifically the GZipStream class (System.IO.Compression) that we are planning to use. As I understand it, this class does not support Seeking or the Position property. Will it still be possible to use the Seek and Position of the underlying FileStream (I'm guessing not)? Basically, is it possible to have a compressed file that we can selectively load from, if so, how do we do it?

Thanks,

Steve

No, if you want to access a specific position in the uncompressed data you will have to decompress it, at least temporarily

This is not a true Seek, but a solution would be:

• keep a track of your position in the file (probably best done by implementing a "myBinaryReader" which inherits from BinaryReader)
• if you are Seeking a position forward from your current position - ReadBytes until you get there.
• if you are Seeking a position prior to your current position - reopen the file for a decompressed read (which resets your current position to zero), and then ReadBytes until you get to where you want to be.

Obviously this is not at all an ideal solution, but it may still give acceptable performance. In my case, the compressed file fits easily within memory (uncompressed does not), so I have loaded the compressed file into memory.

Ideally the underlying deflate class would be changed to support true Seeks.

a better solution:

use GZipStream to create compressed bytes within memory, and then write your own class to control the caching and writing of this to disk (don't use DeflateStream). Also write your own class to for the reading of this data from disk.

You can then ensure the underlying disk stream supports Seeks.