What's the best way to read/write array contents from/to binary files in C#?
I would like to read and write the contents of large, raw volume files (e.g. MRI scans). These files are just a sequence of e.g. 32 x 32 x 32 floats so they map well to 1D arrays. I would like to be able to read the contents of the binary volume files into 1D arrays of e.g. float or ushort (depending on the data type开发者_如何学JAVA of the binary files) and similarly export the arrays back out to the raw volume files.
What's the best way to do this with C#? Read/Write them 1 element at a time with BinaryReader/BinaryWriter? Read them piece-wise into byte arrays with FileStream.Read and then do a System.Buffer.BlockCopy between arrays? Write my own Reader/Writer?
EDIT: It seems it's not possible to work with > 2GB arrays, but the question still stands for smaller arrays (around 256 MB or so)
You're not going to get arrays with more than 2GB data. From what I remember, there's a CLR limit of 1GB per object. It's possible that's been lifted for .NET 4 on 64-bit, but I haven't heard about it.
EDIT: According to this article the limit is 2GB, not 1GB - but you still won't manage more than 2GB.
Do you really have to have all the data in memory at one time? Can you work on chunks of it at a time?
EDIT: Okay, so it's now just about reading from a file into a float array? It's probably simplest to read chunks (either using BinaryReader.Read(byte[], int, int)
or BinaryReader.ReadBytes(int)
) and then use Buffer.BlockCopy
to efficiently convert from bytes to floats etc. Note that this will be endian-sensitive, however. If you want to convert more robustly (so that you can change endianness later, or run on a big-endian platform) you'd probably want to call ReadFloat()
repeatedly.
How convinced are you that you actually have a performance issue in this area of the code? It's worth doing the simplest thing that will work and then profiling it, to start with...
精彩评论