Calculating MD5 hash of a partial stream
I have a large dataset (~1GB) stored in a custom file format, the last 16 bytes of which is an MD5 hash of all previous bytes of the file.
I want to verify the MD5 of this file against开发者_JAVA百科 the embedded MD5 using HashAlgorithm.ComputeHash(Stream), however this will calculate the hash of the entire file INCLUDING the hash in the last 16bytes, which obviously wont work.
How do I compute the MD5 hash of PART of a stream? I know I can read the stream into an array and pass this to HashAlgorithm.ComputeHash(Bytes), however the overhead of duplicating this 1GB of data in memory is prohibitive.
Taken from here where you can also get other ways of doing so.
Make a partial file stream class, read the size you want and make hash of it.
class PartialFileStream : FileStream
{
public PartialFileStream(string path, FileMode mode, long startPosition, long endPosition): base(path, mode)
{
base.Seek(startPosition, SeekOrigin.Begin);
ReadTillPosition = endPosition;
}
public long ReadTillPosition { get; set; }
public override int Read(byte[] array, int offset, int count)
{
if (base.Position >= this.ReadTillPosition)
return 0;
if (base.Position + count > this.ReadTillPosition)
count = (int)(this.ReadTillPosition - base.Position);
return base.Read(array, offset, count);
}
}
You can use the FileStream.Seek option to seek to a particular position of the stream and read from there.
I've found myself needing this for a second time within 6 months, so posting my solution for a partial input stream.
class PartialStream: Stream {
public Stream Source { get; }
public long Offset { get; }
public override long Length { get; }
private long End => Offset + Length;
public override bool CanRead => true;
public override bool CanSeek => false;
public override bool CanWrite => false;
public override long Position {
get => Source.Position - Offset;
set => throw new NotSupportedException();
}
public PartialStream(Stream source, long length) {
Offset = source.Position;
Length = length;
}
public PartialStream(Stream source, long offset, long length, bool seekToOffset = true) {
if (seekToOffset) source.Seek(offset, SeekOrigin.Begin);
Offset = offset;
Length = length;
}
public override int Read(byte[] array, int offset, int count) {
if (Source.Position >= End) return 0;
if (Source.Position + count > End)
count = (int)(End - Source.Position);
return Source.Read(array, offset, count);
}
public override void Flush() => throw new NotSupportedException();
public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException();
public override void SetLength(long value) => throw new NotSupportedException();
public override void Write(byte[] buffer, int offset, int count) => throw new NotSupportedException();
}
精彩评论