Will buffer size make a difference in time cost while reading files?

I have two options concerning buffer sizes when reading files.

char* buffer = new char[aBlock];
myFile.read(buffer,aBlock);

and,

char* buffer = new char;
while (!myFIle.eof())
    myFile.read(buffer,1);

Will there be a considerable time cost difference? Be aware that as buffer I'm referring to that char* buffer in the code, I'm开发者_高级运维 not talking about the OS file buffers

Yes there will be. In all practical operating systems, the price of doing i/o is quite a bit higher than the tradeoff of having less memory available. Practical buffer sizes are much smaller than might be expected. The C runtime library default buffer size for a FILE * of 512 bytes is pretty good—really good in fact for the multitude of situations it is used. And that was developed for a 65,536 byte memory space on Unix V6 (c. 1978).

Carefully measuring throughput, CPU load, and overall system load to optimize buffer size has always led me to choose a buffer size in the range of 1024 to 16384 bytes. The only exception is for files a little larger than that range, in which case it is optimal to hold the whole file in memory when memory is available.

You will already be reading buffered, as it does not read directly from the file to the buffer you put into the call to read, but will first go to the buffer inside the fstream (a filebuf).

The first sequence will still be quicker because it will loop fewer times, but not as drastic as people may think because the file I/O itself won't be any slower.

You can change the internal buffer of the fstream but that is a more complex issue.

The runtime library will buffer for you. The extra cost here is mostly in executing the same instructions over and over to read one byte versus blocked reads. Number of calls is aBlock times greater if you read one byte at a time.

It's much faster to use a large buffer, because the drive will then be able to read eg. an entire cylinder at once, instead of reading a sector, then waiting for the next request to read another. Also, there's a lot of overhead involved in each request, like getting access to the bus and setting the dma controller. Just take to don't use so much memory that you'll need to swap data out to the disk, which will slow things down a lot.

There is a actually a huge difference, depending on the implementation. In Visual C++ 2008, there is a critical section that is entered on each call to read(). So the second set of code in the question will be entering "aBlock" more critical sections that the first set of code.