Reading a binary file in Java vs C++

I have a binary file (about 100 MB) that I need to read in quickly. In C++ I could just load the file into a char pointer and march through it by incrementing the pointer. This of course would be very fast.

Is there a com开发者_开发知识库parably fast way to do this in Java?

If you use a memory mapped file or regular buffer you will be able to read the data as fast your hardware allows.

File tmp = File.createTempFile("deleteme", "bin");
tmp.deleteOnExit();
int size = 1024 * 1024 * 1024;

long start0 = System.nanoTime();
FileChannel fc0 = new FileOutputStream(tmp).getChannel();
ByteBuffer bb = ByteBuffer.allocateDirect(32 * 1024).order(ByteOrder.nativeOrder());

for (int i = 0; i < size; i += bb.capacity()) {
    fc0.write(bb);
    bb.clear();
}
long time0 = System.nanoTime() - start0;
System.out.printf("Took %.3f ms to write %,d MB using ByteBuffer%n", time0 / 1e6, size / 1024 / 1024);

long start = System.nanoTime();
FileChannel fc = new FileInputStream(tmp).getChannel();
MappedByteBuffer buffer = fc.map(FileChannel.MapMode.READ_ONLY, 0, size);
LongBuffer longBuffer = buffer.order(ByteOrder.nativeOrder()).asLongBuffer();
long total = 0; // used to prevent a micro-optimisation.
while (longBuffer.remaining() > 0)
    total += longBuffer.get();
fc.close();
long time = System.nanoTime() - start;
System.out.printf("Took %.3f ms to read %,d MB MemoryMappedFile%n", time / 1e6, size / 1024 / 1024);

long start2 = System.nanoTime();
FileChannel fc2 = new FileInputStream(tmp).getChannel();
bb.clear();
while (fc2.read(bb) > 0) {
    while (bb.remaining() > 0)
        total += bb.get();
    bb.clear();
}
fc2.close();
long time2 = System.nanoTime() - start2;
System.out.printf("Took %.3f ms to read %,d MB File via NIO%n", time2 / 1e6, size / 1024 / 1024);

prints

Took 305.243 ms to write 1,024 MB using ByteBuffer
Took 286.404 ms to read 1,024 MB MemoryMappedFile
Took 155.598 ms to read 1,024 MB File via NIO

This is for a file 10x larger than what you want. Its this fast because the data is being cached in memory (and I have an SSD drive). If you have fast hardware, the data can be read pretty fast.

Sure, you could use a memory mapped file.

Here are two good links with sample code:

• Thinking in Java: Memory-mapped files
• Java Tips: How to create a memory-mapped file

If you don't want to go this route, just use an ordinary InputStream (such as a DataInputStream after wrapping it in a BufferedInputStream.

Most files will not need memory mapping but can simply be read by the standard Java I/O, especially since your file is so small. A reasonable way to read said files is by using a BufferedInputStream.

InputStream in = new BufferedInputStream(new FileInputStream("somefile.ext"));

Buffering is already optimized in Java for most computers. If you had a larger file, say 100MB, then you would look at optimizing it further.

Reading the file from the disk is going to be the slowest part by miles, so it's likely to make no difference whatsoever. Of this individual operation, of course- the JVM still takes a decade to start up, so add that time in.

Take a look at this blog post here on how to read a binary file into a byte array in Java:

http://www.spartanjava.com/2008/read-a-file-into-a-byte-array/

Copied from link:

File file = new File("/somepath/myfile.ext");
FileInputStream is = new FileInputStream(file);

// Get the size of the file
long length = file.length();

if (length > Integer.MAX_VALUE) {
    throw new IOException("The file is too big");
}

// Create the byte array to hold the data
byte[] bytes = new byte[(int)length];

// Read in the bytes
int offset = 0;
int numRead = 0;
while (offset < bytes.length
       && (numRead=is.read(bytes, offset, bytes.length-offset)) >= 0) {
    offset += numRead;
}

// Ensure all the bytes have been read in
if (offset < bytes.length) {
    throw new IOException("The file was not completely read: "+file.getName());
}

// Close the input stream, all file contents are in the bytes variable
is.close()    

Using the DataInputStream of the Java SDK can be helpful here. DataInputStream provide such functions as readByte() or readChar(), if that's what needed. A simple example can be:

DataInputStream dis = new DataInputStream(new FileInputStream("file.dat")); 
try {
   while(true) {
      byte b = dis.readByte();
      //Do something with the byte
   } 
} catch (EOFException eofe) {
//Stream Ended
} catch (IOException ioe) {
//Input exception
}

Hope it helps. You can, of course, read the entire stream to a byte array and iterate through it as well...