Resizing huge arrays

I'm dealing with huge arrays in my application and need to resize them.

Let's say you have an array of 2Gb and you want to resize it to 3Gb. Is there a way to resize it without needing temporarily 5Gb?

For instance, given a 1Gb heap using the -Xmx1G flag:

public class Biggy {
    public static void main(String[] args) {
        int[] array;

        array = new int[100 * 1000 * 1000]; // needs 400Mb, works
        array = null; // needed for G开发者_Go百科C
        array = new int[150 * 1000 * 1000]; // needs 600Mb, works
        array = null; // needed for GC
        array = new int[100 * 1000 * 1000]; // needs 400Mb, works
        array = Arrays.copyOf(array, 150 * 1000 * 1000); // needs 1000Mb, throws out of memory
    }
}

So, is there a way to resize arrays without requiring that extra temporary memory?

I would use a List<int[]> where each int[] is a fixed size. e.g. 128 million. To grow the whole "collection" only involves added another array. I use IntBuffer in direct memory which avoids the need to use heap. (or use memory mapped files which means it doesn't use heap or direct memory ;) This is ugly, and I use a wrapper class to hide the ugliness. It does perform pretty nicely. With memory mapped files I can use an "array" which is larger than the physical memory.

private final List<IntBuffer> array = new ArrayList<IntBuffer>();

public int get(long n) {
    return array.get((int)(n >> 27)).get(n & ((1 << 27) -1));
}

public void put(long n, int v) {
    return array.get((int)(n >> 27)).put(n & ((1 << 27) -1), v);
}

I can't figure out a solution to your problem using arrays, if you want to create a new bigger array you have to keep the old one in memory until the data is copied to the new one, so you must have both in memory even if for a short amount of time. (Correct me if I'm wrong)

I wonder if you need to grow your int collection continuously, why don't you use a List<int> instead of an int[] ?