Creating a Large Matrix in ff
I am trying to create a huge matrix in ff, and I know that ff is good for this开发者_StackOverflow sort of thing.
But, there is a major problem. The dimensions of the matrix exceed .Machine$max_integer! I am running on a 64 bit machine, using 64bit R and 64bit ff.
Is there any way to get around this problem?
It's been suggested that R is using the MAXINT value from stdint.h. Is there any way to fix this without changing that file and possibly breaking build?
> ffMatrix <- ff(vmode="boolean", dim=c(300000,300000))
Error in if (length < 0 || length > .Machine$integer.max) stop("length must be between 1 and .Machine$integer.max") :
missing value where TRUE/FALSE needed
In addition: Warning message:
In ff(vmode = "boolean", dim = c(300000, 300000)) :
NAs introduced by coercion
> 300000**2 > .Machine$integer.max
[1] TRUE
That is still a very large matrix, you would need a parallel computer :-). 300000x300000 > 2^32 (int is likely to be 32 bits, even on 64-bit machine) .
See also this link
精彩评论