开发者

could not find function inside foreach loop

I'm trying to use foreach to do multicore computing in R.

A <-function(....) {
    foreach(i=1:10) %dopar% {
    B()
    }
}

then I call function A in the console. The problem is I'm calling a function Posdef inside B that is defined in another script file which I source. I had to put Posdef in the list of export argument of foreach: .export=c("Posdef"). However I get the following error:

Error in { : 开发者_StackOverflowtask 3 failed - "could not find function "Posdef""

Why cant R find this defined function?


So I can reproduce this, for the curious:

require(doSNOW)
registerDoSNOW(makeCluster(5, type="SOCK"))
getDoParWorkers()
getDoParName()
getDoParVersion()

fib <- function(n) {
  if (n <= 1) { return(1) }
  return(fib(n-1) + fib(n-2))
}

my.matrix <- matrix(runif(2500, 10, 50), nrow=50)

calcLotsaFibs <- function() {
  result <- foreach(row.num=1:nrow(my.matrix), .export=c("fib", "my.matrix")) %dopar% {
    return(Vectorize(fib)(my.matrix[row.num,]))
  }
  return(result)
}

lotsa.fibs <- calcLotsaFibs()

I have been able to get around this by putting the function in another file and loading that file in the body of the foreach. You could also obviously move the function definition into the body of the foreach itself.

[EDIT -- I had previously suggested that perhaps .export doesn't work properly with function names, but was corrected below.]


The short answer is that this was a bug in parallel backends such as doSNOW, doParallel and doMPI, but it has since been fixed.

The slightly longer answer is that foreach exports functions to the workers using a special "export" environment, not the global environment. That used to cause problems for functions that were created in the global environment, because the "export" environment wasn't in their scope, even though they were now defined in that same "export" environment. Thus, they couldn't see any other functions or variables defined in the "export" environment, such as "Posdef" in your case.

The doSNOW, doParallel and doMPI backends now change the associated environment from the global to the "export" environment for functions exported via ".export", and seems to have resolved these issues.


Quick fix for problem with foreach %dopar% is to reinstall these packages:

install.packages("doSNOW")

install.packages("doParallel") 

install.packages("doMPI")

It worked in my case.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜