开发者

How can you initialize workers with doSMP?

Is there a way to initialize a doSMP cluster similar to clusterEvalQ and clusterExport in the snow package? For example:

x <- 1:10
y <- 10:1
z <- rnorm(10)
cl <- makeSOCKcluster(2)
clusterEvalQ(cl, library(quantmod))
clusterExport(cl, list("x","y","z"))
clusterEvalQ(cl, ls())
clusterEvalQ(cl, search())

There is an initEnvir option to doSMP, but ?doSMP says

 ‘initEnvir’ is a function to be executed by each worker before any
 tasks are executed associated with a foreach.  Its purpose is to
 initialize the execution environment, or the worker in general.
 It is only executed by a worker if that worker executes at least
 one task associated with the foreach.

Each worker needs a copy of several large objects in order to run the expression I send to foreach. Additionally, I need to call foreach several hundred times, with identical versions of these large objects. It would be inefficient to copy these objects for every call to foreach.

开发者_Python百科

Even if there isn't a ready-made way to do this, I'd appreciate a kludge.


The first argument to foreach is ... which transports the iterated objects for the evaluation of target expression. If you only need part of the object, then it may be more efficient to use iter to only pass portions of an object.

w <- startWorkers(workerCount = 4)
registerDoSMP(w)

foreach(x=iter(x),y=iter(y),z=iter(z) ) %dopar% (x*y*z)

The objects in the calling environment are still available:

foreach(1:10 ) %dopar% (x*y*z)  # Somewhat repetitious # 

zed <- 20:1
foreach(x=iter(x) ) %dopar% (x*zed)


Try this:

library(doSMP)
library(foreach)
w <- startWorkers(workerCount = 4)
registerDoSMP(w)
foreach(i = 1:3) %dopar% sqrt(i)

Or you could use doSNOW and

registerDoSNOW(cl)

http://cran.r-project.org/web/packages/doSNOW/doSNOW.pdf

Finally, there is doMC (not for windows)

library(doMC)
registerDoMC()
foreach(i = 1:3) %dopar% sqrt(i)

All these techniques work before the new parallel package in 2.14, which I believe has done some unification of these techniques.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜