Bug in foreach? Error in task AFTER all tasks have completed/combined?
I have a foreach, using .combine = rbind and with 112 tasks, 6 parallel operations at a time. With verbose on, I can see that all tasks complete AND combine, but then I get the message "task 6 failed - "missing value where TRUE/FALSE needed"
So I tried to run just the last 12 tasks and everything worked fine! So the last 12 don't seem to have any issues and per the verbose output, the first 100 combined successfully.
help! =)
numValues: 112, numResults: 0, stopped: TRUE
automatically exporting the following variables from the local environment:
data, dependentVariables, groupByVariable, numberOfBinsForMissingValueHistogram, outputDirectory
got results for task 1
numValues: 112, numResults: 1, stopped: TRUE
returning status FALSE
...
got results for task 100
numValues: 112, numResults: 100, stopped: TRUE
first call to combine function
evaluating call object to combine results:
fun(result.1, result.2, result.3, result.4, result.5, result.7,
result.8, result.9, result.10, result.11, result.12, result.13,
result.14, result.15, result.16, result.17, result.18, result.19,
result.20, result.21, re开发者_高级运维sult.22, result.23, result.24, result.25,
result.26, result.27, result.28, result.29, result.30, result.31,
result.32, result.33, result.34, result.35, result.36, result.37,
result.38, result.39, result.40, result.41, result.42, result.43,
result.44, result.45, result.46, result.47, result.48, result.49,
result.50, result.51, result.52, result.53, result.54, result.55,
result.56, result.57, result.59, result.60, result.61, result.62,
result.63, result.64, result.65, result.66, result.67, result.68,
result.69, result.70, result.71, result.72, result.73, result.74,
result.75, result.76, result.77, result.78, result.79, result.80,
result.81, result.82, result.83, result.84, result.85, result.86,
result.87, result.88, result.89, result.90, result.91, result.92,
result.93, result.94, result.95, result.96, result.97, result.98,
result.99, result.100)
returning status FALSE
got results for task 101
numValues: 112, numResults: 101, stopped: TRUE
...
got results for task 112
numValues: 112, numResults: 112, stopped: TRUE
calling combine function
evaluating call object to combine results:
fun(accum, result.101, result.102, result.103, result.104, result.105,
result.106, result.107, result.108, result.109, result.110,
result.111, result.112)
returning status TRUE
Error in { : task 6 failed - "missing value where TRUE/FALSE needed"
It would help to know what exactly you're doing... Things like this can happen with random processes, advanced fitting methods that can be dependent on the start parameters, disrupted input objects and so on. A traceback()
could help as well, although the output of the traceback()
can be pretty unreadible in the case of parallel tasks. But in essence, something is going wrong with the combination of a specific input and your function.
As for the error, I hope you noticed that result.6 never got included in the combine. So I would first look at what exactly happens in task6, and why it doesn't return any output.
This is a very old question, however, I discovered it after facing similar issues with the foreach
in R. This may be of help to others.
I can't see the original code, but I found when I reduced the number of cores in parallel::makeCluster()
from 24 to 16, the error "task XX failed" wasn't reproducible. It's been years since I spent time digging away at parallel code, but I'll fathom a guess that a node was either assigned nothing to do or had nothing to return.
精彩评论