Why does my fork/join deadlock?
Consider the following snipped of code, which calculates the size of all paths given.
def pathSizes = []
paths.each { rootPath ->
pathSizes.addAll(
withPool { pool ->
runForkJoin(rootPath) { path ->
def headSizes = [:]
println path
def lines = ["ls", "-al", path].execute().text.readLines()
(0..<3).each { lines.remove(0) }
lines.each { line ->
def fields = line.split(/\s+/)
if (fields[0] =~ /^d/)
forkOffChild("$path/${fields.last()}")
else {
def userName = fields[2]
def fileSize = fields[4] as long
if (headSizes[userName] == null)
headSizes[userName] = fileSize
else
headSizes[userName] += fileSize
}
}
quietlyJoin()
System.gc()
def shallowSizes =
headSizes.collectEntries { userName, fileSize ->
def childResult =
childrenResults.sum {
it.shallowSizes[userName] ? it.shall开发者_JS百科owSizes[userName] : 0
} ?: 0
return [userName, fileSize + childResult]
}
def deepSizes =
childrenResults.sum { it.deepSizes ?: [] } +
shallowSizes.collect { userName, fileSize ->
[userName: userName, path: path, fileSize: fileSize]
}
return [shallowSizes: shallowSizes, deepSizes: deepSizes]
}.deepSizes
})
}
Why does this snippet of code deadlock? There are no interactions between threads except possibly with the system call and other parts of the Java framework. If the system calls are the problem, then how can I fix it, without removing the system calls (they are slow, hence the need to parallelize)?
精彩评论