ParVector map is not running in parallel
I have a bit of code like:
val data = List(obj1, obj2, obj3, obj4, ...).par.map { ... }
and the ParVector is roughly 12 elements large. I noticed that all of my work is being done in the main thread so I traced down the stacktrace and found that in the following line开发者_运维知识库 (in the link below), ifParallel is false (from CanBuildFrom). Any hints as to why it's false, and what I can do to help it?
https://github.com/paulp/scala-full/blob/2.9.0.1/src/library/scala/collection/parallel/ParIterableLike.scala#L504
I can't reproduce this:
case class Obj(i: Int)
val list = List(1 to 12 map Obj: _*)
def f(o: Obj) = { println("f " + o); Obj(o.i + 1) }
val data = list.par.map(f)
Prints:
f Obj(1)
f Obj(2)
f Obj(3)
f Obj(4)
f Obj(5)
f Obj(6)
f Obj(7)
f Obj(8)
f Obj(10)
f Obj(11)
f Obj(12)
f Obj(9)
// data: ParVector(Obj(2), Obj(3), Obj(4), Obj(5), Obj(6), Obj(7), Obj(8), Obj(9),
// Obj(10), Obj(11), Obj(12), Obj(13))
See scala: parallel collections not working? for a similar symptoms and where adding some artificial delay showed things could happen in parallel.
How do you find out ifParallel
is taking the otherwise route? I do this:
scala> collection.parallel.ParSeq.canBuildFrom[Int].isParallel
res0: Boolean = true
Also, what is the value of Runtime.getRuntime.availableProcessors
?
There was an error in the version of scala I was using. It has since been fixed.
精彩评论