开发者

ParVector map is not running in parallel

I have a bit of code like:

val data = List(obj1, obj2, obj3, obj4, ...).par.map { ... }

and the ParVector is roughly 12 elements large. I noticed that all of my work is being done in the main thread so I traced down the stacktrace and found that in the following line开发者_运维知识库 (in the link below), ifParallel is false (from CanBuildFrom). Any hints as to why it's false, and what I can do to help it?

https://github.com/paulp/scala-full/blob/2.9.0.1/src/library/scala/collection/parallel/ParIterableLike.scala#L504


I can't reproduce this:

case class Obj(i: Int)
val list = List(1 to 12 map Obj: _*)
def f(o: Obj) = { println("f " + o); Obj(o.i + 1) }
val data = list.par.map(f)

Prints:

f Obj(1)
f Obj(2)
f Obj(3)
f Obj(4)
f Obj(5)
f Obj(6)
f Obj(7)
f Obj(8)
f Obj(10)
f Obj(11)
f Obj(12)
f Obj(9)
// data: ParVector(Obj(2), Obj(3), Obj(4), Obj(5), Obj(6), Obj(7), Obj(8), Obj(9),
// Obj(10), Obj(11), Obj(12), Obj(13))

See scala: parallel collections not working? for a similar symptoms and where adding some artificial delay showed things could happen in parallel.

How do you find out ifParallel is taking the otherwise route? I do this:

scala> collection.parallel.ParSeq.canBuildFrom[Int].isParallel
res0: Boolean = true

Also, what is the value of Runtime.getRuntime.availableProcessors?


There was an error in the version of scala I was using. It has since been fixed.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜