开发者

Parititioned Data Map/Reduce

I have written my custom partitioner for partitioning datasets. I want to partition two datasets using the same partitioner and then in the next mapreduce job, I want each mapper to handle the same partition fr开发者_如何学Goom the two sources and perform some function such as joining etc. How I can I ensure that one mapper gets the split that corresponds to same partition from both the sources?

Any help would be highly appreciated.


What you are describing is one variation of a map-side join. Chapter 8 of Pro Hadoop or org.apache.hadoop.mapred.join

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜