Separating Hadoop Map and Reduce tasks
In a 3 node hadoop cluster. I would like the master to be 1 node. Map task taking place in one node and reduce tasks in 1 node. Map and reduce tasks should be separated. Is it possible? As far as i noticed both run together. It will开发者_运维技巧 be great if you can shed some light. Thank you !
-Sethu
This is everything else than optimal because the map output must ALWAYS be copied to another server.
But you can simply modify your mapred-site.xml on the servers.
<property>
<name>mapred.tasktracker.map.tasks.maximum</name>
<value>4</value>
<description>The maximum number of map tasks that will be run simultaneously by a task tracker.</description>
</property>
<property>
<name>mapred.tasktracker.reduce.tasks.maximum</name>
<value>4</value>
<description>The maximum number of reduce tasks that will be run simultaneously by a task tracker.</description>
</property>
On the server where no reducers should run you put into the reduce.task.maximum a zero. And vice versa for the other servers.
精彩评论