The right place for "io.sort.mb" in Hadoop?
I am a bit confused, in the Hadoop cluster setup, in section "Real-World Cluster Configurations", an example is given where properties like io.sort.mb & io.sort.factor goes in core-site.xml. But in the default configuration files these properties appears in mapred-site.xml!!! which one I should follow? Am i free to put it anywhere? If yes, what is the difference be开发者_如何学JAVAtween the three configuration file (core, mapred, and hdfs .xml)?
Is there any complete list of all possible properties?
Any help is much appreciated..
EDIT: Trial and error I found io.sort.mb belongs to mapred-site.xml.
core-default.html, hdfs-default.html and mapred-default.html have all the properties with their defaults.
According to the 'Hadoop : The Definitive Guide'
core-site.xml has 'Configuration settings for Hadoop Core, such as I/O settings that are common to HDFS and MapReduce.'
hdfs-site.xml has 'Configuration settings for HDFS daemons: the namenode, the secondary namenode, and the datanodes.'
mapred-site.xml had 'Configuration settings for MapReduce daemons: the jobtracker, and the tasktrackers.'
Find more information about the configuration files here.
精彩评论