Add Entire Files Text as Map Key in Hadoop
I'm looking for a way to loa开发者_StackOverflow中文版d an entire file text into my map. Not a single line at a time like TextInputFormat
does.
value.toString
in my map it gives me the entire input to work with.You have to put every line into a StringBuilder
until you've reached the end of file. Or you override your own RecordReader that provides this functionality. But I would not recommend this.
I would path name of the file to the mapper, and then will be free to load is entirely or do some kind of streaming processing.
精彩评论