Scala Java interop typechecking
I am unable to fix these typechecking errors in Scala:
package junk
import org.apache.hadoop.conf.Configuration
import org.apache.hadoop.io.IntWritable
import org.apache.hadoop.io.Text
import org.apache.hadoop.mapreduce.lib.input.TextInputFormat
import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat
import org.apache.hadoop.mapreduce.Job
object Sample {
val conf: Configuration = new Configuration()
val job: Job = new Job(conf, "sample Hadoop MapReduce program");
job.setInputFormatClass(classOf[TextInputFormat])
job.setOutputFormatClass(classOf[TextOutputFormat])
job.setOutputFormatClass(classOf[TextOutputFormat[Text, IntWritable]])
}
The error messages are
Sample.scala:14: error: type mismatch;
found : java.lang.Class[org.apache.hadoop.mapreduce.lib.input.TextInputFormat](classOf[org.apache.hadoop.mapreduce.lib.input.TextInputFormat])
required: java.lang.Class[_ <: org.apache.hadoop.mapreduce.InputFormat]
job.setInputFormatClass(classOf[TextInputFormat])
^
Sample.scala:15: error: class TextOutputFormat takes type parameters
job.setOutputFormatClass(classOf[TextOutputFormat])
^
Sample.scala:16: error: type mismatch;
found : java.lang.Class[org.apache.hadoop.mapreduce.lib.output.TextOutputFormat[org.apache.hadoop.io.Text,org.apache.hadoop.io.IntWritable]](classOf[org.apache.hadoop.mapreduce.lib.output.TextOutput开发者_运维百科Format])
required: java.lang.Class[_ <: org.apache.hadoop.mapreduce.OutputFormat]
job.setOutputFormatClass(classOf[TextOutputFormat[Text, IntWritable]])
Note that I have tried to setOutputFormatClass in two different ways and both fail.
I am trying to write Hadoop WordCount: http://wiki.apache.org/hadoop/WordCount in Scala. The equivalent lines in Java are
job.setInputFormatClass(TextInputFormat.class);
job.setOutputFormatClass(TextOutputFormat.class);
API docs for Job: http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/Job.html
See Programming In Scala, Existential Types.
val textInputFormatClass = classOf[TextInputFormat].
asInstanceOf[Class[T] forSome {type T <: InputFormat[String, String]}]
job.setInputFormatClass(textInputFormatClass)
Substitute the types that you are using for [String, String]
scala> val textInputFormatClass = classOf[TextInputFormat].asInstanceOf[Class[T] forSome {type T <: InputFormat[String, String]}]
textInputFormatClass: java.lang.Class[_ <: org.apache.hadoop.mapreduce.InputFormat[String,String]] = class org.apache.hadoop.mapreduce.lib.input.TextInputFormat
scala> val job = new Job
job: org.apache.hadoop.mapreduce.Job = org.apache.hadoop.mapreduce.Job@4a205144
scala> job.setInputFormatClass(textInputFormatClass)
scala>
As of Scala 2.9.1 (not Scala 2.9.0 -- see https://issues.scala-lang.org/browse/SI-4603), this also works:
job.setInputFormatClass(classOf[TextInputFormat])
精彩评论