java - Caught ClassCastException in my mapReduce code -
java - Caught ClassCastException in my mapReduce code -
here attached code, mash avro files big one. when run it, classcastexception caught, can help me out?
thanks in advance! bobo
public class filecompactiondriver extends configured implements tool {
public static void main(string[] args) throws exception { int returncode = toolrunner.run(new filecompactiondriver(), args); /*terminate job*/ system.exit(returncode); } public int run(string[] arg) throws exception { /*setup configuration*/ configuration conf = this.getconf(); /*test if valid running command*/ if(arg.length !=2){ system.err.println("usage: filecompaction <input path> <output path>"); system.exit(2); } //creating job object configuration job job = job.getinstance(conf); job.setjobname("filecompactor"); job.setjarbyclass(filecompactionmapper.class); /*setup mapper , reducer classes*/ job.setmapperclass(filecompactionmapper.class); job.setreducerclass(filecompactionreducer.class); job.setnumreducetasks(1); /*setup output types*/ job.setoutputkeyclass(avrokey.class); job.setoutputvalueclass(nullwritable.class); /*setup input , output classes format*/ /*handle , writing avro container files*/ job.setinputformatclass(avrokeyinputformat.class); job.setoutputformatclass(avrokeyoutputformat.class); /*specify input , output directories*/ fileinputformat.setinputpaths(job, new path(arg[0])); fileoutputformat.setoutputpath(job, new path(arg[1])); homecoming (job.waitforcompletion(true) ? 0 : 1); }
}
i guess need specify job.setmapoutputkeyclass
, job.setmapoutputvalueclass
rid of classcastexception
.
java hadoop mapreduce
Comments
Post a Comment