java - hadoop - map reduce task and static variable -

java - hadoop - map reduce task and static variable -

i started working on hadoop/hbase mapreduce job (using cloudera) , have next question :

let's say, have java class main , static viariable. class define inner class corresponding mapper , reducer tasks. before lauching job, main initialize static variable. variable read in mapper class. class launched using 'hadoop jar' on cluster.

my question: don't see how map , cut down tasks on other nodes can see static variable. there "hadoop magic" allow nodes share jvm or static variables ? how can work ? have work on class doing that, , can't figure out how ok in non-mononode cluster. give thanks you

in distributed hadoop cluster each map/reduce task runs in it's own separate jvm. there's no way share static variable between different class instances running on different jvms (and on different nodes).

but if want share immutable info between tasks, can utilize configuration class:

// driver code configuration config = configuration.create(); config.setlong("foo.bar.somelong",1337); ... // mapper code public class somemapper ... { private long somelong = 0; public void setup(context context) { configuration config = context.getconfiguration(); somelong = config.getlong("foo.bar.somelong"); } }

java hadoop cloudera