Version

    Connecting to YARN

    If you run YARN (aka MapReduce 2.0, or MRv2) instead of the first generation of MapReduce framework on your Hadoop cluster, the following steps are required to configure the CloverDX Hadoop connection:

    1. Write an arbitrary value into the MapReduce JobTracker Host field. This value will not be used, but will ensure that MapReduce job execution is enabled for this Hadoop connection.

    2. Add this key-value pair to Hadoop Parameters: mapreduce.framework.name=yarn

    3. In the Hadoop Parameters, add the key yarn.resourcemanager.address with a value in form of a colon separated hostname and port of your YARN ResourceManager, e.g. yarn.resourcemanager.address=my-resourcemanager.example.com:8032.

    You will probably have to specify the yarn.application.classpath parameter too, if the default value from yarn-default.xml isn’t working. In this case, you would probably find some java.lang.NoClassDefFoundError in the log of the failed YARN application container.

    See also:

    HadoopReader

    HadoopWriter