Posts

Showing posts from March, 2015

Yarn command

To list nodes consuming YARN resources, you can use the command:
> yarn node -list Id State Http-Address Running-Containers s1.altiscale.com:45454 RUNNING s1.altiscale.com:8042 1  Then, to see a node’s resource consumption:

> yarn node -status s1.altiscale.com:45454

 Node Report : Node-Id : s1.altiscale.com:45454 Rack : /default-rack Node-State : RUNNING Node-Http-Address : s1.altiscale.com:8042 Last-Health-Update : Tue 19/Aug/14 03:59:36:360PDT Health-Report : Containers : 1 Memory-Used : 250MB Memory-Capacity : 2250MB CPU-Used : 1 vcores CPU-Capacity : 8 vcores

Configure and run Hadoop 2 mapreduce job history server

#1 - Create the yarn pid dir <HADOOP_YARN_HOME>/work/pids
#2 - Enable log aggregation <property><name>yarn.log-aggregation-enable</name><value>true</value></property>
#3 - Add defult yarn pid dir to the yarnenv.sh # default pid directory if["$YARN_PID_DIR"=""];then YARN_LOG_DIR="$HADOOP_YARN_HOME/work/pids"fi
#4 - After you start your dfs (<HADOOP_HOME>/sbin/start-dfs.sh) and yarn (<HADOOP_HOME>/sbin/start-yarn.sh) start your yarn mapreduce history server [hduser@hostname sbin]$ ./mr-jobhistory-daemon.sh start historyserver starting historyserver, logging to <HADOOP_HOME>/logs/mapred-hduser-historyserver-step-cca-sjc-lnx-lab9.out[hduser@hostname sbin]$