2.7.3YARN获取debug命令:resourceManager+nodeManager

2019独角兽企业重金招聘Python工程师标准>>>

1 源码和二进制下载

http://archive.apache.org/dist/hadoop/core/hadoop-2.7.3/

源码

http://archive.apache.org/dist/hadoop/core/hadoop-2.7.3/hadoop-2.7.3-src.tar.gz

二进制

http://archive.apache.org/dist/hadoop/core/hadoop-2.7.3/hadoop-2.7.3.tar.gz

2yarn启动在machine0上

3启动脚本分析./sbin/start-yarn.sh

3.1正常打印

略

3.2libexec/yarn-config.sh【完毕】

3.2.1libexec/hadoop-config.sh【ing】

关于配置文件的位置说明

直接看代码

# Allow alternate conf dir location.

if [ -e "${HADOOP_PREFIX}/conf/hadoop-env.sh" ]; then

DEFAULT_CONF_DIR&＃61;"conf"

else

DEFAULT_CONF_DIR&＃61;"etc/hadoop"

3.2.1.1 hadoop-env.sh【完毕】

层次还挺深...

3.3sbin/yarn-daemon.sh启动resourcemanager【完毕】

真实的命令

# start resourceManager

"$bin"/yarn-daemon.sh --config $YARN_CONF_DIR start resourcemanager

让我们看看具体做了什么

规范&＃xff1a;

yarn-daemon.sh [--config ] [--hosts hostlistfile] (start|stop)

3.3.1 libexec/yarn-config.sh【同上】

3.3.2/etc/hadoop/yarn-env.sh【完毕】

3.3.3执行start函数获得resourceManager启动命令

/root/jdk1.8.0_181/bin/java -Dproc_resourcemanager -Xmx100m -Dhadoop.log.dir&＃61;/root/yarn_study/hadoop-2.7.3/logs -Dyarn.log.dir&＃61;/root/yarn_study/hadoop-2.7.3/logs -Dhadoop.log.file&＃61;yarn-yarn-resourcemanager-machine0.log -Dyarn.log.file&＃61;yarn-yarn-resourcemanager-machine0.log -Dyarn.home.dir&＃61; -Dyarn.id.str&＃61;yarn -Dhadoop.root.logger&＃61;INFO,RFA -Dyarn.root.logger&＃61;INFO,RFA -Djava.library.path&＃61;/root/yarn_study/hadoop-2.7.3/lib/native -Dyarn.policy.file&＃61;hadoop-policy.xml -Dhadoop.log.dir&＃61;/root/yarn_study/hadoop-2.7.3/logs -Dyarn.log.dir&＃61;/root/yarn_study/hadoop-2.7.3/logs -Dhadoop.log.file&＃61;yarn-yarn-resourcemanager-machine0.log -Dyarn.log.file&＃61;yarn-yarn-resourcemanager-machine0.log -Dyarn.home.dir&＃61;/root/yarn_study/hadoop-2.7.3 -Dhadoop.home.dir&＃61;/root/yarn_study/hadoop-2.7.3 -Dhadoop.root.logger&＃61;INFO,RFA -Dyarn.root.logger&＃61;INFO,RFA -Djava.library.path&＃61;/root/yarn_study/hadoop-2.7.3/lib/native -classpath /root/yarn_study/hadoop-2.7.3/etc/hadoop:/root/yarn_study/hadoop-2.7.3/etc/hadoop:/root/yarn_study/hadoop-2.7.3/etc/hadoop:/root/yarn_study/hadoop-2.7.3/share/hadoop/common/lib/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/common/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/hdfs:/root/yarn_study/hadoop-2.7.3/share/hadoop/hdfs/lib/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/hdfs/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/yarn/lib/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/yarn/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/mapreduce/lib/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/mapreduce/*:/root/yarn_study/hadoop-2.7.3/contrib/capacity-scheduler/*.jar:/root/yarn_study/hadoop-2.7.3/contrib/capacity-scheduler/*.jar:/root/yarn_study/hadoop-2.7.3/contrib/capacity-scheduler/*.jar:/root/yarn_study/hadoop-2.7.3/share/hadoop/yarn/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/yarn/lib/*:/root/yarn_study/hadoop-2.7.3/etc/hadoop/rm-config/log4j.properties org.apache.hadoop.yarn.server.resourcemanager.ResourceManager

3.4sbin/yarn-daemons.sh启动nodemanager

3.4.1 libexec/yarn-config.sh【结束】

3.4.2sbin/slaves.sh

3.4.2.1libexec/hadoop-config.sh【结束】

3.4.2.2/etc/hadoop/hadoop-env.sh

3.4.2.3循环连接slave执行远程命令

cd /root/yarn_study/hadoop-2.7.3 ;

/root/yarn_study/hadoop-2.7.3/sbin/yarn-daemon.sh --config /root/yarn_study/hadoop-2.7.3/etc/hadoop start nodemanager

这个就类似于3.3的命令了&＃xff01;

回去处理3.3就可以&＃xff01;

4 本质调用的/bin/yarn --config $YARN_CONF_DIR $command "$&＃64;"

细节不说了&＃xff01;

5总结2个的jdb命令

5.1 resourcemananger

/root/jdk1.8.0_181/bin/java -Dproc_resourcemanager -Xmx100m -Dhadoop.log.dir&＃61;/root/yarn_study/hadoop-2.7.3/logs -Dyarn.log.dir&＃61;/root/yarn_study/hadoop-2.7.3/logs -Dhadoop.log.file&＃61;yarn-root-resourcemanager-machine0.log -Dyarn.log.file&＃61;yarn-root-resourcemanager-machine0.log -Dyarn.home.dir&＃61; -Dyarn.id.str&＃61;root -Dhadoop.root.logger&＃61;INFO,RFA -Dyarn.root.logger&＃61;INFO,RFA -Djava.library.path&＃61;/root/yarn_study/hadoop-2.7.3/lib/native -Dyarn.policy.file&＃61;hadoop-policy.xml -Dhadoop.log.dir&＃61;/root/yarn_study/hadoop-2.7.3/logs -Dyarn.log.dir&＃61;/root/yarn_study/hadoop-2.7.3/logs -Dhadoop.log.file&＃61;yarn-root-resourcemanager-machine0.log -Dyarn.log.file&＃61;yarn-root-resourcemanager-machine0.log -Dyarn.home.dir&＃61;/root/yarn_study/hadoop-2.7.3 -Dhadoop.home.dir&＃61;/root/yarn_study/hadoop-2.7.3 -Dhadoop.root.logger&＃61;INFO,RFA -Dyarn.root.logger&＃61;INFO,RFA -Djava.library.path&＃61;/root/yarn_study/hadoop-2.7.3/lib/native -classpath /root/yarn_study/hadoop-2.7.3/etc/hadoop:/root/yarn_study/hadoop-2.7.3/etc/hadoop:/root/yarn_study/hadoop-2.7.3/etc/hadoop:/root/yarn_study/hadoop-2.7.3/share/hadoop/common/lib/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/common/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/hdfs:/root/yarn_study/hadoop-2.7.3/share/hadoop/hdfs/lib/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/hdfs/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/yarn/lib/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/yarn/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/mapreduce/lib/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/mapreduce/*:/root/yarn_study/hadoop-2.7.3/contrib/capacity-scheduler/*.jar:/root/yarn_study/hadoop-2.7.3/contrib/capacity-scheduler/*.jar:/root/yarn_study/hadoop-2.7.3/contrib/capacity-scheduler/*.jar:/root/yarn_study/hadoop-2.7.3/share/hadoop/yarn/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/yarn/lib/*:/root/yarn_study/hadoop-2.7.3/etc/hadoop/rm-config/log4j.properties org.apache.hadoop.yarn.server.resourcemanager.ResourceManager

5.2nodemanange

/root/jdk1.8.0_181/bin/java -Dproc_nodemanager -Xmx100m -Dhadoop.log.dir&＃61;/root/yarn_study/hadoop-2.7.3/logs -Dyarn.log.dir&＃61;/root/yarn_study/hadoop-2.7.3/logs -Dhadoop.log.file&＃61;yarn-root-nodemanager-machine1.log -Dyarn.log.file&＃61;yarn-root-nodemanager-machine1.log -Dyarn.home.dir&＃61; -Dyarn.id.str&＃61;root -Dhadoop.root.logger&＃61;INFO,RFA -Dyarn.root.logger&＃61;INFO,RFA -Djava.library.path&＃61;/root/yarn_study/hadoop-2.7.3/lib/native -Dyarn.policy.file&＃61;hadoop-policy.xml -server -Dhadoop.log.dir&＃61;/root/yarn_study/hadoop-2.7.3/logs -Dyarn.log.dir&＃61;/root/yarn_study/hadoop-2.7.3/logs -Dhadoop.log.file&＃61;yarn-root-nodemanager-machine1.log -Dyarn.log.file&＃61;yarn-root-nodemanager-machine1.log -Dyarn.home.dir&＃61;/root/yarn_study/hadoop-2.7.3 -Dhadoop.home.dir&＃61;/root/yarn_study/hadoop-2.7.3 -Dhadoop.root.logger&＃61;INFO,RFA -Dyarn.root.logger&＃61;INFO,RFA -Djava.library.path&＃61;/root/yarn_study/hadoop-2.7.3/lib/native -classpath /root/yarn_study/hadoop-2.7.3/etc/hadoop:/root/yarn_study/hadoop-2.7.3/etc/hadoop:/root/yarn_study/hadoop-2.7.3/etc/hadoop:/root/yarn_study/hadoop-2.7.3/share/hadoop/common/lib/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/common/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/hdfs:/root/yarn_study/hadoop-2.7.3/share/hadoop/hdfs/lib/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/hdfs/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/yarn/lib/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/yarn/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/mapreduce/lib/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/mapreduce/*:/root/yarn_study/hadoop-2.7.3/contrib/capacity-scheduler/*.jar:/root/yarn_study/hadoop-2.7.3/contrib/capacity-scheduler/*.jar:/root/yarn_study/hadoop-2.7.3/share/hadoop/yarn/*:/root/yarn_study/hadoop-2.7.3/share/hadoop/yarn/lib/*:/root/yarn_study/hadoop-2.7.3/etc/hadoop/nm-config/log4j.properties org.apache.hadoop.yarn.server.nodemanager.NodeManager