作者:原文W | 来源:互联网 | 2023-09-07 16:09
1修改配置文件mapred-site.xml:
mapreduce.framework.name
yarn
说明mapreduce是有yarn控制
yarn-site.xml:
yarn.nodemanager.aux-services
mapreduce_shuffle
yarn.resourcemanager.ha.enabled是否开启高可用
true
yarn.resourcemanager.cluster-id
cluster1
yarn.resourcemanager.ha.rm-ids逻辑名称
rm1,rm2
yarn.resourcemanager.hostname.rm1
slave2
yarn.resourcemanager.hostname.rm2
slave3
yarn.resourcemanager.zk-address
slave1:2181,slave2:2181,slave3:2181
分发配置文件到slave1、2、3
scp mapred-site.xml yarn-site.xml slave1:/usr/local/src/hadoop-2.6.5/etc/hadoop
slave2 slave3免密钥登陆
cat id_rsa.pub >> authorized_keys
scp id_rsa.pub slave3:/root/.ssh/slave2.pub
2、开始启动集群
启动zookeeper: zkServer.sh start
启动hdfs: start-dfs.sh
启动yarn:start-yarn.sh
slave2 slave3启动Resourcemanager: yarn-daemon.sh start resourcemanager
8088端口可查看mapreduce相关情况:
利用框架jar包封装的单词统计样例测试:
hadoop jar hadoop-mapreduce-examples-2.6.5.jar wordcount /user/root/test.txt /usr(输出目录)
关闭集群:
slave2、3:yarn-daemon.sh stop resourcemanager
master:stop-all.sh