AmbariHDP集群搭建文档

二、为每台主机关闭iptables,SELinux,启动ntpd服务

三、安装Mysql,JDK

查看linux发行版本

从网站 http://dev.mysql.com/downloads/repo/yum/ 下载对应版本yum资源

安装

yum -y install mysql57-community-release-el7-7.noarch.rpm

安装mysql数据库的服务器版本

yum -y install mysql-community-server

 

启动数据库

service   mysqld start

systemctl start  mysqld.service

 

获取初始密码

这样，判断密码的标准就基于密码的长度了。这个由validate_password_length参数来决定。

validate_password_length参数默认为8

set global validate_password_length=4;

 

安装JDK，配置环境变量

export JAVA_HOME=/usr/local/jdk1.8.0

export JRE_HOME=${JAVA_HOME}/jre

export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib

 

export PATH=${JAVA_HOME}/bin:$PATH

使环境变量立即生效

source ~/.bash_profile

四、集群服务器间配置时间同步

http://2d67df38.fromwiz.com/share/s/0JpZYU2X7kl626aE730Q_Flq2OyZYU3BDQcP2rnK4F3zASlT

五、安装Ambari

wget -P /etc/yum.repos.d/ http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.0.1/ambari.repo

 

wget -P /etc/yum.repos.d/ http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.2.2.0/ambari.repo

 

yum clean all

 

yum list|grep ambari

 

安装完后进行设置

ambari-server setup

 

yum install mysql-connector-java*

 报错：

grant all privileges on *.* to ambari@'localhost' identified by 'bigdata';

flush privileges;

admin/admin

 登录系统操作后发现还是报错：

原因：Ambari默认的 mysql jdbc 驱动不支持 5.6以上版本

解决方法：下载新的 mysql jdbc驱动

ln -s mysql-connector-java-5.1.40-bin.jar mysql-connector-java.jar

ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar

 

配置主机时报错

Permanently added 'node1.hadoop' (ECDSA) to the list of known hosts

解决方法

在 master主机用户根目录的.ssh/ssh_config 文件中新增一行

UserKnownHostsFile ~/.ssh/known_hosts

继续报错

 

 注意HOST名字要写全名

node2节点报错：

ERROR: Bootstrap of host node2.hadoop fails because previous action finished with non-zero exit code (255)
ERROR MESSAGE: tcgetattr: Invalid argument
Connection to node2.hadoop closed.

STDOUT: tput: unknown terminal "unknown"
('', None)
('', None)

wget -nv http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.2.1.0/ambari.repo -O /etc/yum.repos.d/ambari.repo

 

yum repolist

 

yum install ambari-agent

 

vi /etc/ambari-agent/conf/ambari-agent.ini

[server]

hostname=

url_port=8440

secured_url_port=8441

 

ambari-agent start


 

看后台日志，报错

pip install flask -i  http://pypi.douban.com/simple  --trusted-host  pypi.douban.com

 

删除所有ambari* from /usr/lib/python2.6/site-packages，重新安装ambari-agent,杀死ambari进程，重新启动

rm - f / usr / lib / python2 . 6 / site - packages / ambari_commons
rm - rf / usr / lib / python2 . 6 / site - packages / resource_management
rm - rf / usr / lib / python2 . 6 / site - packages / ambari_jinja2
rm - rf / var / lib / ambari - agent /
rm - rf / usr / lib / ambari - agent /
yum remove ambari-agent

yum install ambari-agent

 

后面又报错：

 

解决方法1：

（此方法未验证）

Resolved: The problem was related to proxy Authentication. So, I installed and configured CNTLM to access ISA Proxy Server on Centos 6.

Look at the article :

http://wingloon.com/2014/04/11/how-to-install-and-configure-cntlm-to-access-isa-server-on-centos-6-5/

1.Install and configure cntlm

2. configure yum to use through ISA Proxy

• vi /etc/yum.conf

add a line:

proxy=http://127.0.0.1:3128/

• vi .bash_profile

add :

export http_proxy=http://localhost:3128/

export https_proxy=${http_proxy}

export ftp_proxy=${http_proxy}

• logoff root and login again
• yum clean all
• yum install XXXX

yum update, yum repolist or yum install now works fine

 

解决方法2：

在对应报错服务器上的HDP.repo文件中添加一行   proxy = _none_

 

I have fixed it myself. For those who might face the same issue. This is what I did. Basically HDP.repo should be told to skip proxy server, but if you edit HDP.repo, Ambari wizard will overwrite it. So what is the workaround?. This is how you make HDP.repo skip the proxy. Just add the following lines to the yum.conf file for all the nodes. It worked for me. Hope this helps! :)

[HDP-2.4]

name=HDP-2.4

proxy = _none_

baseurl=http://item-70288/hdp/HDP/centos6/2.x/updates/2.4.0.0

enabled=1

gpgcheck=0