42.Impala负载均衡实现—HAProxy

42.1 操作演示

HAProxy安装及启停

• 在集群中选择一个节点&＃xff0c;使用yum方式安装HAProxy服务

[root&＃64;ip-172-31-9-33 ~]# yum -y install haproxy


• 启动与停止HAProxy服务&＃xff0c;并将服务添加到自启动列表

[root&＃64;ip-172-31-9-33 ~]# service haproxy start
[root&＃64;ip-172-31-9-33 ~]# service haproxy stop
[root&＃64;ip-172-31-9-33 ~]# chkconfig haproxy on


HAProxy配置Impala负载均衡

• 将/etc/haproxy目录下的haproxy.cfg文件备份&＃xff0c;新建haproxy.cfg文件&＃xff0c;添加如下配置
  • 主要配置了HAProxy的http状态管理界面、impalashell和impalajdbc的负载均衡。

#---------------------------------------------------------------------
# Example configuration for a possible web application. See the
# full configuration options online.
#
# http://haproxy.1wt.eu/download/1.4/doc/configuration.txt
#
#---------------------------------------------------------------------#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
globallog 127.0.0.1 local2chroot /var/lib/haproxypidfile /var/run/haproxy.pidmaxconn 4000user haproxygroup haproxydaemon# turn on stats unix socketstats socket /var/lib/haproxy/stats#---------------------------------------------------------------------
# common defaults that all the &＃39;listen&＃39; and &＃39;backend&＃39; sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaultsmode httplog globaloption httplogoption dontlognull#option http-server-close#option forwardfor except 127.0.0.0/8option redispatchretries 3timeout http-request 10stimeout queue 1mtimeout connect 10stimeout client 1mtimeout server 1mtimeout http-keep-alive 10stimeout check 10smaxconn 3000listen statsbind 0.0.0.0:1080mode httpoption httplogmaxconn 5000stats refresh 30sstats uri /stats listen impalashellbind 0.0.0.0:25003mode tcpoption tcplogbalance leastconnserver ip-172-31-9-33.fayson.com ip-172-31-9-33.fayson.com:21000 checkserver ip-172-31-5-190.fayson.com ip-172-31-5-190.fayson.com:21000 checkserver ip-172-31-10-118.fayson.com ip-172-31-10-118.fayson.com:21000 checklisten impalajdbcbind 0.0.0.0:25004mode tcpoption tcplogbalance leastconnserver ip-172-31-9-33.fayson.com ip-172-31-9-33.fayson.com:21050 checkserver ip-172-31-5-190.fayson.com ip-172-31-5-190.fayson.com:21050 checkserver ip-172-31-10-118.fayson.com ip-172-31-10-118.fayson.com:21050 check


• 重启HAProxy服务

[root&＃64;ip-172-31-9-33 haproxy]# service haproxy restart


• 浏览器访问http://{hostname}:1080/stats&＃xff0c;查看Impala服务的负载均衡是否配置成功

Impala Shell测试

• 使用多个终端同时访问&＃xff0c;并执行SQL语句&＃xff0c;查看是否会通过HAProxy服务自动负载到其它Impala Daemon节点
• 使用Impala shell访问HAProxy服务的25003端口&＃xff0c;命令如下

[root&＃64;ip-172-31-6-148 ~]# impala-shell -i ip-172-31-9-33.fayson.com:25003


• 打开第一个终端访问并执行SQL

[root&＃64;ip-172-31-6-148 ~]# impala-shell -i ip-172-31-9-33.fayson.com:25003
...
[ip-172-31-9-33.fayson.com:25003] > select * from my_first_table;
...
&＃43;----&＃43;------&＃43;
| id | name |
&＃43;----&＃43;------&＃43;
| 1 | john |
| 2 | tom |
| 3 | jim |
&＃43;----&＃43;------&＃43;
Fetched 3 row(s) in 7.20s
[ip-172-31-9-33.fayson.com:25003] >


• 同时打开第二个终端访问并执行SQL

[root&＃64;ip-172-31-6-148 ~]# impala-shell -i ip-172-31-9-33.fayson.com:25003
...
[ip-172-31-9-33.fayson.com:25003] > select * from my_first_table;
...
&＃43;----&＃43;------&＃43;
| id | name |
&＃43;----&＃43;------&＃43;
| 2 | tom |
| 3 | jim |
| 1 | john |
&＃43;----&＃43;------&＃43;
Fetched 3 row(s) in 1.81s
[ip-172-31-9-33.fayson.com:25003] >


• 通过以上测试可以看到&＃xff0c;两个终端执行的SQL不在同一个Impala Daemon&＃xff0c;这样就实现了Impala Daemon服务的负载均衡。

大数据视频推荐&＃xff1a;
CSDN