热门标签 | HotTags
当前位置:  开发笔记 > 编程语言 > 正文

Greenplum的安装

Geenplum是一款基于postgresql开发的mpp数据库,适合于数据仓库数据存储,是对hadoop无法有效处理结构化数据的一个很好的补充。五年前曾经使用过greenplum
Geenplum是一款基于postgresql开发的mpp数据库,适合于数据仓库数据存储,是对hadoop无法有效处理结构化数据的一个很好的补充。五年前曾经使用过greenplum,那时的版本还是3.6,现在已经更新到4.3.5了,有消息称今年将开源。和以前的相比,安装方式也有了一些小变化,简记如下,如要更详细的说明,还是得看官方文档。

一、测试环境
CentOS 6.5 8台虚机
Greenplum 4.3.5

二、环境准备
1.修改系统参数
编辑 /etc/sysctl.conf ,以下是最小配置 
kernel.shmmax = 500000000
kernel.shmmni = 4096
kernel.shmall = 4000000000
kernel.sem = 250 512000 100 2048
kernel.sysrq = 1
kernel.core_uses_pid = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.msgmni = 2048
net.ipv4.tcp_synCOOKIEs = 1
net.ipv4.ip_forward = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.conf.all.arp_filter = 1
net.ipv4.ip_local_port_range = 1025 65535
net.core.netdev_max_backlog = 10000
net.core.rmem_max = 2097152
net.core.wmem_max = 2097152
vm.overcommit_memory = 2  

2.修改文件打开数等限制
编辑/etc/security/limits.conf
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072

编辑/etc/security/limits.d/90-nproc.conf
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072

3.修改scheduler
编辑scheduler,原先默认是cfq
echo deadline > /sys/block/vda/queue/scheduler
[root@bigdata03 ~]# more  /sys/block/vda/queue/scheduler 
noop anticipatory [deadline] cfq 

4.编辑/boot/grub/grub.conf
新增 elevator=deadline
[root @bigdata03 ~]# more /boot/grub/grub.conf
...elevator=deadline crashkernel=auto ...

5.编辑设备块大小
[root@bigdata03 ~]# /sbin/blockdev --setra 16384 /dev/vda
[root@bigdata03 ~]# /sbin/blockdev --getra /dev/vda
16384

6.配置hosts文件
编辑/etc/hosts,添加,选中203作为master,其他为segment
172.30.13.203   bigdata03.rdchz mdw	
172.30.13.204	bigdata04.rdchz sdw1
172.30.13.205	bigdata05.rdchz sdw2
172.30.13.206	bigdata06.rdchz sdw3
172.30.13.207	bigdata07.rdchz sdw4
172.30.13.208	bigdata08.rdchz sdw5
172.30.13.209	bigdata09.rdchz sdw6
172.30.13.210	bigdata10.rdchz sdw7

7.分别创建各数据节点文件(一个含master,一个不含)
[root@bigdata03 ~]# more /tmp/hostfile_exkeys 
mdw
sdw1
sdw2
sdw3
sdw4
sdw5
sdw6
sdw7

[root@bigdata03 data]# more /tmp/hostfile_segments 
sdw1
sdw2
sdw3
sdw4
sdw5
sdw6
sdw7
三、集群安装
1.软件下载,上传、解压缩
下载地址:https://network.pivotal.io/products/pivotal-gpdb
unzip greenplum-db-4.3.5.2-build-1-RHEL5-x86_64.zip
/bin/bash greenplum-db-4.3.5.2-build-1-RHEL5-x86_64.bin

输入yes同意aggrement
输入安装路径,比如/usr/local/gpmaster

最后安装完后,结果是这样的:
[root@bigdata03 gpmaster]# ll
total 284
drwxr-xr-x 3  gpadmin gpadmin 4096 Jun 24 12:29 bin
drwxr-xr-x 2  gpadmin goadmin 4096 Jun 24 12:09 demo
drwxr-xr-x 5  gpadmin gpadmin 4096 Jun 24 12:09 docs
drwxr-xr-x 2  gpadmin gpadmin 4096 Jun 24 12:09 etc
drwxr-xr-x 3  gpadmin gpadmin 4096 Jun 24 12:09 ext
-rw-r--r-- 1  gpadmin gpadmin 43025 Jun 24 12:29 GPDB-LICENSE.txt
-rw-r--r-- 1  gpadmin gpadmin 664 Jul 28 14:51 greenplum_path.sh
drwxr-xr-x 6  gpadmin gpadmin 4096 Jun 24 12:09 include
drwxr-xr-x 9  gpadmin gpadmin 12288 Jun 24 12:09 lib
-rw-r--r-- 1  gpadmin gpadmin 193083 Jun 24 12:29 LICENSE.thirdparty
drwxr-xr-x 2  gpadmin gpadmin 4096 Jun 24 12:29 sbin
drwxr-xr-x 3  gpadmin gpadmin 4096 Jun 24 11:57 share

2.建立master与各节点的信任关系
[root@bigdata03 ~]# source /usr/local/greenplum-db/greenplum_path.sh
[root@bigdata03 ~]# gpssh-exkeys -f /tmp/hostfile_exkeys 
[STEP 1 of 5] create local ID and authorize on local host
[STEP 2 of 5] keyscan all hosts and update known_hosts file
[STEP 3 of 5] authorize current user on remote hosts
  ... send to sdw1
  ***
  *** Enter password for sdw1: 
  ... send to sdw2
  ... send to sdw3
  ... send to sdw4
  ... send to sdw5
  ... send to sdw6
  ... send to sdw7
[STEP 4 of 5] determine common authentication file content
[STEP 5 of 5] copy authentication files to all remote hosts
  ... finished key exchange with sdw1
  ... finished key exchange with sdw2
  ... finished key exchange with sdw3
  ... finished key exchange with sdw4
  ... finished key exchange with sdw5
  ... finished key exchange with sdw6
  ... finished key exchange with sdw7
[INFO] completed successfully

实质就是master与各个节点创建了个ssh的信任关系,信任的key存放在/root/.ssh/authorized_keys里
--测试是否无需密码登陆,ok
[root@bigdata03 ~]# ssh root@172.30.13.204
Last login: Wed Jul 29 08:45:58 2015 from 172.30.11.31
[root@bigdata04 ~]# exit
logout
Connection to 172.30.13.204 closed.
[root@bigdata03 ~]#
[root@bigdata03 ~]# ssh root@sdw7
Last login: Wed Jul 29 08:49:20 2015 from 172.30.11.31
[root@bigdata10 ~]# exit
logout
Connection to sdw7 closed.
[root@bigdata03 ~]#

3.安装各个节点准备
首先把包传送到各个节点上,并解压缩以及创建用户,组等
[root@bigdata03 ~]# gpseginstall -f /tmp/hostfile_exkeys -u gpadmin -p 123456
20150729:09:20:20:029326 gpseginstall:bigdata03:root-[INFO]:-Installation Info:
link_name greenplum-db
binary_path /usr/local/gpmaster
binary_dir_location /usr/local
binary_dir_name gpmaster
20150729:09:20:20:029326 gpseginstall:bigdata03:root-[INFO]:-check cluster password access
20150729:09:20:22:029326 gpseginstall:bigdata03:root-[INFO]:-de-duplicate hostnames
20150729:09:20:22:029326 gpseginstall:bigdata03:root-[INFO]:-master hostname: bigdata03.rdchz
20150729:09:20:23:029326 gpseginstall:bigdata03:root-[INFO]:-check for user gpadmin on cluster
20150729:09:20:24:029326 gpseginstall:bigdata03:root-[INFO]:-add user gpadmin on master
20150729:09:20:24:029326 gpseginstall:bigdata03:root-[INFO]:-add user gpadmin on cluster
20150729:09:20:24:029326 gpseginstall:bigdata03:root-[INFO]:-chown -R gpadmin:gpadmin /usr/local/greenplum-db
20150729:09:20:24:029326 gpseginstall:bigdata03:root-[INFO]:-chown -R gpadmin:gpadmin /usr/local/gpmaster
20150729:09:20:25:029326 gpseginstall:bigdata03:root-[INFO]:-rm -f /usr/local/gpmaster.tar; rm -f /usr/local/gpmaster.tar.gz
20150729:09:20:25:029326 gpseginstall:bigdata03:root-[INFO]:-cd /usr/local; tar cf gpmaster.tar gpmaster
20150729:09:20:27:029326 gpseginstall:bigdata03:root-[INFO]:-gzip /usr/local/gpmaster.tar
20150729:09:20:54:029326 gpseginstall:bigdata03:root-[INFO]:-remote command: mkdir -p /usr/local
20150729:09:20:55:029326 gpseginstall:bigdata03:root-[INFO]:-remote command: rm -rf /usr/local/gpmaster
20150729:09:20:56:029326 gpseginstall:bigdata03:root-[INFO]:-scp software to remote location
20150729:09:21:00:029326 gpseginstall:bigdata03:root-[INFO]:-remote command: gzip -f -d /usr/local/gpmaster.tar.gz
20150729:09:21:06:029326 gpseginstall:bigdata03:root-[INFO]:-md5 check on remote location
20150729:09:21:09:029326 gpseginstall:bigdata03:root-[INFO]:-remote command: cd /usr/local; tar xf gpmaster.tar
20150729:09:21:11:029326 gpseginstall:bigdata03:root-[INFO]:-remote command: rm -f /usr/local/gpmaster.tar
20150729:09:21:12:029326 gpseginstall:bigdata03:root-[INFO]:-remote command: cd /usr/local; rm -f greenplum-db; ln -fs gpmaster greenplum-db
20150729:09:21:13:029326 gpseginstall:bigdata03:root-[INFO]:-remote command: chown -R gpadmin:gpadmin /usr/local/greenplum-db
20150729:09:21:14:029326 gpseginstall:bigdata03:root-[INFO]:-remote command: chown -R gpadmin:gpadmin /usr/local/gpmaster
20150729:09:21:15:029326 gpseginstall:bigdata03:root-[INFO]:-rm -f /usr/local/gpmaster.tar.gz
20150729:09:21:15:029326 gpseginstall:bigdata03:root-[INFO]:-Changing system passwords ...
20150729:09:21:17:029326 gpseginstall:bigdata03:root-[INFO]:-exchange ssh keys for user root
20150729:09:21:22:029326 gpseginstall:bigdata03:root-[INFO]:-exchange ssh keys for user gpadmin
20150729:09:21:30:029326 gpseginstall:bigdata03:root-[INFO]:-/usr/local/greenplum-db/./sbin/gpfixuserlimts -f /etc/security/limits.conf -u gpadmin
20150729:09:21:30:029326 gpseginstall:bigdata03:root-[INFO]:-remote command: . /usr/local/greenplum-db/./greenplum_path.sh; /usr/local/greenplum-db/./sbin/gpfixuserlimts -f /etc/security/limits.conf -u gpadmin
20150729:09:21:31:029326 gpseginstall:bigdata03:root-[INFO]:-version string on master: gpssh version 4.3.5.2 build 1
20150729:09:21:31:029326 gpseginstall:bigdata03:root-[INFO]:-remote command: . /usr/local/greenplum-db/./greenplum_path.sh; /usr/local/greenplum-db/./bin/gpssh --version
20150729:09:21:32:029326 gpseginstall:bigdata03:root-[INFO]:-remote command: . /usr/local/gpmaster/greenplum_path.sh; /usr/local/gpmaster/bin/gpssh --version
20150729:09:21:38:029326 gpseginstall:bigdata03:root-[INFO]:-SUCCESS -- Requested commands completed
[root@bigdata03 ~]# 
--或者用gp自带的工具包测试能否无密码登陆已经安装包是否已经被解压到各节点上面,正常情况下会显示出各个节点的gphome信息
[root@bigdata03 ~]# gpssh -f hostfile_exkeys -e ls -l $GPHOME

4.创建各个节点的数据文件夹,并授权,保证数据文件夹是干净的
[root@bigdata03 data]# gpssh -f /tmp/hostfile_segments -e ‘mkdir -p /data/primary‘
[sdw4] mkdir -p /data/primary
[sdw3] mkdir -p /data/primary
[sdw5] mkdir -p /data/primary
[sdw1] mkdir -p /data/primary
[sdw6] mkdir -p /data/primary
[sdw2] mkdir -p /data/primary
[sdw7] mkdir -p /data/primary
[root@bigdata03 data]# gpssh -f /tmp/hostfile_segments -e ‘chown gpadmin /data/primary‘
[sdw4] chown gpadmin /data/primary
[sdw3] chown gpadmin /data/primary
[sdw5] chown gpadmin /data/primary
[sdw1] chown gpadmin /data/primary
[sdw6] chown gpadmin /data/primary
[sdw2] chown gpadmin /data/primary
[sdw7] chown gpadmin /data/primary

5.时钟同步
[root@bigdata03 ~]# vi /etc/ntp.conf
server 172.30.11.8

各个节点上的时钟以master为准,如04节点,需要改在第一行
[root@bigdata04 ~]# vi /etc/ntp.conf
server mdw

[root@bigdata03 ~]# gpssh -f /tmp/hostfile_exkeys -v -e ‘ntpd‘
[Reset ...]
[INFO] login sdw4
[INFO] login sdw3
[INFO] login mdw
[INFO] login sdw5
[INFO] login sdw1
[INFO] login sdw6
[INFO] login sdw2
[INFO] login sdw7
[sdw4] ntpd
[sdw3] ntpd
[ mdw] ntpd
[sdw5] ntpd
[sdw1] ntpd
[sdw6] ntpd
[sdw2] ntpd
[sdw7] ntpd
[INFO] completed successfully
[Cleanup...]
[root@bigdata03 ~]#

6.检查参数设置
检查参数设置,正常情况是这样的
[root@bigdata03 ~]# gpcheck -f /tmp/hostfile_exkeys -m mdw -s sdw1
20150729:11:18:20:001805 gpcheck:bigdata03:root-[INFO]:-dedupe hostnames
20150729:11:18:20:001805 gpcheck:bigdata03:root-[INFO]:-Detected platform: Generic Linux Cluster
20150729:11:18:20:001805 gpcheck:bigdata03:root-[INFO]:-generate data on servers
20150729:11:18:20:001805 gpcheck:bigdata03:root-[INFO]:-copy data files from servers
20150729:11:18:21:001805 gpcheck:bigdata03:root-[INFO]:-delete remote tmp files
20150729:11:18:21:001805 gpcheck:bigdata03:root-[INFO]:-Using gpcheck config file: /usr/local/greenplum-db/./etc/gpcheck.cnf
20150729:11:18:21:001805 gpcheck:bigdata03:root-[INFO]:-GPCHECK_NORMAL
20150729:11:18:21:001805 gpcheck:bigdata03:root-[INFO]:-gpcheck completing...
[root@bigdata03 ~]# 

7.检查网络
千兆的交换机,只能用千兆的网卡,网络吞吐一般
[root@bigdata03 ~]# gpcheckperf -f /tmp/hostfile_segments -r n -d /tmp >test.out
[root@bigdata03 ~]# more test.out 
/usr/local/greenplum-db/./bin/gpcheckperf -f /tmp/hostfile_segments -r n -d /tmp

-------------------
--  NETPERF TEST
-------------------

====================
==  RESULT
====================
Netperf bisection bandwidth test
sdw1 -> sdw2 = 845.140000
sdw2 -> sdw1 = 762.840000
sdw3 -> sdw4 = 932.170000
sdw4 -> sdw3 = 1021.450000
sdw5 -> sdw6 = 701.030000
sdw6 -> sdw5 = 806.280000
sdw7 -> sdw1 = 927.360000
sdw1 -> sdw7 = 1097.420000

Summary:
sum = 7093.69 MB/sec
min = 701.03 MB/sec
max = 1097.42 MB/sec
avg = 886.71 MB/sec
median = 927.36 MB/sec

[Warning] connection between sdw1 and sdw2 is no good
[Warning] connection between sdw2 and sdw1 is no good
[Warning] connection between sdw3 and sdw4 is no good
[Warning] connection between sdw5 and sdw6 is no good
[Warning] connection between sdw6 and sdw5 is no good
[Warning] connection between sdw7 and sdw1 is no good
[root@bigdata03 ~]#

8.检查io
可以选其中几个节点测试,它会创建几个大文件来测试
[root@bigdata03 .ssh]# more /tmp/hostfile_2 
sdw1
sdw2
sdw3
[root@bigdata03 ~]# gpcheckperf  -f /tmp/hostfile_2 -r ds -D -d /data/primary/ -d /data/primary/ -d /data/primary/
/usr/local/greenplum-db/./bin/gpcheckperf -f /tmp/hostfile_2 -r ds -D -d /data/primary/ -d /data/primary/ -d /data/primary/

--------------------
--  DISK WRITE TEST
--------------------

--------------------
--  DISK READ TEST
--------------------

--------------------
--  STREAM TEST
--------------------

====================
==  RESULT
====================

 disk write avg time (sec): 1573.83
 disk write tot bytes: 379920384000
 disk write tot bandwidth (MB/s): 233.53
 disk write min bandwidth (MB/s): 65.47 [sdw2]
 disk write max bandwidth (MB/s): 86.37 [sdw1]
 -- per host bandwidth --
    disk write bandwidth (MB/s): 86.37 [sdw1]
    disk write bandwidth (MB/s): 65.47 [sdw2]
    disk write bandwidth (MB/s): 81.69 [sdw3]

 disk read avg time (sec): 17.75
 disk read tot bytes: 379920384000
 disk read tot bandwidth (MB/s): 20440.61
 disk read min bandwidth (MB/s): 6588.84 [sdw2]
 disk read max bandwidth (MB/s): 7142.13 [sdw3]
 -- per host bandwidth --
    disk read bandwidth (MB/s): 6709.64 [sdw1]
    disk read bandwidth (MB/s): 6588.84 [sdw2]
    disk read bandwidth (MB/s): 7142.13 [sdw3]

 stream tot bandwidth (MB/s): 12247.97
 stream min bandwidth (MB/s): 3392.68 [sdw3]
 stream max bandwidth (MB/s): 5069.60 [sdw2]
 -- per host bandwidth --
    stream bandwidth (MB/s): 3785.69 [sdw1]
    stream bandwidth (MB/s): 5069.60 [sdw2]
    stream bandwidth (MB/s): 3392.68 [sdw3]
四、集群初始化
做了那么多的铺垫准备后,进入真正的安装环节
在master上以gpadmin用户登陆
[root@bigdata03 ~]# su - gpadmin
[gpadmin@bigdata03 ~]$ source /usr/local/gpmaster/greenplum_path.sh
[gpadmin@bigdata03 ~]$ mkdir gpconfigs
[gpadmin@bigdata03 ~]$ cp $GPHOME/docs/cli_help/gpconfigs/gpinitsystem_config /home/gpadmin/gpconfigs/gpinitsystem_config

配置gpinitsystem_config文件,每个segment上1个数据文件夹,因为只有1块网卡,没有配置镜像
declare -a DATA_DIRECTORY=(/data/primary)
MASTER_PORT=1949
CHECK_POINT_SEGMENTS=64
ENCODING=UTF8

初始化
[gpadmin@bigdata03 gpconfigs]$ gpinitsystem -c gpconfigs/gpinitsystem_config -h /tmp/hostfile_segments
20150729:15:41:27:030870 gpinitsystem:bigdata03:gpadmin-[INFO]:-Checking configuration parameters, please wait...
20150729:15:41:27:gpinitsystem:bigdata03:gpadmin-[FATAL]:-Configuration file gpconfigs/gpinitsystem_config does not exist. Script Exiting!
[gpadmin@bigdata03 gpconfigs]$ cd ..
[gpadmin@bigdata03 ~]$ gpinitsystem -c gpconfigs/gpinitsystem_config -h /tmp/hostfile_segments
20150730:16:52:55:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Checking configuration parameters, please wait...
20150730:16:52:55:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Reading Greenplum configuration file gpconfigs/gpinitsystem_config
20150730:16:52:55:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Locale has not been set in gpconfigs/gpinitsystem_config, will set to default value
20150730:16:52:55:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Locale set to en_US.utf8
20150730:16:52:55:031413 gpinitsystem:bigdata03:gpadmin-[WARN]:-Master hostname mdw does not match hostname output
20150730:16:52:55:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Checking to see if mdw can be resolved on this host
20150730:16:52:56:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Can resolve mdw to this host
20150730:16:52:56:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-No DATABASE_NAME set, will exit following template1 updates
20150730:16:52:56:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-MASTER_MAX_CONNECT not set, will set to default value 250
20150730:16:52:56:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Checking configuration parameters, Completed
20150730:16:52:56:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Commencing multi-home checks, please wait...
.......
20150730:16:52:58:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Configuring build for standard array
20150730:16:52:58:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Commencing multi-home checks, Completed
20150730:16:52:58:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Building primary segment instance array, please wait...
.......
20150730:16:53:03:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Checking Master host
20150730:16:53:03:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Checking new segment hosts, please wait...
.......
20150730:16:53:19:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Checking new segment hosts, Completed
20150730:16:53:19:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Greenplum Database Creation Parameters
20150730:16:53:19:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:---------------------------------------
20150730:16:53:19:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Master Configuration
20150730:16:53:19:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:---------------------------------------
20150730:16:53:19:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Master instance name       = EMC Greenplum DW
20150730:16:53:19:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Master hostname            = mdw
20150730:16:53:19:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Master port                = 1949
20150730:16:53:19:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Master instance dir        = /data/master/gpseg-1
20150730:16:53:19:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Master LOCALE              = en_US.utf8
20150730:16:53:19:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Greenplum segment prefix   = gpseg
20150730:16:53:19:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Master Database            = 
20150730:16:53:19:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Master cOnnections= 250
20150730:16:53:19:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Master buffers             = 128000kB
20150730:16:53:19:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Segment cOnnections= 750
20150730:16:53:19:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Segment buffers            = 128000kB
20150730:16:53:19:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Checkpoint segments        = 64
20150730:16:53:19:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Encoding                   = UTF8
20150730:16:53:19:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Postgres param file        = Off
20150730:16:53:19:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Initdb to be used          = /usr/local/greenplum-db/bin/initdb
20150730:16:53:19:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-GP_LIBRARY_PATH is         = /usr/local/greenplum-db/lib
20150730:16:53:20:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Ulimit check               = Passed
20150730:16:53:20:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Array host connect type    = Single hostname per node
20150730:16:53:20:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Master IP address [1]      = ::1
20150730:16:53:20:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Master IP address [2]      = 172.30.13.203
20150730:16:53:20:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Master IP address [3]      = fe80::5054:ff:fec7:b4f5
20150730:16:53:20:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Standby Master             = Not Configured
20150730:16:53:20:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Primary segment #          = 1
20150730:16:53:20:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Total Database segments    = 7
20150730:16:53:20:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Trusted shell              = ssh
20150730:16:53:20:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Number segment hosts       = 7
20150730:16:53:20:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Mirroring cOnfig= OFF
20150730:16:53:20:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:----------------------------------------
20150730:16:53:20:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Greenplum Primary Segment Configuration
20150730:16:53:20:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:----------------------------------------
20150730:16:53:20:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-sdw1 	/data/primary/gpseg0 	40000 	2 	0
20150730:16:53:20:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-sdw2 	/data/primary/gpseg1 	40000 	3 	1
20150730:16:53:20:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-sdw3 	/data/primary/gpseg2 	40000 	4 	2
20150730:16:53:20:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-sdw4 	/data/primary/gpseg3 	40000 	5 	3
20150730:16:53:20:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-sdw5 	/data/primary/gpseg4 	40000 	6 	4
20150730:16:53:20:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-sdw6 	/data/primary/gpseg5 	40000 	7 	5
20150730:16:53:20:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-sdw7 	/data/primary/gpseg6 	40000 	8 	6
Continue with Greenplum creation Yy/Nn>
y
20150730:16:53:29:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Building the Master instance database, please wait...
20150730:16:54:21:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Starting the Master in admin mode
20150730:16:54:32:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Commencing parallel build of primary segment instances
20150730:16:54:32:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Spawning parallel processes    batch [1], please wait...
.......
20150730:16:54:32:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Waiting for parallel processes batch [1], please wait...
................................................................................................................
20150730:16:56:25:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:------------------------------------------------
20150730:16:56:25:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Parallel process exit status
20150730:16:56:26:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:------------------------------------------------
20150730:16:56:26:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Total processes marked as completed           = 7
20150730:16:56:26:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Total processes marked as killed              = 0
20150730:16:56:26:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Total processes marked as failed              = 0
20150730:16:56:26:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:------------------------------------------------
20150730:16:56:26:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Deleting distributed backout files
20150730:16:56:26:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Removing back out file
20150730:16:56:26:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-No errors generated from parallel processes
20150730:16:56:26:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Restarting the Greenplum instance in production mode
20150730:16:56:26:022809 gpstop:bigdata03:gpadmin-[INFO]:-Starting gpstop with args: -a -i -m -d /data/master/gpseg-1
20150730:16:56:26:022809 gpstop:bigdata03:gpadmin-[INFO]:-Gathering information and validating the environment...
20150730:16:56:26:022809 gpstop:bigdata03:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information
20150730:16:56:26:022809 gpstop:bigdata03:gpadmin-[INFO]:-Obtaining Segment details from master...
20150730:16:56:26:022809 gpstop:bigdata03:gpadmin-[INFO]:-Greenplum Version: ‘postgres (Greenplum Database) 4.3.5.2 build 1‘
20150730:16:56:27:022809 gpstop:bigdata03:gpadmin-[INFO]:-There are 0 connections to the database
20150730:16:56:27:022809 gpstop:bigdata03:gpadmin-[INFO]:-Commencing Master instance shutdown with mode=‘immediate‘
20150730:16:56:27:022809 gpstop:bigdata03:gpadmin-[INFO]:-Master host=bigdata03.rdchz
20150730:16:56:27:022809 gpstop:bigdata03:gpadmin-[INFO]:-Commencing Master instance shutdown with mode=immediate
20150730:16:56:27:022809 gpstop:bigdata03:gpadmin-[INFO]:-Master segment instance directory=/data/master/gpseg-1
20150730:16:56:28:022809 gpstop:bigdata03:gpadmin-[INFO]:-Attempting forceful termination of any leftover master process
20150730:16:56:28:022809 gpstop:bigdata03:gpadmin-[INFO]:-Terminating processes for segment /data/master/gpseg-1
20150730:16:56:28:022899 gpstart:bigdata03:gpadmin-[INFO]:-Starting gpstart with args: -a -d /data/master/gpseg-1
20150730:16:56:28:022899 gpstart:bigdata03:gpadmin-[INFO]:-Gathering information and validating the environment...
20150730:16:56:28:022899 gpstart:bigdata03:gpadmin-[INFO]:-Greenplum Binary Version: ‘postgres (Greenplum Database) 4.3.5.2 build 1‘
20150730:16:56:28:022899 gpstart:bigdata03:gpadmin-[INFO]:-Greenplum Catalog Version: ‘201310150‘
20150730:16:56:28:022899 gpstart:bigdata03:gpadmin-[INFO]:-Starting Master instance in admin mode
20150730:16:56:29:022899 gpstart:bigdata03:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information
20150730:16:56:29:022899 gpstart:bigdata03:gpadmin-[INFO]:-Obtaining Segment details from master...
20150730:16:56:29:022899 gpstart:bigdata03:gpadmin-[INFO]:-Setting new master era
20150730:16:56:29:022899 gpstart:bigdata03:gpadmin-[INFO]:-Master Started...
20150730:16:56:29:022899 gpstart:bigdata03:gpadmin-[INFO]:-Shutting down master
20150730:16:56:31:022899 gpstart:bigdata03:gpadmin-[INFO]:-Commencing parallel segment instance startup, please wait...
... 
20150730:16:56:34:022899 gpstart:bigdata03:gpadmin-[INFO]:-Process results...
20150730:16:56:34:022899 gpstart:bigdata03:gpadmin-[INFO]:-----------------------------------------------------
20150730:16:56:34:022899 gpstart:bigdata03:gpadmin-[INFO]:-   Successful segment starts                                            = 7
20150730:16:56:34:022899 gpstart:bigdata03:gpadmin-[INFO]:-   Failed segment starts                                                = 0
20150730:16:56:34:022899 gpstart:bigdata03:gpadmin-[INFO]:-   Skipped segment starts (segments are marked down in configuration)   = 0
20150730:16:56:34:022899 gpstart:bigdata03:gpadmin-[INFO]:-----------------------------------------------------
20150730:16:56:34:022899 gpstart:bigdata03:gpadmin-[INFO]:-
20150730:16:56:34:022899 gpstart:bigdata03:gpadmin-[INFO]:-Successfully started 7 of 7 segment instances 
20150730:16:56:34:022899 gpstart:bigdata03:gpadmin-[INFO]:-----------------------------------------------------
20150730:16:56:34:022899 gpstart:bigdata03:gpadmin-[INFO]:-Starting Master instance bigdata03.rdchz directory /data/master/gpseg-1 
20150730:16:56:35:022899 gpstart:bigdata03:gpadmin-[INFO]:-Command pg_ctl reports Master bigdata03.rdchz instance active
20150730:16:56:35:022899 gpstart:bigdata03:gpadmin-[INFO]:-No standby master configured.  skipping...
20150730:16:56:35:022899 gpstart:bigdata03:gpadmin-[INFO]:-Database successfully started
20150730:16:56:35:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Completed restart of Greenplum instance in production mode
20150730:16:56:35:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Loading gp_toolkit...
20150730:16:56:38:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Scanning utility log file for any warning messages
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[WARN]:-*******************************************************
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[WARN]:-Scan of log file indicates that some warnings or errors
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[WARN]:-were generated during the array creation
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Please review contents of log file
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-/home/gpadmin/gpAdminLogs/gpinitsystem_20150730.log
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-To determine level of criticality
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-These messages could be from a previous run of the utility
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-that was called today!
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[WARN]:-*******************************************************
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Greenplum Database instance successfully created
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-------------------------------------------------------
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-To complete the environment configuration, please 
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-update gpadmin .bashrc file with the following
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-1. Ensure that the greenplum_path.sh file is sourced
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-2. Add "export MASTER_DATA_DIRECTORY=/data/master/gpseg-1"
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-   to access the Greenplum scripts for this instance:
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-   or, use -d /data/master/gpseg-1 option for the Greenplum scripts
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-   Example gpstate -d /data/master/gpseg-1
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Script log file = /home/gpadmin/gpAdminLogs/gpinitsystem_20150730.log
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-To remove instance, run gpdeletesystem utility
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-To initialize a Standby Master Segment for this Greenplum instance
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Review options for gpinitstandby
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-------------------------------------------------------
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-The Master /data/master/gpseg-1/pg_hba.conf post gpinitsystem
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-has been configured to allow all hosts within this new
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-array to intercommunicate. Any hosts external to this
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-new array must be explicitly added to this file
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-Refer to the Greenplum Admin support guide which is
20150730:16:56:39:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-located in the /usr/local/greenplum-db/docs directory
20150730:16:56:40:031413 gpinitsystem:bigdata03:gpadmin-[INFO]:-------------------------------------------------------
[gpadmin@bigdata03 ~]$

连接
[gpadmin@bigdata03 ~]$ psql -p 1949 -d postgres
psql (8.2.15)
Type "help" for help.

postgres=# \l
                  List of databases
   Name    |  Owner  | Encoding |  Access privileges  
-----------+---------+----------+---------------------
 postgres  | gpadmin | UTF8     | 
 template0 | gpadmin | UTF8     | =c/gpadmin          
                                : gpadmin=CTc/gpadmin
 template1 | gpadmin | UTF8     | =c/gpadmin          
                                : gpadmin=CTc/gpadmin
(3 rows)

postgres=# \q
[gpadmin@bigdata03 ~]$

关闭与启动
关闭与启动
[gpadmin@bigdata03 ~]$ gpstart
[gpadmin@bigdata03 ~]$ gpstop -M fast

Greenplum的安装


推荐阅读
  • 高质量SQL书写的30条建议
    本文提供了30条关于优化SQL的建议,包括避免使用select *,使用具体字段,以及使用limit 1等。这些建议是基于实际开发经验总结出来的,旨在帮助读者优化SQL查询。 ... [详细]
  • 在说Hibernate映射前,我们先来了解下对象关系映射ORM。ORM的实现思想就是将关系数据库中表的数据映射成对象,以对象的形式展现。这样开发人员就可以把对数据库的操作转化为对 ... [详细]
  • 本文介绍了使用Java实现大数乘法的分治算法,包括输入数据的处理、普通大数乘法的结果和Karatsuba大数乘法的结果。通过改变long类型可以适应不同范围的大数乘法计算。 ... [详细]
  • HDU 2372 El Dorado(DP)的最长上升子序列长度求解方法
    本文介绍了解决HDU 2372 El Dorado问题的一种动态规划方法,通过循环k的方式求解最长上升子序列的长度。具体实现过程包括初始化dp数组、读取数列、计算最长上升子序列长度等步骤。 ... [详细]
  • 本文讨论了Alink回归预测的不完善问题,指出目前主要针对Python做案例,对其他语言支持不足。同时介绍了pom.xml文件的基本结构和使用方法,以及Maven的相关知识。最后,对Alink回归预测的未来发展提出了期待。 ... [详细]
  • 本文介绍了C#中数据集DataSet对象的使用及相关方法详解,包括DataSet对象的概述、与数据关系对象的互联、Rows集合和Columns集合的组成,以及DataSet对象常用的方法之一——Merge方法的使用。通过本文的阅读,读者可以了解到DataSet对象在C#中的重要性和使用方法。 ... [详细]
  • 本文介绍了OC学习笔记中的@property和@synthesize,包括属性的定义和合成的使用方法。通过示例代码详细讲解了@property和@synthesize的作用和用法。 ... [详细]
  • Mac OS 升级到11.2.2 Eclipse打不开了,报错Failed to create the Java Virtual Machine
    本文介绍了在Mac OS升级到11.2.2版本后,使用Eclipse打开时出现报错Failed to create the Java Virtual Machine的问题,并提供了解决方法。 ... [详细]
  • 本文介绍了在SpringBoot中集成thymeleaf前端模版的配置步骤,包括在application.properties配置文件中添加thymeleaf的配置信息,引入thymeleaf的jar包,以及创建PageController并添加index方法。 ... [详细]
  • 知识图谱——机器大脑中的知识库
    本文介绍了知识图谱在机器大脑中的应用,以及搜索引擎在知识图谱方面的发展。以谷歌知识图谱为例,说明了知识图谱的智能化特点。通过搜索引擎用户可以获取更加智能化的答案,如搜索关键词"Marie Curie",会得到居里夫人的详细信息以及与之相关的历史人物。知识图谱的出现引起了搜索引擎行业的变革,不仅美国的微软必应,中国的百度、搜狗等搜索引擎公司也纷纷推出了自己的知识图谱。 ... [详细]
  • 《数据结构》学习笔记3——串匹配算法性能评估
    本文主要讨论串匹配算法的性能评估,包括模式匹配、字符种类数量、算法复杂度等内容。通过借助C++中的头文件和库,可以实现对串的匹配操作。其中蛮力算法的复杂度为O(m*n),通过随机取出长度为m的子串作为模式P,在文本T中进行匹配,统计平均复杂度。对于成功和失败的匹配分别进行测试,分析其平均复杂度。详情请参考相关学习资源。 ... [详细]
  • 本文介绍了通过ABAP开发往外网发邮件的需求,并提供了配置和代码整理的资料。其中包括了配置SAP邮件服务器的步骤和ABAP写发送邮件代码的过程。通过RZ10配置参数和icm/server_port_1的设定,可以实现向Sap User和外部邮件发送邮件的功能。希望对需要的开发人员有帮助。摘要长度:184字。 ... [详细]
  • 动态规划算法的基本步骤及最长递增子序列问题详解
    本文详细介绍了动态规划算法的基本步骤,包括划分阶段、选择状态、决策和状态转移方程,并以最长递增子序列问题为例进行了详细解析。动态规划算法的有效性依赖于问题本身所具有的最优子结构性质和子问题重叠性质。通过将子问题的解保存在一个表中,在以后尽可能多地利用这些子问题的解,从而提高算法的效率。 ... [详细]
  • 本文介绍了指针的概念以及在函数调用时使用指针作为参数的情况。指针存放的是变量的地址,通过指针可以修改指针所指的变量的值。然而,如果想要修改指针的指向,就需要使用指针的引用。文章还通过一个简单的示例代码解释了指针的引用的使用方法,并思考了在修改指针的指向后,取指针的输出结果。 ... [详细]
  • 在project.properties添加#Projecttarget.targetandroid-19android.library.reference.1..Sliding ... [详细]
author-avatar
batman@zhou
这个家伙很懒,什么也没留下!
PHP1.CN | 中国最专业的PHP中文社区 | DevBox开发工具箱 | json解析格式化 |PHP资讯 | PHP教程 | 数据库技术 | 服务器技术 | 前端开发技术 | PHP框架 | 开发工具 | 在线工具
Copyright © 1998 - 2020 PHP1.CN. All Rights Reserved | 京公网安备 11010802041100号 | 京ICP备19059560号-4 | PHP1.CN 第一PHP社区 版权所有