Sqoop1InstallAndTestMySQL/PostgreSQL

环境:  Hadoop2.7.2(NameNode HA,ResourceManage HA)+HBASE+HIVE
环境:  Hadoop2.7.2(NameNode HA,ResourceManage HA)


1.下载
[root@sht-sgmhadoopnn-01 hadoop]# wget http://apache.01link.hk/sqoop/1.4.6/sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz

[root@sht-sgmhadoopnn-01 hadoop]# tar xzvf sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz
[root@sht-sgmhadoopnn-01 hadoop]# ln -s /hadoop/sqoop-1.4.6.bin__hadoop-2.0.4-alpha sqoop

2.配置sqoop-env.sh
[root@sht-sgmhadoopnn-01 conf]# pwd
/hadoop/sqoop/conf
[root@sht-sgmhadoopnn-01 conf]# cp sqoop-env-template.sh sqoop-env.sh
[root@sht-sgmhadoopnn-01 conf]#
[root@sht-sgmhadoopnn-01 conf]# vi sqoop-env.sh
export HADOOP_COMMON_HOME=/hadoop/hadoop
export HADOOP_MAPRED_HOME=/hadoop/hadoop/share/hadoop/mapreduce
export HBASE_HOME=/hadoop/hbase
export HIVE_HOME=/hadoop/hive

3.配置 mysql，postgresql jdbc驱动包
[root@sht-sgmhadoopnn-01 lib]# pwd
/hadoop/sqoop/lib
[root@sht-sgmhadoopnn-01 lib]# ll
total 17088
..............
..............
-rw-r--r-- 1 root root  703265 Jul 29 21:41 mysql-connector-java-5.1.6-bin.jar
..............
..............
-rw-r--r-- 1 root root  684978 Jul 30 10:13 postgresql-9.4.1209.jre6.jar
..............
..............
[root@sht-sgmhadoopnn-01 lib]#

4.测试连接mysql，列举databases
[root@sht-sgmhadoopnn-01 bin]# sqoop list-databases --connect jdbc:mysql://sht-sgmhadoopnn-01:3306/ --username root --password agm43gadsg
Warning: /hadoop/sqoop/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /hadoop/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /hadoop/zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
16/07/30 10:01:12 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
16/07/30 10:01:12 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
16/07/30 10:01:12 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
information_schema
dba
hive
hive_remote_meta
mysql
performance_schema
repl
test1

5.测试连接postgres，列举databases
[root@sht-sgmhadoopnn-01 bin]# sqoop-list-databases --connect jdbc:postgresql://172.16.101.54:5432/denali --username denaliadmin --password "rdyhdenali"
Warning: /hadoop/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /hadoop/zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
16/07/30 10:28:31 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
16/07/30 10:28:31 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
16/07/30 10:28:31 INFO manager.SqlManager: Using default fetchSize of 1000
template1
template0
postgres
denali
[root@sht-sgmhadoopnn-01 bin]#