黑猴子的家：Sqoop命令参数：merge

将HDFS中不同目录下面的数据合并在一起并放入指定目录中

1、数据

new_staff

1 AAA male
2 BBB male
3 CCC male
4 DDD male


old_staff

1 AAA female
2 CCC female
3 BBB female
6 DDD female


尖叫提示&＃xff1a;上边数据的列之间的分隔符应该为\t&＃xff0c;行与行之间的分割符为\n&＃xff0c;不要复制&＃xff0c;在vim编辑器里面手敲

2、创建数据&＃xff0c;上传hdfs

[yinggu&＃64;hadoop102 sqoop]$ mkdir tdata
[yinggu&＃64;hadoop102 sqoop]$ cd tdata/
[yinggu&＃64;hadoop102 tdata]$ mkdir newdata
[yinggu&＃64;hadoop102 tdata]$ mkdir olddata
[yinggu&＃64;hadoop102 tdata]$ vim newdata/new.txt
[yinggu&＃64;hadoop102 tdata]$ vim olddata/old.txt
[yinggu&＃64;hadoop102 sqoop]$ ../hadoop-2.8.2/bin/hadoop fs -put tdata/ /


3、创建JavaBean

[victor&＃64;node1 sqoop-1.4.7]$ bin/sqoop codegen \
--connect jdbc:mysql://node1:3306/company \
--username root \
--password 000000 \
--table staff \
--bindir /opt/module/sqoop/staff \
--class-name Staff \
--fields-terminated-by "\t"


4、开始合并

[victor&＃64;node1 sqoop-1.4.7]$ bin/sqoop merge \
--new-data /tdata/newdata/ \
--onto /tdata/olddata/ \
--target-dir /tdata/merged \
--jar-file /opt/module/sqoop/staff/Staff.jar \
--class-name Staff \
--merge-key id


5、结果

1 AAA MALE
2 BBB MALE
3 CCC MALE
4 DDD MALE
6 DDD FEMALE


6、参数