Sqoop导入-带有CDH5的镶木地板文件

我正在尝试将数据直接从mysql导入到镶木地板,但它似乎无法正常工作......

我正在使用CDH5.3,其中包括Sqoop 1.4.5.

这是我的命令行:

sqoop import --connect jdbc:mysql://xx.xx.xx.xx/database --username username --password mypass --query 'SELECT page_id,user_id FROM pages_users WHERE $CONDITIONS' --split-by page_id --hive-import --hive-table default.pages_users3 --target-dir hive_pages_users --as-parquetfile

然后我收到这个错误:

Warning: /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
15/01/09 14:31:49 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5-cdh5.3.0
15/01/09 14:31:49 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
15/01/09 14:31:49 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
15/01/09 14:31:49 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
15/01/09 14:31:49 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
15/01/09 14:31:49 INFO tool.CodeGenTool: Beginning code generation
15/01/09 14:31:50 INFO manager.SqlManager: Executing SQL statement: SELECT page_id,user_id FROM pages_users WHERE  (1 = 0) 
15/01/09 14:31:50 INFO manager.SqlManager: Executing SQL statement: SELECT page_id,user_id FROM pages_users WHERE  (1 = 0) 
15/01/09 14:31:50 INFO manager.SqlManager: Executing SQL statement: SELECT page_id,user_id FROM pages_users WHERE  (1 = 0) 
15/01/09 14:31:50 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce
Note: /tmp/sqoop-root/compile/b90e7b492f5b66554f2cca3f88ef7a61/QueryResult.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
15/01/09 14:31:51 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/b90e7b492f5b66554f2cca3f88ef7a61/QueryResult.jar
15/01/09 14:31:51 INFO mapreduce.ImportJobBase: Beginning query import.
15/01/09 14:31:51 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
15/01/09 14:31:51 INFO manager.SqlManager: Executing SQL statement: SELECT page_id,user_id FROM pages_users WHERE  (1 = 0) 
15/01/09 14:31:51 INFO manager.SqlManager: Executing SQL statement: SELECT page_id,user_id FROM pages_users WHERE  (1 = 0) 
15/01/09 14:31:51 WARN spi.Registration: Not loading URI patterns in org.kitesdk.data.spi.hive.Loader
15/01/09 14:31:51 ERROR sqoop.Sqoop: Got exception running Sqoop: org.kitesdk.data.DatasetNotFoundException: Unknown dataset URI: hive?dataset=default.pages_users3
org.kitesdk.data.DatasetNotFoundException: Unknown dataset URI: hive?dataset=default.pages_users3
    at org.kitesdk.data.spi.Registration.lookupDatasetUri(Registration.java:109)
    at org.kitesdk.data.Datasets.create(Datasets.java:189)
    at org.kitesdk.data.Datasets.create(Datasets.java:240)
    at org.apache.sqoop.mapreduce.ParquetJob.createDataset(ParquetJob.java:81)
    at org.apache.sqoop.mapreduce.ParquetJob.configureImportJob(ParquetJob.java:70)
    at org.apache.sqoop.mapreduce.DataDrivenImportJob.configureMapper(DataDrivenImportJob.java:112)
    at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:262)
    at org.apache.sqoop.manager.SqlManager.importQuery(SqlManager.java:721)
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:499)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:236)

我没有问题,将数据导入配置单元文件格式,但实木复合地板是一个问题......你有会出现这种情况的任何想法？

谢谢 :)

请不要使用.

相反,请使用--hive-database --hive-table.对于你的命令,它应该是:

    sqoop import --connect jdbc:mysql://xx.xx.xx.xx/database \
    --username username --password mypass \
    --query 'SELECT page_id,user_id FROM pages_users WHERE $CONDITIONS' --split-by page_id \
    --hive-import --hive-database default --hive-table pages_users3 \
    --target-dir hive_pages_users --as-parquetfile

--驭风--

这个家伙很懒，什么也没留下！

Tags | 热门标签

• netty
• version
• web3
• char
• bitmap
• string
• solr
• const
• cookie
• jsp
• java
• export
• shell
• php5
• web
• subset
• timezone
• require
• yaml
• default
• eval
• grid
• less
• replace
• vbscript
• php8
• format
• get
• flutter
• object

RankList | 热门文章

• 1部署Maven环境
• 2创建ipadWEB应用程序到主屏幕
• 3TigerGraph101系列课程四直播报名及往期回顾
• 4我必须更改我的数字的最后一位和第一位，但不能仅对整数或循环使用函数。例如从12345到52341
• 5Android系统启动过程分析
• 6开启 windows10上的mysql的远程连接
• 72018.3.17
• 8[分享]用于Asp.Net的敏感内容评审控件
• 9NETBpm的安装
• 10typescript 中的接口和类有什么区别
• 11vb.net打印报表,vb 打印
• 12mysql 数据库连接池小_一个Demo小例子让你了解数据库连接池实现的4种方式
• 13UPDATE能走索引还会锁全表吗
• 14小程序同一页面不同按钮转发不同样式
• 15石子合并——区间dp

PHP1.CN | 中国最专业的PHP中文社区 | DevBox开发工具箱 | json解析格式化 |PHP资讯 | PHP教程 | 数据库技术 | 服务器技术 | 前端开发技术 | PHP框架 | 开发工具 | 在线工具
Copyright © 1998 - 2020 PHP1.CN. All Rights Reserved | 京公网安备 11010802041100号 | 京ICP备19059560号-4 | PHP1.CN 第一PHP社区 版权所有