热门标签 | HotTags
当前位置:  开发笔记 > 编程语言 > 正文

Hive出现returncode2fromorg.apache.hadoop.hive.ql.exec.mr.MapRedTa

returncode2fromorg.apache.hadoop.hive.ql.exec.mr.MapRedTasHadoopjobinformationforStage-

return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTas
Hadoop job information for Stage-1: number of mappers: 0; number of reducers: 0
2018-06-06 23:34:46,458 Stage-1 map = 0%, reduce = 0%
Ended Job = job_1528299123875_0001 with errors
Error during job, obtaining debugging information…
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Stage-Stage-1: HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
2018-06-06 23:48:09,482 Stage-1 map = 0%, reduce = 0%
2018-06-06 23:49:09,973 Stage-1 map = 0%, reduce = 0%
2018-06-06 23:50:10,388 Stage-1 map = 0%, reduce = 0%
2018-06-06 23:51:10,733 Stage-1 map = 0%, reduce = 0%
2018-06-06 23:52:11,271 Stage-1 map = 0%, reduce = 0%
2018-06-06 23:53:11,424 Stage-1 map = 0%, reduce = 0%
2018-06-06 23:54:11,589 Stage-1 map = 0%, reduce = 0%
2018-06-06 23:55:11,793 Stage-1 map = 0%, reduce = 0%
一直出现这个问题!!!!
 我们都知道,启用MapReduce Job是会消耗系统开销的。对于这个问题,从Hive0.10.0版本开始,对于简单的不需要聚合的类似SELECT 《col》 from 《table》 LIMIT n语句,不需要起MapReduce job,直接通过Fetch task获取数据,可以通过下面几种方法实现:
 解决方法:
 hive》 set hive.fetch.task.cOnversion=more;
hive》 SELECT id, money FROM m limit 10;
OK
1 122
1 185
1 231
1 292
1 316
1 329
1 355
1 356
1 362
1 364
Time taken: 0.138 seconds, Fetched: 10 row(s)
上面 set hive.fetch.task.cOnversion=more;开启了Fetch任务,所以对于上述简单的列查询不在启用MapReduce job!
方法二:
bin/hive –hiveconf hive.fetch.task.cOnversion=more
方法三:
上面的两种方法都可以开启了Fetch任务,但是都是临时起作用的;如果你想一直启用这个功能,可以在${HIVE_HOME}/conf/hive-site.xml里面加入以下配置:《property》
《name》hive.fetch.task.conversion《/name》
《value》more《/value》
《description》
Some select queries can be converted to single FETCH task
minimizing latency.Currently the query should be single
sourced not having any subquery and should not have
any aggregations or distincts (which incurrs RS),
lateral views and joins.
1. minimal : SELECT STAR, FILTER on partition columns, LIMIT only
2. more : SELECT, FILTER, LIMIT only (+TABLESAMPLE, virtual columns)
《/description》
《/property》

由于新版本的csdn不太熟练,使用《》 代替<>!


推荐阅读
author-avatar
美娟婉燕6386
这个家伙很懒,什么也没留下!
PHP1.CN | 中国最专业的PHP中文社区 | DevBox开发工具箱 | json解析格式化 |PHP资讯 | PHP教程 | 数据库技术 | 服务器技术 | 前端开发技术 | PHP框架 | 开发工具 | 在线工具
Copyright © 1998 - 2020 PHP1.CN. All Rights Reserved | 京公网安备 11010802041100号 | 京ICP备19059560号-4 | PHP1.CN 第一PHP社区 版权所有