热门标签 | HotTags
当前位置:  开发笔记 > 编程语言 > 正文

Kylin下构建Cube第一步出错:shell-init:errorretrievingcurrentdirectory

问题背景:生产环境部署的Kylin-2.1,官方发布的最新安装包并不支持更改hbase存储的namespace,修改源码后重新打包部署过程中,buildcube第一步出错大概错误信息

问题背景:
生产环境部署的Kylin-2.1,官方发布的最新安装包并不支持更改hbase存储的namespace,修改源码后重新打包部署过程中,build cube第一步出错

大概错误信息是:

OS command error exit with 5 – hive -e "USE default;
DROP TABLE IF EXISTS kylin_intermediate_kylin_sales_cube_desc_20120101000000_20160502000000;
CREATE EXTERNAL TABLE IF NOT EXISTS kylin_intermediate_kylin_sales_cube_desc_20120101000000_20160502000000 ( DEFAULT_KYLIN_SALES_PART_DT date ,DEFAULT_KYLIN_SALES_LEAF_CATEG_ID bigint ,DEFAULT_KYLIN_SALES_LSTG_SITE_ID int ,DEFAULT_KYLIN_CATEGORY_GROUPINGS_META_CATEG_NAME string ,DEFAULT_KYLIN_CATEGORY_GROUPINGS_CATEG_LVL2_NAME string ,DEFAULT_KYLIN_CATEGORY_GROUPINGS_CATEG_LVL3_NAME string ,DEFAULT_KYLIN_SALES_LSTG_FORMAT_NAME string ,DEFAULT_KYLIN_SALES_PRICE decimal(19,4) ,DEFAULT_KYLIN_SALES_SELLER_ID bigint ) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\177' STORED AS SEQUENCEFILE LOCATION '/kylin/kylin_metadata/kylin-5020d91f-3114-421b-929b-f733d438411a/kylin_intermediate_kylin_sales_cube_desc_20120101000000_20160502000000';
SET dfs.replication=2;
SET dfs.block.size=33554432;
SET hive.exec.compress.output=true;
SET hive.auto.convert.join.nocOnditionaltask=true;
SET hive.auto.convert.join.noconditionaltask.size=300000000;
SET mapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.SnappyCodec;
SET mapreduce.output.fileoutputformat.compress.codec=org.apache.hadoop.io.compress.SnappyCodec;
SET hive.merge.mapfiles=true;
SET hive.merge.mapredfiles=true;
SET mapred.output.compression.type=BLOCK;
SET hive.merge.size.per.task=256000000;
SET hive.support.cOncurrency=false;
SET mapreduce.job.split.metainfo.maxsize=-1;
INSERT OVERWRITE TABLE kylin_intermediate_kylin_sales_cube_desc_20120101000000_20160502000000 SELECT KYLIN_SALES.PART_DT ,KYLIN_SALES.LEAF_CATEG_ID ,KYLIN_SALES.LSTG_SITE_ID ,KYLIN_CATEGORY_GROUPINGS.META_CATEG_NAME ,KYLIN_CATEGORY_GROUPINGS.CATEG_LVL2_NAME ,KYLIN_CATEGORY_GROUPINGS.CATEG_LVL3_NAME ,KYLIN_SALES.LSTG_FORMAT_NAME ,KYLIN_SALES.PRICE ,KYLIN_SALES.SELLER_ID FROM DEFAULT.KYLIN_SALES as KYLIN_SALES INNER JOIN DEFAULT.KYLIN_CAL_DT as KYLIN_CAL_DT ON KYLIN_SALES.PART_DT = KYLIN_CAL_DT.CAL_DT INNER JOIN DEFAULT.KYLIN_CATEGORY_GROUPINGS as KYLIN_CATEGORY_GROUPINGS ON KYLIN_SALES.LEAF_CATEG_ID = KYLIN_CATEGORY_GROUPINGS.LEAF_CATEG_ID AND KYLIN_SALES.LSTG_SITE_ID = KYLIN_CATEGORY_GROUPINGS.SITE_ID WHERE (KYLIN_SALES.PART_DT >= '2012-01-01' AND KYLIN_SALES.PART_DT <'2016-05-02') ;
"
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
Unable to determine Hadoop version information.
'hadoop version' returned:
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
Error occurred during initialization of VM java.lang.Error: Properties init: Could not determine current working directory. at java.lang.System.initProperties(Native Method) at java.lang.System.initializeSystemClass(System.java:1119)

Kylin之前是没有问题的,构建了很多cube都没有问题,即便是出错按理讲也不会出现在第一步,第一步的动作是创建一个宽表,放到hive作为临时表。
解决思路:
第一感觉是权限问题,修改了sh脚本的执行权限,修改了hdfs上 kylin 工作目录的权限后,问题仍然存在。
重新找原因,其中有报工作目录的问题,看上去是本地目录并不是Hdfs上的目录,这才想起可能是上次重新部署Kylin的时候把旧版本的直接删除 ,但并没有停掉进程。

ps aux | grep kylin

查看进程查然有两个Kylin进程在跑,于是Kill掉旧的进程

突然想起之前有人误删了kylin的文件夹,所以去查看了一下,果然某一台机器下面存在两个kylin进程。kill 后重启就好了。

猜测的原因是原进程的安装目录已经不存在了,并行着两个Kylin
进程产生冲突。


推荐阅读
author-avatar
宝一一0702
这个家伙很懒,什么也没留下!
PHP1.CN | 中国最专业的PHP中文社区 | DevBox开发工具箱 | json解析格式化 |PHP资讯 | PHP教程 | 数据库技术 | 服务器技术 | 前端开发技术 | PHP框架 | 开发工具 | 在线工具
Copyright © 1998 - 2020 PHP1.CN. All Rights Reserved | 京公网安备 11010802041100号 | 京ICP备19059560号-4 | PHP1.CN 第一PHP社区 版权所有