作者：8877Lyt_953 | 来源：互联网 | 2023-09-12 10:20
本文介绍brewinstallhive并修改默认的metastore存储方案，改Derby数据库为mysql的方法以及可能遇到的问题的解决方案。1.通过homebrew安装hive
本文介绍brew install hive并修改默认的metastore存储方案，改Derby数据库为mysql的方法以及可能遇到的问题的解决方案。
1. 通过homebrew安装hive
2. 添加hadoop和hive的环境变量
|
1
2
3
4
5
6
|
sudo
vim ~/.bash_profile
export
HADOOP_HOME=
/usr/local/Cellar/hadoop/hadoop
.version.no
export
HIVE_HOME=
/usr/local/Cellar/hive/hive
.version.no
/libexec
source
~/.bash_profile
|
3. 下载mysql connector
|
1
2
3
|
curl -L
'http://www.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.22.tar.gz/from/http://mysql.he.net/'
|
tar
xz
sudo
cp
mysql-connector-java-5.1.15
/mysql-connector-java-5
.1.22-bin.jar
/usr/local/Cellar/hive/hive
.version.no
/libexec/lib/
|
4. 创建mysql metastore
|
1
2
3
4
|
mysql>
CREATE
DATABASE
metastore;
mysql> USE metastore;
mysql>
CREATE
USER
'hiveuser'
@
'localhost'
IDENTIFIED
BY
'password'
;
mysql>
GRANT
SELECT
,
INSERT
,
UPDATE
,
DELETE
,
ALTER
,
CREATE
ON
metastore.*
TO
'hiveuser'
@
'localhost'
;
|
5. 配置hive的配置文件
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
|
cd
/usr/local/Cellar/hive/hive
.version.no
/libexec/conf
cp
hive-default.xml.template hive-site.xml
#添加或者编辑如下内容
javax.jdo.option.ConnectionURL<
/name
>
jdbc:mysql:
//localhost/metastore
<
/value
>
<
/property
>
javax.jdo.option.ConnectionDriverName<
/name
>
com.mysql.jdbc.Driver<
/value
>
<
/property
>
javax.jdo.option.ConnectionUserName<
/name
>
hiveuser<
/value
>
<
/property
>
javax.jdo.option.ConnectionPassword<
/name
>
password<
/value
>
<
/property
>
datanucleus.fixedDatastore<
/name
>
false
<
/value
>
<
/property
>
|
6. 测试hive是否工作
|
1
2
3
|
$ hive;
hive > show tables;
hive> create table temp_table temp_col string;
|
7. Revoke few permissions on the mysql metastore
|
1
2
|
$ mysql
mysql> REVOKE ALTER,CREATE ON metastore.* FROM
'hiveuser'
@
'localhost'
;
|
9. Further troubleshooting :
(a) If you get a bin log error saying statement format is not support. Login to your mysql console as root
$ mysql -uroot
mysql > SET GLOBAL binlog_format = 'ROW';
(b) You could also try reading the logs as follows. Logs can be emitted to the bash prompt while running hive by setting hive.root.logger to INFO,console.
$ hive -hiveconf hive.root.logger=INFO,console
(c)You could also read the raw hive logs which is usually located at /tmp/user_name/hive.log
(d)If you still have any errors, feel free to comment.
在配置完成后，可能遇到的问题解决方案：
一，Reference error
解决方案：
使用sequel pro， 修改hive用户的权限，添加Reference的global权限。
二，启动hive时遇到的" Caused by: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D "
解决方案：
修改hive-site.xml 配置中的以下key value即可：
hive.exec.scratchdir
/tmp/hive-${user.name}
hive.exec.local.scratchdir
/tmp/${user.name}
hive.downloaded.resources.dir
/tmp/${user.name}_resources
hive.scratch.dir.permission
733
restart hive metastore and hiveserver2
京公网安备 11010802041100号