基于Feign初探RangerApi

Ranger Api之User管理

在大数据平台之权限管理组件 - Aapche Ranger一文中我们了解了Ranger以及安装部署过程以及Admin可视化界面的使用。

除了可以在可视化的Ranger Admin界面上进行权限、用户等管理外，Ranger还支持通过REST API来完成这些操作。因为我们如果要开发自己的大数据平台，可能并不会使用Ranger Admin的可视化界面，而是希望在自己的大数据平台界面去操作Ranger。有了Ranger Api的支持，我们就可以轻易实现这一点。

关于Ranger Api的官方文档如下：

• https://ranger.apache.org/apidocs/index.html

本小节简单演示下User Api的使用，User Api用于管理用户，对用户进行增删改查。首先创建一个空Maven项目。由于要通过http请求Api，所以需要用到http请求工具。这里用到的是feign，完整的pom文件内容如下：


4.0.0
org.example
ranger-client
1.0-SNAPSHOT


junit
junit
4.12


org.projectlombok
lombok
1.18.4


com.netflix.feign
feign-core
8.18.0


com.netflix.feign
feign-jackson
8.18.0


com.netflix.feign
feign-okhttp
8.18.0


com.fasterxml.jackson.core
jackson-databind
2.9.8


com.google.guava
guava
21.0


org.slf4j
slf4j-api
1.7.25






org.apache.maven.plugins
maven-compiler-plugin
3.8.0

1.8
1.8






首先定义一个配置类，配置用于访问ranger api的用户名密码：

package com.example.ranger.config;
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@JsonIgnoreProperties(ignoreUnknown = true)
public class RangerAuthConfig {
private String username = "admin";
private String password = "admin";
}

再定义一个配置类，用于提供http客户端配置：

package com.example.ranger.config;
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import feign.Logger;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@JsonIgnoreProperties(ignoreUnknown = true)
public class RangerClientConfig {
private int cOnnectionTimeoutMills= 5 * 1000;
private int readTimeoutMills = 30 * 1000;
private Logger.Level level = Logger.Level.BASIC;
private String url = "http://192.168.243.161:6080";
private RangerAuthConfig authCOnfig= new RangerAuthConfig();
}

声明一个请求拦截器，用于在发起请求之前添加一些请求头：

package com.example.ranger.interceptor;
import feign.RequestInterceptor;
import feign.RequestTemplate;
public class RangerHeadersInterceptor implements RequestInterceptor {
@Override
public void apply(RequestTemplate template) {
template.header("Accept", "application/json");
template.header("X-XSRF_HEADER", "\"\"");
template.header("Content-Type", "application/json");
}
}

通常在实际的开发中，我们会定义一个业务异常，用于对异常信息进行自定义封装：

package com.example.ranger.exception;
import java.io.Serializable;
public class RangerClientException extends RuntimeException implements Serializable {
private static final long serialVersiOnUID= -4441189815976639860L;
private Throwable cause;
private int status;
private String message;
public RangerClientException(int status, String message) {
this.status = status;
this.message = message;
}
@Override
public String getMessage() {
return String.format("%s http status = %s", message, status);
}
@Override
public String toString() {
return String.format("%s http status = %s", message, status);
}
}

自定义一个异常信息解析器，当请求发生异常时，feign会调用该方法返回我们自定义的异常类：

package com.example.ranger.decoder;
import com.example.ranger.exception.RangerClientException;
import feign.Response;
import feign.Util;
import feign.codec.ErrorDecoder;
import java.io.IOException;
public class RangerErrorDecoder implements ErrorDecoder {
@Override
public Exception decode(String methodKey, Response response) {
return new RangerClientException(
response.status(), errorMessage(methodKey, response)
);
}
private String errorMessage(String methodKey, Response response) {
String msg = String.format("status %s reading %s", response.status(), methodKey);
if (response.body() != null) {
try {
msg += "content:\n" + Util.toString(response.body().asReader());
} catch (IOException e) {
e.printStackTrace();
}
}
return msg;
}
}

完成上面feign相关的前置准备后，我们就可以开始编写请求ranger api的代码了。首先，定义用户接口的请求和响应实体：

package com.example.ranger.model;
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;
import java.util.List;
/**
* 用户信息实体类
* https://ranger.apache.org/apidocs/json_VXUser.html
*
* @author 01
* @date 2020-11-12
**/
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@JsonIgnoreProperties(ignoreUnknown = true)
public class User {
private int id;
private String name;
private String createDate;
private String updateDate;
private String owner;
private String updateBy;
private String firstName;
private String lastName;
private String emailAddress;
private String password;
private String description;
private int status;
private int isVisible;
private int userSource;
private List userRoleList;
}

定义用户api相关的接口，这是Feign这种声明式http客户端的做法：

package com.example.ranger.api;
import com.example.ranger.model.User;
import feign.Param;
import feign.RequestLine;
/**
* 用户相关api
* https://ranger.apache.org/apidocs/resource_XUserREST.html
*
* @author 01
* @date 2020-11-12
**/
public interface UserFeignClient {
/**
* 创建用户接口
* https://ranger.apache.org/apidocs/resource_XUserREST.html#resource_XUserREST_secureCreateXUser_POST
*
* @param user user
* @return 用户信息
*/
@RequestLine("POST /service/xusers/secure/users")
User createUser(User user);
/**
* 删除用户
* https://ranger.apache.org/apidocs/resource_XUserREST.html#resource_XUserREST_deleteSingleUserByUserId_DELETE
*
* @param id 用户id
* @param forceDelete 是否强制删除
*/
@RequestLine("DELETE /service/xusers/secure/users/id/{id}?forceDelete={forceDelete}")
void deleteUser(@Param("id") Integer id,
@Param("forceDelete") boolean forceDelete);
/**
* 获取用户信息
* https://ranger.apache.org/apidocs/resource_XUserREST.html#resource_XUserREST_getXUserByUserName_GET
*
* @param name 用户名称
* @return 用户信息
*/
@RequestLine("GET /service/xusers/users/userName/{name} ")
User getUserByName(@Param("name") String name);
}

然后我们在此之外再包一层，通常我们会在这一层做一些额外的处理，例如参数校验、结果校验之类的：

package com.example.ranger.api;
import com.example.ranger.exception.RangerClientException;
import com.example.ranger.model.User;
import lombok.AllArgsConstructor;
@AllArgsConstructor
public class UserApi {
private final UserFeignClient userClient;
public User createUser(User user) throws RangerClientException {
return userClient.createUser(user);
}
public void deleteUser(Integer id, boolean forceDelete) {
userClient.deleteUser(id, forceDelete);
}
public User getUserByName(String name) throws RangerClientException {
return userClient.getUserByName(name);
}
}

最后定义一个客户端工具类，提供一个统一的操作入口，以便于外部使用：

package com.example.ranger;
import com.example.ranger.api.PolicyApi;
import com.example.ranger.api.PolicyFeignClient;
import com.example.ranger.api.UserApi;
import com.example.ranger.api.UserFeignClient;
import com.example.ranger.config.RangerClientConfig;
import com.example.ranger.decoder.RangerErrorDecoder;
import com.example.ranger.interceptor.RangerHeadersInterceptor;
import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import feign.Feign;
import feign.Logger;
import feign.Request;
import feign.auth.BasicAuthRequestInterceptor;
import feign.jackson.JacksonDecoder;
import feign.jackson.JacksonEncoder;
import feign.okhttp.OkHttpClient;
import lombok.Getter;
import lombok.extern.slf4j.Slf4j;
import java.util.concurrent.atomic.AtomicBoolean;
@Slf4j
public class RangerClient {
@Getter
private UserApi userApi;
@Getter
private PolicyApi policyApi;
private final RangerClientConfig rangerClientConfig;
public RangerClient(RangerClientConfig rangerClientConfig) {
this.rangerClientCOnfig= rangerClientConfig;
}
private static final ObjectMapper MAPPER = new ObjectMapper()
.setSerializationInclusion(JsonInclude.Include.NON_NULL)
.configure(SerializationFeature.INDENT_OUTPUT, true)
.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
private static final JacksonEncoder ENCODER = new JacksonEncoder(MAPPER);
private static final JacksonDecoder DECODER = new JacksonDecoder(MAPPER);
/**
* 标识client是否已启动
*/
private final AtomicBoolean started = new AtomicBoolean(false);
/**
* 配置client的构建信息
*
* @return {@link Feign.Builder}
*/
private Feign.Builder feignBuilder() {
return Feign.builder()
.logger(new Logger.JavaLogger())
.logLevel(rangerClientConfig.getLevel())
.options(new Request.Options(
rangerClientConfig.getConnectionTimeoutMills(),
rangerClientConfig.getReadTimeoutMills()
)).encoder(ENCODER).decoder(DECODER)
.client(new OkHttpClient())
.errorDecoder(new RangerErrorDecoder())
.requestInterceptor(new RangerHeadersInterceptor())
.requestInterceptor(new BasicAuthRequestInterceptor(
rangerClientConfig.getAuthConfig().getUsername(),
rangerClientConfig.getAuthConfig().getPassword()
));
}
/**
* 启动client
*/
public void start() {
if (started.get()) {
log.info("ranger client is already started");
return;
}
userApi = new UserApi(feignBuilder().target(
UserFeignClient.class, rangerClientConfig.getUrl()
));
policyApi = new PolicyApi(feignBuilder().target(
PolicyFeignClient.class, rangerClientConfig.getUrl()
));
started.set(true);
}
/**
* 停止client
*/
public void stop() {
if (started.get()) {
started.set(false);
} else {
log.info("ranger client is not started");
}
}
}

完成以上的功能代码编写后，我们来写一些单元测试，去验证一下功能是否都正确实现了：

package com.example.ranger.api;
import com.example.ranger.RangerClient;
import com.example.ranger.config.RangerClientConfig;
import com.example.ranger.model.User;
import org.junit.Before;
import org.junit.Test;
import java.util.Collections;
import static org.junit.Assert.assertNotNull;
public class UserApiTest {
private static RangerClient rangerClient;
@Before
public void initRangerClient() {
rangerClient = new RangerClient(new RangerClientConfig());
rangerClient.start();
}
@Test
public void testCreateUser() {
User user = User.builder().name("test")
.firstName("first").lastName("last").password("user@123")
.isVisible(1).status(1).userSource(0)
.userRoleList(Collections.singletonList("ROLE_USER"))
.build();
User result = rangerClient.getUserApi().createUser(user);
assertNotNull(result);
System.out.println(result);
}
@Test
public void testDeleteUser() {
User result = rangerClient.getUserApi().getUserByName("test");
assertNotNull(result);
rangerClient.getUserApi().deleteUser(result.getId(), true);
}
@Test
public void testGetUserByName() {
User result = rangerClient.getUserApi().getUserByName("test");
assertNotNull(result);
System.out.println(result);
}
}

运行testCreateUser这个单元测试，然后到ranger admin上查看是否有新增相应的用户：

然后再运行testDeleteUser这个单元测试，看看该用户是否能被正常删除：

Ranger Api之Policy管理

本小节将介绍使用Policy Api对Ranger上的权限策略进行管理。首先定义接口的请求/响应实体类，由于Policy稍微复杂点，需要定义的类也比较多：

/**
* 策略所作用的资源，即hdfs目录、hive的库/表/列等
* https://ranger.apache.org/apidocs/json_RangerPolicyResource.html
*
* @author 01
* @date 2020-11-12
**/
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@JsonIgnoreProperties(ignoreUnknown = true)
public class PolicyResource {
private List values = Lists.newArrayList();
private Boolean isExcludes;
private Boolean isRecursive;
}
/**
* https://ranger.apache.org/apidocs/json_RangerPolicyItemCondition.html
*
* @author 01
* @date 2020-11-12
**/
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@JsonIgnoreProperties(ignoreUnknown = true)
public class PolicyItemCondition {
private String type;
private List value = Lists.newArrayList();
}
/**
* 策略条件项中的权限信息，即在该项中拥有哪些权限，对应“Permissions”
* https://ranger.apache.org/apidocs/json_RangerPolicyItemAccess.html
*
* @author 01
* @date 2020-11-12
**/
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@JsonIgnoreProperties(ignoreUnknown = true)
public class PolicyItemAccess {
private String type;
private Boolean isAllowed;
}
/**
* 策略中的条件项，对应“Allow Conditions”或“Deny Conditions”中的每一栏信息
* https://ranger.apache.org/apidocs/json_RangerPolicyItem.html
*
* @author 01
* @date 2020-11-12
**/
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@JsonIgnoreProperties(ignoreUnknown = true)
public class PolicyItem {
private List

accesses = Lists.newArrayList();
private Set users = Sets.newHashSet();
private List groups = Lists.newArrayList();
private List

cOnditions= Lists.newArrayList();
private Boolean delegateAdmin;
}
/**
* 策略实体
* https://ranger.apache.org/apidocs/json_RangerPolicy.html
*
* @author 01
* @date 2020-11-12
**/
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@JsonIgnoreProperties(ignoreUnknown = true)
public class Policy {
private Map resources;
private List

policyItems = Lists.newArrayList();
private List

denyPolicyItems = Lists.newArrayList();
private List

allowExceptiOns= Lists.newArrayList();
private List

denyExceptiOns= Lists.newArrayList();
private List dataMaskPolicyItems = Lists.newArrayList();
private List rowFilterPolicyItems = Lists.newArrayList();
private int id;
private String guid;
private boolean isEnabled;
private int version;
private String service;
private String name;
private int policyType;
private String description;
private boolean isAuditEnabled;
}

定义权限策略相关api的接口：

package com.example.ranger.api;
import com.example.ranger.model.Policy;
import feign.Param;
import feign.RequestLine;
import java.util.List;
/**
* 权限策略相关api
* https://ranger.apache.org/apidocs/resource_PublicAPIsv2.html
* https://ranger.apache.org/apidocs/resource_ServiceREST.html
*
* @author 01
* @date 2020-11-12
**/
public interface PolicyFeignClient {
/**
* 创建策略
* https://ranger.apache.org/apidocs/resource_PublicAPIsv2.html#resource_PublicAPIsv2_createPolicy_POST
*
* @param policy 策略信息
* @return 策略信息
*/
@RequestLine("POST /service/public/v2/api/policy")
Policy createPolicy(Policy policy);
/**
* 删除策略
* https://ranger.apache.org/apidocs/resource_ServiceREST.html#resource_ServiceREST_deletePolicy_DELETE
*
* @param id 策略id
*/
@RequestLine("DELETE /service/plugins/policies/{id}")
void deletePolicy(@Param("id") Integer id);
/**
* 通过服务和策略名称获取策略信息
* https://ranger.apache.org/apidocs/resource_PublicAPIsv2.html#resource_PublicAPIsv2_getPolicyByName_GET
*
* @param serviceName 服务名称
* @param policyName 策略名称
* @return 策略信息
*/
@RequestLine("GET /service/public/v2/api/service/{serviceName}/policy/{policyName}")
Policy getPolicyByName(@Param("serviceName") String serviceName,
@Param("policyName") String policyName);
/**
* 获取指定服务下的策略信息列表
*
* @param serviceName 服务名称
* @return 该服务下的策略信息列表
*/
@RequestLine("GET /service/public/v2/api/service/{serviceName}/policy")
List

getAllPoliciesByService(@Param("serviceName") String serviceName);
}

同样，在接口之上再包一层：

package com.example.ranger.api;
import com.example.ranger.model.Policy;
import lombok.AllArgsConstructor;
import java.util.List;
@AllArgsConstructor
public class PolicyApi {
private final PolicyFeignClient policyFeignClient;
public Policy getPolicyByName(String serviceName, String policyName) {
return policyFeignClient.getPolicyByName(serviceName, policyName);
}
public List

getAllPoliciesByService(String serviceName) {
return policyFeignClient.getAllPoliciesByService(serviceName);
}
public Policy createPolicy(Policy policy) {
return policyFeignClient.createPolicy(policy);
}
public void deletePolicy(Integer id) {
policyFeignClient.deletePolicy(id);
}
}

修改RangerClient，增加PolicyApi相关代码：

@Slf4j
public class RangerClient {
@Getter
private PolicyApi policyApi;
...
/**
* 启动client
*/
public void start() {
if (started.get()) {
log.info("ranger client is already started");
return;
}
userApi = new UserApi(feignBuilder().target(
UserFeignClient.class, rangerClientConfig.getUrl()
));
policyApi = new PolicyApi(feignBuilder().target(
PolicyFeignClient.class, rangerClientConfig.getUrl()
));
started.set(true);
}
...
}

编写单元测试：

package com.example.ranger.api;
import com.example.ranger.RangerClient;
import com.example.ranger.config.RangerClientConfig;
import com.example.ranger.model.Policy;
import com.example.ranger.model.PolicyItem;
import com.example.ranger.model.PolicyItemAccess;
import com.example.ranger.model.PolicyResource;
import org.junit.Before;
import org.junit.Test;
import java.util.*;
import static org.junit.Assert.assertNotNull;
public class PolicyApiTest {
private static RangerClient rangerClient;
@Before
public void initRangerClient() {
rangerClient = new RangerClient(new RangerClientConfig());
rangerClient.start();
}
@Test
public void testCreatePolicy() {
PolicyResource policyResource = PolicyResource.builder()
.values(Collections.singletonList("/testdir2"))
.isRecursive(true)
.build();
Map policyResourceMap = new HashMap<>();
policyResourceMap.put("path", policyResource);
Set users = new HashSet<>();
users.add("hive");
List

policyItemAccessList = new ArrayList<>();
policyItemAccessList.add(PolicyItemAccess.builder().type("read").build());
policyItemAccessList.add(PolicyItemAccess.builder().type("write").build());
policyItemAccessList.add(PolicyItemAccess.builder().type("execute").build());
PolicyItem policyItem = PolicyItem.builder()
.delegateAdmin(true).users(users)
.accesses(policyItemAccessList)
.build();
Policy policy = Policy.builder()
.service("dev_hdfs")
.name("test_ranger_api")
.isEnabled(true).policyType(0)
.resources(policyResourceMap)
.policyItems(Collections.singletonList(policyItem))
.build();
Policy result = rangerClient.getPolicyApi().createPolicy(policy);
assertNotNull(result);
System.out.println(result.getName());
}
@Test
public void testGetPolicyByName() {
Policy result = rangerClient.getPolicyApi()
.getPolicyByName("dev_hdfs", "test_ranger_api");
assertNotNull(result);
System.out.println(result.getName());
}
@Test
public void testGetAllPoliciesByService() {
List

result = rangerClient.getPolicyApi()
.getAllPoliciesByService("dev_hdfs");
assertNotNull(result);
System.out.println(result.size());
}
@Test
public void testDeletePolicy() {
Policy result = rangerClient.getPolicyApi()
.getPolicyByName("dev_hdfs", "test_ranger_api");
assertNotNull(result);
rangerClient.getPolicyApi().deletePolicy(result.getId());
System.out.println(result.getName());
}
}

执行testCreatePolicy单元测试，到ranger admin上验证是否创建了相应的策略：

查看策略内容是否与代码中定义的一致：

本文的代码仓库：

• https://gitee.com/Zero-One/ranger-client-demo