在上一篇博客中，我们准备好了数据。现在数据已经以我们需要的格式，存放在Elasticsearch中了。本文讲述如何在Elasticsearch中进行空间GEO查询和聚合查询，以及如何准备ajax
在上一篇博客中，我们准备好了数据。现在数据已经以我们需要的格式，存放在Elasticsearch中了。
本文讲述如何在Elasticsearch中进行空间GEO查询和聚合查询，以及如何准备ajax接口。
平台的服务端部分使用的springboot+mybatis的基本开发模式。工程结构如下。
可以看到本工程有三个module：
1）moonlight-web是controller和service层的实现；
2）moonlight-dsl封装了ES空间索引查询和聚合查询的方法；
3）moonlight-dao封装了持久化地理围栏的方法。
我们以客户端请求的处理顺序为例进行讲解。
1、controller
在controller层中，我们实现了4个接口，分别是circle、box、polygon、heatmap，也就是圆形圈选，矩形圈选，多边形圈选和热力图。
先看一下代码的具体实现。
@RestController
@RequestMapping("/moonlight")
public class MoonlightController {
protected final Logger logger = LoggerFactory.getLogger(this.getClass());
@Autowired
private MoonlightService moonlightService;
@RequestMapping(value = "/circle", method = RequestMethod.GET)
public ResponseEntity circle(HttpServletRequest request, HttpServletResponse response) {
String point = request.getParameter("point");
String radius = request.getParameter("radius");
try {
Map result = moonlightService.circle(point, radius);
logger.info("circle圈选成功, points={}, radius={}, result={}", point, radius, result);
return new ResponseEntity<>(
new Response(ResultCode.SUCCESS, "circle圈选成功", result),
HttpStatus.OK);
} catch (Exception e) {
logger.error("circle圈选失败, points={}, radius={}, result={}", point, radius, null, e);
return new ResponseEntity<>(
new Response(ResultCode.EXCEPTION, "circle圈选失败", null),
HttpStatus.INTERNAL_SERVER_ERROR);
}
}
@RequestMapping(value = "/box", method = RequestMethod.GET)
public ResponseEntity box(HttpServletRequest request, HttpServletResponse response) {
String point1 = request.getParameter("point1");
String point2 = request.getParameter("point2");
String point3 = request.getParameter("point3");
String point4 = request.getParameter("point4");
try {
Map result = moonlightService.boundingBox(point1, point2, point3, point4);
logger.info("box圈选成功, point1={}, point2={}, point3={}, point4={}, result={}", point1, point2, point3, point4, result);
return new ResponseEntity<>(
new Response(ResultCode.SUCCESS, "box圈选成功", result),
HttpStatus.OK);
} catch (Exception e) {
logger.error("box圈选失败, point1={}, point2={}, point3={}, point4={}, result={}", point1, point2, point3, point4, null, e);
return new ResponseEntity<>(
new Response(ResultCode.EXCEPTION, "box圈选失败", null),
HttpStatus.INTERNAL_SERVER_ERROR);
}
}
@RequestMapping(value = "/polygon", method = RequestMethod.GET)
public ResponseEntity polygon(HttpServletRequest request, HttpServletResponse response) {
List points = new ArrayList<>();
Enumeration paramNames = request.getParameterNames();
while (paramNames.hasMoreElements()) {
String paramName = paramNames.nextElement();
if (paramName.startsWith("point")) {
points.add(request.getParameter(paramName));
}
}
try {
Map result = moonlightService.polygon(points);
logger.info("polygon圈选成功, points={}, result={}", points, result);
return new ResponseEntity<>(
new Response(ResultCode.SUCCESS, "polygon圈选成功", result),
HttpStatus.OK);
} catch (Exception e) {
logger.error("polygon圈选失败, points={}, result={}", points, null, e);
return new ResponseEntity<>(
new Response(ResultCode.EXCEPTION, "polygon圈选失败", null),
HttpStatus.INTERNAL_SERVER_ERROR);
}
}
@RequestMapping(value = "/heatMap", method = RequestMethod.GET)
public ResponseEntity heatMap(HttpServletRequest request, HttpServletResponse response) {
try {
List> result = moonlightService.heatMap();
logger.info("heatMap请求成功, result={}", result);
return new ResponseEntity<>(
new Response(ResultCode.SUCCESS, "heatMap请求成功", result),
HttpStatus.OK);
} catch (Exception e) {
logger.error("heatMap请求失败, result={}", null, e);
return new ResponseEntity<>(
new Response(ResultCode.EXCEPTION, "heatMap请求失败", null),
HttpStatus.INTERNAL_SERVER_ERROR);
}
}
}
我们以圆形圈选（circle接口）为例，circle接口传入两个参数，一个是point，也就是中心点坐标，一个是radius，也就是半径，它干的事情就是圈选出，point点周围radius长度内的所有订单数据，具体实现是调用了service层的方法，controller得到圈选的数据后就返回了。
下面我们来看一下service层。
2、service
service层是具体业务的实现。我们这里的service仍然比较简单，可以看到只是初始化了esDao的句柄，然后进行es的geo查询。
先看一下具体代码。
@Service
public class MoonlightService {
protected final Logger logger = LoggerFactory.getLogger(this.getClass());
@Autowired
private ESDao esDao;
public Map circle(String point, String radius) {
POI center = new POI(point);
return esDao.circle(center, Double.parseDouble(radius));
}
public Map boundingBox(String point1, String point2, String point3, String point4) {
POI poi1 = new POI(point1);
POI poi2 = new POI(point2);
POI poi3 = new POI(point3);
POI poi4 = new POI(point4);
POI topLeft = getTopLeft(poi1, poi2, poi3, poi4);
POI bottomRight = getBottomRight(poi1, poi2, poi3, poi4);
logger.info("topLeft - lat={}, lng={}, bottomRight - lat={}, lng={}",
topLeft.getLat(), topLeft.getLng(), bottomRight.getLat(), bottomRight.getLng());
return esDao.boundingBox(topLeft, bottomRight);
}
public Map polygon(List points) {
List poiList = new ArrayList<>();
for (String point : points) {
POI poi = new POI(point);
poiList.add(poi);
}
return esDao.polygon(poiList);
}
public List> heatMap() {
return esDao.heatMap();
}
private POI getTopLeft(POI poi1, POI poi2, POI poi3, POI poi4) {
POI topLeft = new POI();
List latList = new ArrayList<>();
List lngList = new ArrayList<>();
latList.add(poi1.getLat());
latList.add(poi2.getLat());
latList.add(poi3.getLat());
latList.add(poi4.getLat());
Collections.sort(latList);
Double minLat = latList.get(0);
Double maxLat = latList.get(3);
lngList.add(poi1.getLng());
lngList.add(poi2.getLng());
lngList.add(poi3.getLng());
lngList.add(poi4.getLng());
Collections.sort(lngList);
Double minLng = lngList.get(0);
Double maxLng = lngList.get(3);
topLeft.setLat(maxLat);
topLeft.setLng(minLng);
return topLeft;
}
private POI getBottomRight(POI poi1, POI poi2, POI poi3, POI poi4) {
POI bottomRight = new POI();
List latList = new ArrayList<>();
List lngList = new ArrayList<>();
latList.add(poi1.getLat());
latList.add(poi2.getLat());
latList.add(poi3.getLat());
latList.add(poi4.getLat());
Collections.sort(latList);
Double minLat = latList.get(0);
Double maxLat = latList.get(3);
lngList.add(poi1.getLng());
lngList.add(poi2.getLng());
lngList.add(poi3.getLng());
lngList.add(poi4.getLng());
Collections.sort(lngList);
Double minLng = lngList.get(0);
Double maxLng = lngList.get(3);
bottomRight.setLat(minLat);
bottomRight.setLng(maxLng);
return bottomRight;
}
}
我们仍然是以圆形圈选为例，可以看到，service代码的逻辑就是，创建出圈选需要的数据接口，然后调用Dao层进行查询就是了。
circle圈选需要的是一个中心点POI类型，和一个Double半径。
box矩形查询需要的是左上坐标点和右下坐标点，里面有两个函数getTopLeft、getBottomRight分别可以求出矩形的左上点和右下点。
polygon多边形查询需要的是一系列点，这些点顺序的连接所绘制出来的图形就是目标多边形。
heatmap热力图什么参数也不要，将返回一定精度的经纬度计数值，后面我们会详述。
之后所有的service都调用了Dao层的es查询逻辑。所以最重要的一部分是esDao的实现，下面我们就来看一看。
3、Dao
Dao层代码是整个项目的核心，包括对Elasticsearch数据进行圈选和聚合两部分，此外就是热力图数据的准备。
先看一下代码。
@Component
public class ESDao {
protected final Logger logger = LoggerFactory.getLogger(this.getClass());
@Autowired
private ESClient esClient;
public Map circle(POI center, Double radius) {
TermsQueryBuilder termsQuery = termsQuery("product_id", new double[]{3, 4});
GeoDistanceRangeQueryBuilder geoDistanceRangeQuery = QueryBuilders.geoDistanceRangeQuery("location")
.point(center.getLat(), center.getLng())
.from("0m")
.to(String.format("%fm", radius))
.includeLower(true)
.includeUpper(true)
.optimizeBbox("memory")
.geoDistance(GeoDistance.SLOPPY_ARC);
QueryBuilder queryBuilder = QueryBuilders.boolQuery().must(termsQuery).must(geoDistanceRangeQuery);
SearchRequestBuilder search = esClient.getClient().prepareSearch("moon").setTypes("bj")
.setSearchType(SearchType.DFS_QUERY_AND_FETCH)
.setQuery(queryBuilder);
return agg(search);
}
public Map boundingBox(POI topLeft, POI bottomRight) {
TermsQueryBuilder termsQuery = termsQuery("product_id", new double[]{3, 4});
GeoBoundingBoxQueryBuilder geoBoundingBoxQuery = QueryBuilders.geoBoundingBoxQuery("location")
.topLeft(topLeft.getLat(), topLeft.getLng())
.bottomRight(bottomRight.getLat(), bottomRight.getLng());
QueryBuilder queryBuilder = QueryBuilders.boolQuery().must(termsQuery).must(geoBoundingBoxQuery);
SearchRequestBuilder search = esClient.getClient().prepareSearch("moon").setTypes("bj")
.setSearchType(SearchType.DFS_QUERY_AND_FETCH)
.setQuery(queryBuilder);
return agg(search);
}
public Map polygon(List poiList) {
TermsQueryBuilder termsQuery = termsQuery("product_id", new double[]{3, 4});
GeoPolygonQueryBuilder geoPolygonQuery = QueryBuilders.geoPolygonQuery("location");
for (POI poi : poiList) {
geoPolygonQuery.addPoint(poi.getLat(), poi.getLng());
}
QueryBuilder queryBuilder = QueryBuilders.boolQuery().must(termsQuery).must(geoPolygonQuery);
SearchRequestBuilder search = esClient.getClient().prepareSearch("moon").setTypes("bj")
.setSearchType(SearchType.DFS_QUERY_AND_FETCH)
.setQuery(queryBuilder);
return agg(search);
}
public List> heatMap() {
TermQueryBuilder queryBuilder = termQuery("date", "2017-11-24");
SearchRequestBuilder searchRequestBuilder = esClient.getClient()
.prepareSearch("moon").setTypes("bj");
SearchResponse response = searchRequestBuilder
.setQuery(queryBuilder)
.setFrom(0).setSize(10000)
.setExplain(true).execute().actionGet();
SearchHits hits = response.getHits();
Map countMap = new HashMap<>();
for (SearchHit hit : hits) {
Map source = hit.getSource();
Map locatiOnMap= (Map) source.get("location");
DecimalFormat df = new DecimalFormat("#.000");
String lat = df.format(locationMap.get("lat"));
String lon = df.format(locationMap.get("lon"));
String key = lat+"-"+lon;
if (countMap.containsKey(key)) {
countMap.put(key, countMap.get(key) + 1);
} else {
countMap.put(key, 1);
}
}
List> result = new ArrayList<>();
for (Map.Entry entry : countMap.entrySet()) {
String lat = entry.getKey().split("-")[0];
String lon = entry.getKey().split("-")[1];
Integer count = entry.getValue();
Map map = new HashMap<>();
map.put("lat", Double.parseDouble(lat));
map.put("lng", Double.parseDouble(lon));
map.put("count", count);
result.add(map);
}
return result;
}
private Map agg(SearchRequestBuilder search) {
Map resultMap = new HashMap<>();
GroupBy groupBy = new GroupBy(search, "date_group", "date", true);
groupBy.addSumAgg("pre_total_fee_sum", "pre_total_fee");
groupBy.addCountAgg("order_id_count", "order_id");
groupBy.addSumAgg("cancel_count", "type");
List xAxis = new ArrayList<>();
List profits = new ArrayList<>();
List totals = new ArrayList<>();
List cancelRatios = new ArrayList<>();
List> details = new ArrayList<>();
Map groupbyRespOnse= groupBy.getGroupbyResponse();
for (Map.Entry entry : groupbyResponse.entrySet()) {
String date = entry.getKey();
xAxis.add(date);
Map subAggMap = (Map) entry.getValue();
String profit = subAggMap.get("pre_total_fee_sum");
profits.add(profit);
String total = subAggMap.get("order_id_count");
totals.add(total);
String cancelRatioDouble = new DecimalFormat("#.0000").format(
Double.parseDouble(subAggMap.get("cancel_count")) / Double.parseDouble(subAggMap.get("order_id_count"))
);
String cancelRatio = new DecimalFormat("0.00%").format(
Double.parseDouble(subAggMap.get("cancel_count")) / Double.parseDouble(subAggMap.get("order_id_count"))
);
cancelRatios.add(cancelRatioDouble);
Map tempMap = new HashMap<>();
tempMap.put("profit", profit);
tempMap.put("total", total);
tempMap.put("cancelRatio", cancelRatio);
tempMap.put("date", date);
details.add(tempMap);
}
resultMap.put("xAxis", xAxis);
resultMap.put("profit", profits);
resultMap.put("total", totals);
resultMap.put("cancelRatio", cancelRatios);
resultMap.put("detail", details);
return resultMap;
}
}
es圈选部分
circle为例，我们构造了一个geoDistanceRangeQuery查询，这个查询到上一篇博客准备好的moon索引，bj type中去将数据圈选出来。
类似的我们有矩形geoBoundingBoxQuery查询，多边形geoPolygonQuery查询，具体构造查询的方式可以参照代码，这个代码还是很简单的，熟悉es的同学很快可以上手并且实现这样的查询，不熟悉的话可以自行百度一下。如果还有其他的查询条件，可以通过QueryBuilders.boolQuery().must(termsQuery).must(geoDistanceRangeQuery)加入，例如我这里在圈选之外加入了一个terms查询，这个查询相当于sql中的where product_id in (3,4) and ...。
es聚合部分
es聚合部分做的事情是，对查询出的订单进行了聚合运算，例如求和和计数，是两个最常见的运算，这部分在这里不详细叙述了，请参见这篇博客。
热力图
这里要额外说明的是，热力图heatmap，和圈选不一样，他是查询了最近一天type=bj分区里的所有数据，按照坐标进行了计数，可以看到的是，计数的时候，我们指定了精度，这里是小数点后三位有效数字
DecimalFormat df = new DecimalFormat("#.000");
String lat = df.format(locationMap.get("lat"));
String lon = df.format(locationMap.get("lon"));
String key = lat+"-"+lon;
然后将计数结果返回。百度地图SDK会将计数结果绘制成热力图，这个不用我们管，我会在另一篇博客中讲述这个过程。
到这里，整个工程的基本功能就介绍完了。
京公网安备 11010802041100号