热门标签 | HotTags
当前位置:  开发笔记 > 编程语言 > 正文

HowtoConfigureRocksDBLoggingforAdvancedTroubleshooting

QuestionRocksDB'sLOGfilecomesinhandywhentroubleshootingFlinkwithRocksDB.HowcanIconfig

Question

RocksDB's LOG file comes in handy when troubleshooting Flink with RocksDB. How can I configure RocksDB logging?

 


Answer

Note: This section applies to Flink 1.10 - 1.14

By default, Flink uses the log level HEADER_LEVEL for RocksDB. This essentially disabled RocksDB logging and leave only RocksDB configuration printed in the RocksDB log file. The main reason is that the RocksDB log file is not controllable in size prior to Flink 1.14. The way to configure RocksDB logging depends on the version of Flink you are using. Flink 1.13 or later supports changing RocksDB log level via configuration. Flink 1.14 additionally supports specifying the logging directory which can be located on a (separate) volume that is retained after container shutdown for debugging purposes. Thanks to the new RocksDB version, you can also configure log rotation in Flink 1.14 or later. For other scenarios, you will need to define your own custom Options Factory.


Via Configuration in Flink 1.14 or later

For example:

state.backend.rocksdb.log.level: INFO_LEVEL
state.backend.rocksdb.log.max-file-size: 10MB
state.backend.rocksdb.log.file-num: 10
state.backend.rocksdb.log.dir: /rocksdb/logs

Refer to Flink's documentation for more details of these configuration options.


Via Configuration in Flink 1.13

For example:

state.backend.rocksdb.log.level: INFO_LEVEL

Custom Options Factory

If you are using older versions of Flink (<1.13), you want to configure logging directory in Flink 1.13, or you want to dump RocksDB statistics in RocksDB's LOG file, you can create a custom Options Factory by extending Flink's DefaultConfigurableOptionsFactory. This mechanism gives you options to configure RocksDB logging while still allows your jobs to continue using any other RocksDB tuning options the way you used them before.

Important: DefaultConfigurableOptionsFactory was not really meant for being extended and may change among releases. If you plan to take this into production, you should write your own complete ConfigurableRocksDBOptionsFactory and set all the options you need in there.


Extending DefaultConfigurableOptionsFactory

First, if you have not done so yet, add a dependency to Flink's RocksDB state backend. For example, add this to your Maven project's pom.xml:


org.apache.flink
flink-statebackend-rocksdb_${scala.binary.version}
${flink.version}
provided

Then create your own Options Factory:

package com.ververica.troubleshooting;

import org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory;
import org.rocksdb.DBOptions;
import org.rocksdb.InfoLogLevel;
public class MyCustomRocksDBOptionsFactory extends DefaultConfigurableOptionsFactory {
private static final long serialVersiOnUID= 1L;
private String dbLogDir = "";
@Override
public DBOptions createDBOptions(DBOptions currentOptions,
Collection handlesToClose) {
currentOptiOns= super.createDBOptions(currentOptions, handlesToClose);
currentOptions.setInfoLogLevel(InfoLogLevel.INFO_LEVEL);
currentOptions.setStatsDumpPeriodSec(60);
currentOptions.setDbLogDir(dbLogDir);
return currentOptions;
}
@Override
public String toString() {
return this.getClass().toString() + "{" + super.toString() + '}';
}
/**
* Set directory where RocksDB writes its info LOG file (empty = data dir, otherwise the
* data directory's absolute path will be used as the log file prefix).
*/
public void setDbLogDir(String dbLogDir) {
this.dbLogDir = dbLogDir;
}
}

Three points in createDBOptions are important here:



  • setInfoLogLevel(InfoLogLevel.INFO_LEVEL) sets the logging level to INFO from which you would get a decent amount of logging data (increase if needed)

  • setStatsDumpPeriodSec(60) dumps various RocksDB statistics every this many seconds: this includes compaction statistics.

  • setDbLogDir(dbLogDir) specifies the path where to put the LOG file: depending on what you are trying to troubleshoot, you can just use a local directory or you may need to put this onto a distributed file system (or persistent volume) to survive node/pod/job restarts


Configuring Flink

With the custom Options Factory, you can configure Flink either programmatically or through its flink-conf.yaml.

(1) Programmatically

Note: The state backend interface is changed since Flink 1.13; we provide both versions below.

// Flink 1.13 or later
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
EmbeddedRocksDBStateBackend stateBackend = new EmbeddedRocksDBStateBackend()
env.setStateBackend(stateBackend);
env.getCheckpointConfig().setCheckpointStorage("file:///path/to/checkpoints");

MyCustomRocksDBOptionsFactory optiOns= new MyCustomRocksDBOptionsFactory();
options.setDbLogDir("/path/to/rocksdb/logging/");
stateBackend.setRocksDBOptions(options);

// Flink <1.13
RocksDBStateBackend stateBackend = new RocksDBStateBackend("file:///path/to/checkpoints");
MyCustomRocksDBOptionsFactory optiOns= new MyCustomRocksDBOptionsFactory();
options.setDbLogDir("/path/to/rocksdb/logging/");
stateBackend.setRocksDBOptions(options);

(2) Via Flink configuration (flink-conf.yaml)

If you want to configure your options factory completely via flink-conf.yaml, we may extend the code above to update its settings from the configuration. The code below shows how to do this for the log directory but you can extend it to make further settings configurable.

Note: The interface slightly changed since Flink 1.11; we provide both versions below.

// Flink 1.11 or later
public static final ConfigOption LOG_DIR =
key("state.backend.rocksdb.log.dir")
.stringType()
.noDefaultValue()
.withDescription("Location of RocksDB's info LOG file (empty = data dir, otherwise the " +
"data directory's absolute path will be used as the log file prefix)");
@Override
public DefaultConfigurableOptionsFactory configure(ReadableConfig configuration) {
DefaultConfigurableOptionsFactory optiOnsFactory=
super.configure(configuration);
this.dbLogDir = configuration.getOptional(LOG_DIR).orElse(this.dbLogDir);
return optionsFactory;
}

// Flink 1.10
public static final ConfigOption LOG_DIR =
key("state.backend.rocksdb.log.dir")
.stringType()
.noDefaultValue()
.withDescription("Location of RocksDB's info LOG file (empty = data dir, otherwise the " +
"data directory's absolute path will be used as the log file prefix)");
@Override
public DefaultConfigurableOptionsFactory configure(Configuration configuration) {
DefaultConfigurableOptionsFactory optiOnsFactory=
super.configure(configuration);
this.dbLogDir = configuration.getOptional(LOG_DIR).orElse(this.dbLogDir);
return optionsFactory;
}

With the code additions from above, you can simply adapt your flink-conf.yaml and configure it like this:

state.backend.rocksdb.log.dir: /path/to/rocksdb/logging/
state.backend.rocksdb.options-factory: com.ververica.troubleshooting.MyCustomRocksDBOptionsFactory

Note: Configuring the options factory via flink-conf.yaml will apply the options factory to all jobs started in the Flink cluster. Make sure that this class is available cluster-wide or in all jobs started on this cluster!
(For Ververica Platform deployments, this will not be a problem since each deployment spawns its own Flink cluster.)


Related Information

https://ververica.zendesk.com/hc/en-us/articles/360015933320-How-to-Configure-RocksDB-Logging-for-Advanced-Troubleshooting



推荐阅读
  • Maven + Spring + MyBatis + MySQL 环境搭建与实例解析
    本文详细介绍如何使用MySQL数据库进行环境搭建,包括创建数据库表并插入示例数据。随后,逐步指导如何配置Maven项目,整合Spring框架与MyBatis,实现高效的数据访问。 ... [详细]
  • 【MySQL】frm文件解析
    官网说明:http:dev.mysql.comdocinternalsenfrm-file-format.htmlfrm是MySQL表结构定义文件,通常frm文件是不会损坏的,但是如果 ... [详细]
  • spring boot使用jetty无法启动 ... [详细]
  • 本文详细介绍了如何使用C#实现不同类型的系统服务账户(如Windows服务、计划任务和IIS应用池)的密码重置方法。 ... [详细]
  • Spring Security基础配置详解
    本文详细介绍了Spring Security的基础配置方法,包括如何搭建Maven多模块工程以及具体的安全配置步骤,帮助开发者更好地理解和应用这一强大的安全框架。 ... [详细]
  • 本文详细介绍了如何在 Ubuntu 14.04 系统上搭建仅使用 CPU 的 Caffe 深度学习框架,包括环境准备、依赖安装及编译过程。 ... [详细]
  • Docker安全策略与管理
    本文探讨了Docker的安全挑战、核心安全特性及其管理策略,旨在帮助读者深入理解Docker安全机制,并提供实用的安全管理建议。 ... [详细]
  • 本文详细介绍了如何在Oracle VM VirtualBox中实现主机与虚拟机之间的数据交换,包括安装Guest Additions增强功能,以及如何利用这些功能进行文件传输、屏幕调整等操作。 ... [详细]
  • Beetl是一款先进的Java模板引擎,以其丰富的功能、直观的语法、卓越的性能和易于维护的特点著称。它不仅适用于高响应需求的大型网站,也适合功能复杂的CMS管理系统,提供了一种全新的模板开发体验。 ... [详细]
  • 一、Advice执行顺序二、Advice在同一个Aspect中三、Advice在不同的Aspect中一、Advice执行顺序如果多个Advice和同一个JointPoint连接& ... [详细]
  • ArcBlock 发布 ABT 节点 1.0.31 版本更新
    2020年11月9日,ArcBlock 区块链基础平台发布了 ABT 节点开发平台的1.0.31版本更新,此次更新带来了多项功能增强与性能优化。 ... [详细]
  • 本文详细介绍如何安装和配置DedeCMS的移动端站点,包括新版本安装、老版本升级、模板适配以及必要的代码修改,以确保移动站点的正常运行。 ... [详细]
  • 尽管在WPF中工作了一段时间,但在菜单控件的样式设置上遇到了一些基础问题,特别是关于如何正确配置前景色和背景色。 ... [详细]
  • 本文详细介绍如何在 Apache 中设置虚拟主机,包括基本配置和高级设置,帮助用户更好地理解和使用虚拟主机功能。 ... [详细]
  • 本文详细介绍了在 CentOS 系统中如何创建和管理 SWAP 分区,包括临时创建交换文件、永久性增加交换空间的方法,以及如何手动释放内存缓存。 ... [详细]
author-avatar
DLDLBABY1_182
这个家伙很懒,什么也没留下!
PHP1.CN | 中国最专业的PHP中文社区 | DevBox开发工具箱 | json解析格式化 |PHP资讯 | PHP教程 | 数据库技术 | 服务器技术 | 前端开发技术 | PHP框架 | 开发工具 | 在线工具
Copyright © 1998 - 2020 PHP1.CN. All Rights Reserved | 京公网安备 11010802041100号 | 京ICP备19059560号-4 | PHP1.CN 第一PHP社区 版权所有