大数据技术Flink电商实时数仓DWD数据层准备第2章功能1:环境搭建

2.1 新建工程 gmall2021-realtime

2.2 创建如下结构

这里采用了java 与scala 语言混编，java主要写一个实体类与工具，scala 主要用于写flink 任务，也可以全部采用java写，写scala 的主要原因是防止scala生疏。

2.3 修改配置文件

2.3.1 在pom.xml 添加如下配置


1.8
${java.version}
${java.version}
1.12.0
2.12
2.12
3.1.3



org.scala-lang
scala-library
2.12.10


org.scala-lang
scala-compiler
2.12.10


org.apache.flink
flink-streaming-scala_${scala.version}
${flink.version}


org.apache.flink
flink-connector-kafka_${scala.version}
${flink.version}


org.apache.flink
flink-clients_${scala.version}
${flink.version}


org.apache.flink
flink-cep_${scala.version}
${flink.version}


org.apache.flink
flink-json
${flink.version}


com.alibaba
fastjson
1.2.68



org.apache.hadoop
hadoop-client
${hadoop.version}



org.slf4j
slf4j-api
1.7.25


org.slf4j
slf4j-log4j12
1.7.25


org.apache.logging.log4j
log4j-to-slf4j
2.14.0



org.projectlombok
lombok
1.18.12
provided



commons-beanutils
commons-beanutils
1.9.3



com.google.guava
guava
29.0-jre


mysql
mysql-connector-java
5.1.47





org.apache.maven.plugins
maven-assembly-plugin
3.0.0


jar-with-dependencies




make-assembly
package

single







2.3.2 在resources 目录下创建log4.properties 配置文件

log4j.rootLogger=warn,stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.COnversionPattern=%d %p [%c] - %m%n