Spark应用程序抛出javax.servlet.FilterRegistration

我正在使用Scala在本地创建和运行Spark应用程序.

我的build.sbt:

name : "SparkDemo"
version : "1.0"
scalaVersion : "2.10.4"
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.2.0"    exclude("org.apache.hadoop", "hadoop-client")
libraryDependencies += "org.apache.spark" % "spark-sql_2.10" % "1.2.0"
libraryDependencies += "org.apache.hadoop" % "hadoop-common" % "2.6.0"  excludeAll(
ExclusionRule(organization = "org.eclipse.jetty"))
libraryDependencies += "org.apache.hadoop" % "hadoop-mapreduce-client-core" % "2.6.0"
libraryDependencies += "org.apache.hbase" % "hbase-client" % "0.98.4-hadoop2"
libraryDependencies += "org.apache.hbase" % "hbase-server" % "0.98.4-hadoop2"
libraryDependencies += "org.apache.hbase" % "hbase-common" % "0.98.4-hadoop2"
mainClass in Compile := Some("demo.TruckEvents")

在运行时我得到异常:

调用期间线程"main"java.lang.ExceptionInInitializerError中的异常...引起:java.lang.SecurityException:类"javax.servlet.FilterRegistration"的签名者信息与同一包中其他类的签名者信息不匹配

此处触发异常:

val sc = new SparkContext("local", "HBaseTest")

我正在使用IntelliJ Scala/SBT插件.

我已经看到其他人也有这个问题的建议解决方案.但这是一个maven构建......我的错在哪里？或者任何其他建议我如何解决这个问题？

请在此处查看我对类似问题的回答.类冲突是因为HBase依赖org.mortbay.jetty而且Spark依赖于org.eclipse.jetty.我能够通过org.mortbay.jetty从HBase中排除依赖关系来解决问题.

如果你在拉hadoop-common,那么你可能还需要排除javax.servlet的hadoop-common.我有一个工作的HBase/Spark设置,我的sbt依赖项设置如下:

val clouderaVersion = "cdh5.2.0"
val hadoopVersion = s"2.5.0-$clouderaVersion"
val hbaseVersion = s"0.98.6-$clouderaVersion"
val sparkVersion = s"1.1.0-$clouderaVersion"

val hadoopCommon = "org.apache.hadoop" % "hadoop-common" % hadoopVersion % "provided" excludeAll ExclusionRule(organization = "javax.servlet")
val hbaseCommon = "org.apache.hbase" % "hbase-common" % hbaseVersion % "provided"
val hbaseClient = "org.apache.hbase" % "hbase-client" % hbaseVersion % "provided"
val hbaseProtocol = "org.apache.hbase" % "hbase-protocol" % hbaseVersion % "provided"
val hbaseHadoop2Compat = "org.apache.hbase" % "hbase-hadoop2-compat" % hbaseVersion % "provided"
val hbaseServer = "org.apache.hbase" % "hbase-server" % hbaseVersion % "provided" excludeAll ExclusionRule(organization = "org.mortbay.jetty")
val sparkCore = "org.apache.spark" %% "spark-core" % sparkVersion % "provided"
val sparkStreaming = "org.apache.spark" %% "spark-streaming" % sparkVersion % "provided"
val sparkStreamingKafka = "org.apache.spark" %% "spark-streaming-kafka" % sparkVersion exclude("org.apache.spark", "spark-streaming_2.10")