作者:俊惠芸菁亚扬 | 来源:互联网 | 2023-08-27 17:16
我是Apache Spark的新手,正尝试从Apache Spark连接到Presto。下面是我的连接字符串,它给出了错误。
val jdbcDF = spark.read.format("jdbc").options(Map("url" -> "jdbc:presto://host:port/hive?user=username&SSL=true&SSLTrustStorePath=/path/certificatefile","driver" -> "com.facebook.presto.jdbc.PrestoDriver","dbtable" -> "tablename","fetchSize" -> "10000","partitionColumn" -> "columnname","lowerBound" -> "1988","upperBound" -> "2016","numPartitions" -> "28")).load()
我首先在spark / sbin中启动了start-master.sh。我还尝试过在spark-shell中设置jar和驱动程序类路径,如下所示:
./spark-shell --driver-class-path com.facebook.presto.jdbc.PrestoDriver --jars /path/jar/file
仍然出现以下错误:
java.sql.SQLException: Unsupported type JAVA_OBJECT
at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.org$apache$spark$sql$execution$datasources$jdbc$JdbcUtils$$getcatalystType(JdbcUtils.scala:251)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$8.apply(JdbcUtils.scala:316)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$8.apply(JdbcUtils.scala:316)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.getSchema(JdbcUtils.scala:315)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolvetable(JDBCRDD.scala:63)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation$.getSchema(JDBCRelation.scala:210)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:35)
at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:318)
at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:167)
有人可以帮我吗?谢谢