如何从本地安装的 spark 连接到 aws-redshift？

如何解决如何从本地安装的 spark 连接到 aws-redshift？

下载了必要的库以从本地安装的 Spark 集群连接 redshift 并使用以下命令启动 pyspark，但我收到以下错误消息。

 pyspark --conf spark.executor.extraClasspath=/usr/share/java/redshift-jdbc42-2.0.0.4.jar --driver-class-path /usr/share/java/redshift-jdbc42-2.0.0.4.jar  --jars /usr/share/java/redshift-jdbc42-2.0.0.4.jar



df = spark.read.format("jdbc").option("url","jdbc:redshift://host/dbname").option("driver","com.amazon.redshift.jdbc42.driver").option("dbtable","tablename").option("user","username").option("password","password").load()

error 

 File "/usr/local/spark/python/lib/py4j-0.10.9-src.zip/py4j/protocol.py",line 328,in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o42.load.
: java.lang.classNotFoundException: com.amazon.redshift.jdbc42.driver
    at java.net.urlclassloader.findClass(urlclassloader.java:382)
    at java.lang.classLoader.loadClass(ClassLoader.java:418)
    at java.lang.classLoader.loadClass(ClassLoader.java:351)
    at org.apache.spark.sql.execution.datasources.jdbc.DriverRegistry$.register(DriverRegistry.scala:46)
    at org.apache.spark.sql.execution.datasources.jdbc.JDBcoptions.$anonfun$driverClass$1(JDBcoptions.scala:102)
    at org.apache.spark.sql.execution.datasources.jdbc.JDBcoptions.$anonfun$driverClass$1$adapted(JDBcoptions.scala:102)
    at scala.Option.foreach(Option.scala:407)
    at org.apache.spark.sql.execution.datasources.jdbc.JDBcoptions.<init>(JDBcoptions.scala:102)
    at org.apache.spark.sql.execution.datasources.jdbc.JDBcoptions.<init>(JDBcoptions.scala:38)
    at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:32)
    at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:354)
    at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:326)
    at org.apache.spark.sql.DataFrameReader.$anonfun$load$3(DataFrameReader.scala:308)
    at scala.Option.getorElse(Option.scala:189)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:308)
    at org.apache.spark

解决方法

不是"com.amazon.redshift.jdbc42.driver"， 但是"com.amazon.redshift.jdbc42.Driver"。