本文转载:https://blog.csdn.net/helloxiaozhe/article/details/81027196
主要报错信息:
pyspark连接MysqL出错py4j.protocol.Py4JJavaError: An error occurred while calling o26.load.
: java.sql.sqlException: No suitable driver 解决方法
(1)错误提示: Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties Setting default log level to "WARN". To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel). 18/07/13 10:23:51 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 18/07/13 10:23:51 WARN Utils: Your hostname, localhost resolves to a loopback address: 127.0.0.1; using 10.2.33.175 instead (on interface en0) 18/07/13 10:23:51 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address Traceback (most recent call last): File "/Users/a6/Downloads/speiyou_di/hive/log_task/spark_local.py", line 16, in <module> .option("dbtable", "test_person").option("user", "root").option("password", "yyz!123456").load() File "/Library/Python/2.7/site-packages/pyspark/sql/readwriter.py", line 155, in load return self._df(self._jreader.load()) File "/Library/Python/2.7/site-packages/py4j-0.10.6-py2.7.egg/py4j/java_gateway.py", line 1160, in __call__ answer, self.gateway_client, self.target_id, self.name) File "/Library/Python/2.7/site-packages/pyspark/sql/utils.py", line 63, in deco return f(*a, **kw) File "/Library/Python/2.7/site-packages/py4j-0.10.6-py2.7.egg/py4j/protocol.py", line 320, in get_return_value format(target_id, ".", name), value) py4j.protocol.Py4JJavaError: An error occurred while calling o26.load. : java.sql.sqlException: No suitable driver at java.sql.DriverManager.getDriver(DriverManager.java:315) at org.apache.spark.sql.execution.datasources.jdbc.JDBcoptions$$anonfun$7.apply(JDBcoptions.scala:84) at org.apache.spark.sql.execution.datasources.jdbc.JDBcoptions$$anonfun$7.apply(JDBcoptions.scala:84) at scala.Option.getorElse(Option.scala:121) at org.apache.spark.sql.execution.datasources.jdbc.JDBcoptions.<init>(JDBcoptions.scala:83) at org.apache.spark.sql.execution.datasources.jdbc.JDBcoptions.<init>(JDBcoptions.scala:34) at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:32) at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:330) at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:152) at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:125) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) at py4j.Gateway.invoke(Gateway.java:280) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.GatewayConnection.run(GatewayConnection.java:214) at java.lang.Thread.run(Thread.java:748)
(2)出错代码:
from pyspark import SparkConf, sqlContext from pyspark import SparkContext conf=SparkConf().setAppName("miniProject").setMaster("local[*]") sc = SparkContext(appName="MysqLtest") sqlContext =sqlContext(sc) df = sqlContext.read.format("jdbc").options( url="jdbc:MysqL://xxxx:xxxx/ec_order?user=root&password=xxxx", dbtable="dim_moonr_oms_channel_shop_info").load() df.show()
(3)解决办法:
最下面的java.sql.sqlException: No suitable driver才是重点
解决办法是将MysqL的驱动jar包,放到D:\Program Files\Java\jdk1.8.0_91\jre\lib\ext目录下(JAVA_HOME目录下的jre\lib\ext目录下)
然后就解决了这个问题
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。