如何解决EMR PySpark 写入 Redshift:java.sql.SQLException:[Amazon](500310) 无效操作:会话是只读的
我在 EMR 集群上尝试使用 PySpark 将数据写入 Redshift 时出错。
df.write.format("jdbc") \
.option("url","jdbc:redshift://clustername.yyyyy.us-east-1.redshift.amazonaws.com:5439/db") \
.option("driver","com.amazon.redshift.jdbc42.Driver") \
.option("dbtable","public.table") \
.option("user",user_redshift) \
.option("password",password_redshift) \
.mode("overwrite") \
.save()
我得到的错误是:
py4j.protocol.Py4JJavaError: An error occurred while calling o143.save.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 Failed 4 times,most recent failure: Lost task 0.3 in stage 0.0 (TID 6,executor 1):
java.sql.sqlException: [Amazon](500310) Invalid operation: The session is read-only;
at com.amazon.redshift.client.messages.inbound.ErrorResponse.toErrorException(UnkNown Source)
at com.amazon.redshift.client.pgmessagingContext.handleErrorResponse(UnkNown Source)
at com.amazon.redshift.client.pgmessagingContext.handleMessage(UnkNown Source)
at com.amazon.jdbc.communications.InboundMessagesPipeline.getNextMessageOfClass(UnkNown Source)
at com.amazon.redshift.client.pgmessagingContext.doMovetoNextClass(UnkNown Source)
at com.amazon.redshift.client.pgmessagingContext.getParameterDescription(UnkNown Source)
at com.amazon.redshift.client.PGClient.prepareStatement(UnkNown Source)
at com.amazon.redshift.dataengine.PGQueryExecutor.<init>(UnkNown Source)
at com.amazon.redshift.dataengine.PGDataEngine.prepare(UnkNown Source)
at com.amazon.jdbc.common.SPreparedStatement.<init>(UnkNown Source)
...
感谢您的帮助。谢谢!
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。