如何解决Google Colab Notebook中的PySpark运行问题
在colab笔记本中运行以下代码后:
import findspark
findspark.init("E:/spark-3.0.1-bin-hadoop2.7")
import pyspark
出现此错误:
/usr/local/lib/python3.6/dist-packages/findspark.py in init(spark_home,python_path,edit_rc,edit_profile)
142 try:
--> 143 py4j = glob(os.path.join(spark_python,"lib","py4j-*.zip"))[0]
144 except IndexError:
IndexError: list index out of range
During handling of the above exception,another exception occurred:
Exception Traceback (most recent call last)
1 frames
/usr/local/lib/python3.6/dist-packages/findspark.py in init(spark_home,edit_profile)
144 except IndexError:
145 raise Exception(
--> 146 "Unable to find py4j,your SPARK_HOME may not be configured correctly"
147 )
148 sys.path[:0] = [spark_python,py4j]
Exception: Unable to find py4j,your SPARK_HOME may not be configured correctly
当我验证了spark文件夹时,我发现py4j模块已经存在!如下图所示: enter image description here
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。