如何解决无法在 Databricks 中安装 geopandas Pypi 包运行时 5.5 LTS Spark 2.4.3 Scala 2.11
我无法在 Databricks 中安装 geopandas 包。我正在使用集群运行时 5.5 LTS Spark 2.4.3 Scala 2.11
该软件包已成功安装在其他运行时版本中,但不是我需要的版本。
需要做什么才能在集群运行时 5.5 中安装这个包?
我正在使用以下命令
dbutils.library.installPyPI("geopandas")
以下是错误声明
org.apache.spark.SparkException: Process List(/local_disk0/pythonVirtualEnvDirs/virtualEnv-e9b469dd-aad9-4414-a208-03e3ecd8096c/bin/python,/local_disk0/pythonVirtualEnvDirs/virtualEnv-e9b469dd-aad9-4414-a208-03e3ecd8096c/bin/pip,install,geopandas,--disable-pip-version-check) exited with code 1. Command "python setup.py egg_info" Failed with error code 1 in /tmp/pip-install-bgvkkr58/fiona/
详细错误
---------------------------------------------------------------------------
Py4JJavaError Traceback (most recent call last)
<command-1887950226624660> in <module>()
1
----> 2 dbutils.library.installPyPI("geopandas")
/local_disk0/tmp/1625551234943-0/dbutils.py in installPyPI(self,project,version,repo,extras)
237 def installPyPI(self,version = "",repo = "",extras = ""):
238 return self.print_and_return(self.entry_point.getSharedDriverContext() \
--> 239 .addisolatedPyPILibrary(project,extras))
240
241 def restartPython(self):
/databricks/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py in __call__(self,*args)
1255 answer = self.gateway_client.send_command(command)
1256 return_value = get_return_value(
-> 1257 answer,self.gateway_client,self.target_id,self.name)
1258
1259 for temp_arg in temp_args:
/databricks/spark/python/pyspark/sql/utils.py in deco(*a,**kw)
61 def deco(*a,**kw):
62 try:
---> 63 return f(*a,**kw)
64 except py4j.protocol.Py4JJavaError as e:
65 s = e.java_exception.toString()
/databricks/spark/python/lib/py4j-0.10.7-src.zip/py4j/protocol.py in get_return_value(answer,gateway_client,target_id,name)
326 raise Py4JJavaError(
327 "An error occurred while calling {0}{1}{2}.\n".
--> 328 format(target_id,".",name),value)
329 else:
330 raise Py4JError(
Py4JJavaError: An error occurred while calling o413.addisolatedPyPILibrary.
: org.apache.spark.SparkException: Process List(/local_disk0/pythonVirtualEnvDirs/virtualEnv-e9b469dd-aad9-4414-a208-03e3ecd8096c/bin/python,--disable-pip-version-check) exited with code 1. Command "python setup.py egg_info" Failed with error code 1 in /tmp/pip-install-bgvkkr58/fiona/
at org.apache.spark.util.Utils$.executeAndGetoutput(Utils.scala:1403)
at org.apache.spark.util.Utils$.installLibrary(Utils.scala:836)
at org.apache.spark.SparkContext.addFile(SparkContext.scala:1700)
at org.apache.spark.SparkContext.addFile(SparkContext.scala:1632)
at com.databricks.backend.daemon.driver.SharedDriverContext$$anonfun$addisolatedPyPILibrary$1.apply$mcV$sp(SharedDriverContext.scala:558)
at com.databricks.backend.daemon.driver.SharedDriverContext$$anonfun$addisolatedPyPILibrary$1.apply(SharedDriverContext.scala:558)
at com.databricks.backend.daemon.driver.SharedDriverContext$$anonfun$addisolatedPyPILibrary$1.apply(SharedDriverContext.scala:558)
at com.databricks.logging.UsageLogging$$anonfun$recordOperation$1.apply(UsageLogging.scala:369)
at com.databricks.logging.UsageLogging$$anonfun$withAttributionContext$1.apply(UsageLogging.scala:238)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at com.databricks.logging.UsageLogging$class.withAttributionContext(UsageLogging.scala:233)
at com.databricks.backend.daemon.driver.SharedDriverContext.withAttributionContext(SharedDriverContext.scala:57)
at com.databricks.logging.UsageLogging$class.withAttributionTags(UsageLogging.scala:271)
at com.databricks.backend.daemon.driver.SharedDriverContext.withAttributionTags(SharedDriverContext.scala:57)
at com.databricks.logging.UsageLogging$class.recordOperation(UsageLogging.scala:350)
at com.databricks.backend.daemon.driver.SharedDriverContext.recordOperation(SharedDriverContext.scala:57)
at com.databricks.backend.daemon.driver.SharedDriverContext.addisolatedPyPILibrary(SharedDriverContext.scala:557)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:380)
at py4j.Gateway.invoke(Gateway.java:295)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:251)
at java.lang.Thread.run(Thread.java:748)
enter code here
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。