如何解决Hbase Hortonworks NamespaceNotFoundException
我想使用 Hortonworks 库 (https://mvnrepository.com/artifact/com.hortonworks/shc/1.1.1-2.1-s_2.11) 从 Spark 连接到 HBase。 我将配置文件添加到 spark (hbase-site.xml) 中,该文件具有与我的 HBase 完全相同的属性。
这是完整的命令:
spark-shell --jars /home/me/Downloads/spark-hbase-connector_2.10-1.0.3.jar,/home/me/Downloads/hbase-spark-2.0.2.3.1.0.6-1.jar,/home/me/Downloads/spark-sql_2.12-2.4.7.jar,/home/me/Downloads/shc-core-1.1.1-2.1-s_2.11.jar,/home/me/Downloads/audience-annotations-0.13.0.jar --packages org.apache.hbase:hbase:2.4.1 --files /usr/local/spark/conf/hbase-site.xml
我尝试遵循 Hortonworks 插入方法的这个示例: https://github.com/sparkbyexamples/spark-hbase-hortonworks-examples/blob/master/src/main/scala/com/sparkbyexamples/spark/dataframe/hbase/hortonworks/HBaseSparkInsert.scala
我收到以下错误:
java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/NamespaceNotFoundException
at org.apache.spark.sql.execution.datasources.hbase.DefaultSource.createRelation(HBaseRelation.scala:59)
at org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:45)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:86)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)
at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)
at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:83)
at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:81)
at org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:677)
at org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:677)
at org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:80)
at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:127)
at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:75)
at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:677)
at org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:286)
at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:272)
... 51 elided
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.NamespaceNotFoundException
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
示例类中指定的命名空间是“default”。在我的 HBase 中,我也得到:
NAMESPACE
default
hbase
如果错误是 NamespaceNotFoundException 但我有这个命名空间,可能是什么问题导致了错误?
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。