使用 Spark 和 Hive 运行简单示例抛出异常

如何解决使用 Spark 和 Hive 运行简单示例抛出异常

按照 these 说明，我尝试运行一个同时使用 Spark 和 Hive 的简单程序：

from pyspark.sql import SparkSession
appName = "PySpark Hive Example"
master = "local[*]"
spark = SparkSession.builder \
             .appName(appName) \
             .master(master) \
             .enableHiveSupport() \
             .getorCreate()
# Read data using Spark
df = spark.sql("show databases")
df.show()

我收到此异常：

2021-07-25 20:46:21,477 WARN datanucleus.Query: Query for candidates of org.apache.hadoop.hive.metastore.model.MConstraint and subclasses resulted in no possible candidates
Class "org.apache.hadoop.hive.metastore.model.MDatabase" field "org.apache.hadoop.hive.metastore.model.MDatabase.catalogName" : declared in MetaData,but this field doesnt exist in the class!
org.datanucleus.Metadata.InvalidClassMetaDataException: Class "org.apache.hadoop.hive.metastore.model.MDatabase" field "org.apache.hadoop.hive.metastore.model.MDatabase.catalogName" : declared in MetaData,but this field doesnt exist in the class!
        at org.datanucleus.Metadata.ClassMetaData.populateMemberMetaData(ClassMetaData.java:846)
        at org.datanucleus.Metadata.ClassMetaData.populate(ClassMetaData.java:219)
        at org.datanucleus.Metadata.MetaDataManagerImpl$1.run(MetaDataManagerImpl.java:2896)
        at java.security.AccessController.doPrivileged(Native Method)
        at org.datanucleus.Metadata.MetaDataManagerImpl.populateAbstractClassMetaData(MetaDataManagerImpl.java:2890)
        at org.datanucleus.Metadata.MetaDataManagerImpl.populateFileMetaData(MetaDataManagerImpl.java:2689)
        at org.datanucleus.api.jdo.Metadata.JDOMetaDataManager.loadXMLMetaDataForClass(JDOMetaDataManager.java:806)
        at org.datanucleus.api.jdo.Metadata.JDOMetaDataManager.getMetaDataForClassInternal(JDOMetaDataManager.java:406)
        at org.datanucleus.Metadata.MetaDataManagerImpl.getMetaDataForClass(MetaDataManagerImpl.java:1660)
        at org.datanucleus.Metadata.MetaDataManagerImpl.getMetaDataForClass(MetaDataManagerImpl.java:1607)
        at org.datanucleus.Metadata.AbstractClassMetaData.getReferencedClassMetaData(AbstractClassMetaData.java:1634)
        at org.datanucleus.Metadata.AbstractClassMetaData.getReferencedClassMetaData(AbstractClassMetaData.java:1592)
        at org.datanucleus.Metadata.MetaDataManagerImpl.getReferencedClassMetaData(MetaDataManagerImpl.java:3108)
        at org.datanucleus.Metadata.MetaDataManagerImpl.getReferencedClasses(MetaDataManagerImpl.java:3078)
        at org.datanucleus.store.rdbms.RDBMsstoreManager$ClassAdder.addClasstables(RDBMsstoreManager.java:2998)
        at org.datanucleus.store.rdbms.RDBMsstoreManager$ClassAdder.run(RDBMsstoreManager.java:2886)
        at org.datanucleus.store.rdbms.AbstractSchemaTransaction.execute(AbstractSchemaTransaction.java:119)
        at org.datanucleus.store.rdbms.RDBMsstoreManager.manageClasses(RDBMsstoreManager.java:1627)
        at org.datanucleus.store.rdbms.RDBMsstoreManager.getDatastoreClass(RDBMsstoreManager.java:672)
        at org.datanucleus.store.rdbms.query.RDBMSQueryUtils.getStatementForCandidates(RDBMSQueryUtils.java:425)
        at org.datanucleus.store.rdbms.query.JDOQLQuery.compileQueryFull(JDOQLQuery.java:865)
        at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:347)
        at org.datanucleus.store.query.Query.executeQuery(Query.java:1816)
        at org.datanucleus.store.query.Query.executeWithArray(Query.java:1744)
        at org.datanucleus.store.query.Query.execute(Query.java:1726)
        at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:374)
        at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:216)
        at org.apache.hadoop.hive.metastore.metastoreDirectsql.ensureDbInit(metastoreDirectsql.java:190)
        at org.apache.hadoop.hive.metastore.metastoreDirectsql.<init>(metastoreDirectsql.java:144)
        at org.apache.hadoop.hive.metastore.ObjectStore.initializeHelper(ObjectStore.java:410)
        at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:342)
        at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:303)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
        at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:58)
        at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67)
        at org.apache.hadoop.hive.metastore.Hivemetastore$HMSHandler.newRawStoreForConf(Hivemetastore.java:628)
        at org.apache.hadoop.hive.metastore.Hivemetastore$HMSHandler.getMSForConf(Hivemetastore.java:594)
        at org.apache.hadoop.hive.metastore.Hivemetastore$HMSHandler.getMS(Hivemetastore.java:588)
        at org.apache.hadoop.hive.metastore.Hivemetastore$HMSHandler.createDefaultDB(Hivemetastore.java:655)
        at org.apache.hadoop.hive.metastore.Hivemetastore$HMSHandler.init(Hivemetastore.java:431)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.hive.metastore.retryingHMSHandler.invokeInternal(retryingHMSHandler.java:148)
        at org.apache.hadoop.hive.metastore.retryingHMSHandler.invoke(retryingHMSHandler.java:107)
        at org.apache.hadoop.hive.metastore.retryingHMSHandler.<init>(retryingHMSHandler.java:79)
        at org.apache.hadoop.hive.metastore.retryingHMSHandler.getProxy(retryingHMSHandler.java:92)
        at org.apache.hadoop.hive.metastore.Hivemetastore.newretryingHMSHandler(Hivemetastore.java:6902)
        at org.apache.hadoop.hive.metastore.HivemetastoreClient.<init>(HivemetastoreClient.java:164)
        at org.apache.hadoop.hive.ql.Metadata.SessionHivemetastoreClient.<init>(SessionHivemetastoreClient.java:70)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.apache.hadoop.hive.metastore.metastoreUtils.newInstance(metastoreUtils.java:1707)
        at org.apache.hadoop.hive.metastore.retryingmetastoreClient.<init>(retryingmetastoreClient.java:83)
        ...

nestedThrowablesstackTrace:
Identifier principalName is unresolved (not a static field)
org.datanucleus.exceptions.NucleusUserException: Identifier principalName is unresolved (not a static field)

不知道如何解决这个问题。