在HiveSQL执行过程中，报错Exception: Too many counters: 121 max=120

报错信息如下：

INFO  : Exception: Too many counters: 121 max=120
at org.apache.tez.common.counters.Limits.checkCounters(Limits.java:86)
at org.apache.tez.common.counters.Limits.incrCounters(Limits.java:93)
at org.apache.tez.common.counters.AbstractCounterGroup.addCounter(AbstractCounterGroup.java:76)
at org.apache.tez.common.counters.AbstractCounterGroup.addCounterImpl(AbstractCounterGroup.java:93)
at org.apache.tez.common.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:104)
at org.apache.tez.common.counters.AbstractCounterGroup.aggrAllCounters(AbstractCounterGroup.java:204)
at org.apache.tez.common.counters.AbstractCounters.aggrAllCounters(AbstractCounters.java:372)
at org.apache.tez.dag.app.dag.impl.DAGImpl.aggrTaskCounters(DAGImpl.java:773)
at org.apache.tez.dag.app.dag.impl.DAGImpl.getAllCounters(DAGImpl.java:722)
at org.apache.tez.dag.app.dag.impl.DAGImpl.getDAGStatus(DAGImpl.java:938)
at org.apache.tez.dag.app.dag.impl.DAGImpl.getDAGStatus(DAGImpl.java:981)

 

这是在执行Hive sql的时候报的错，引擎使用的是Tez。

解决方法：

需要修改Hadoop的mapred-site.xml配置文件的mapreduce.job.counters.max参数，默认为120，需要设置成更大的值。

<property> 
    <name>mapreduce.job.counters.max</name> 
    <value>200</value> 
</property> 

 

调整后再次报错：Concurrency mode is disabled, not creating a lock manager

报错信息如下：

ERROR : Failed: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask. 
INFO  : Completed executing command(queryId=dingli_20200311174547_a0c682f0-af9e-4f13-97c3-f4f076e88621); Time taken: 63.102 seconds
INFO  : Concurrency mode is disabled, not creating a lock manager
Error: Error while processing statement: Failed: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask. (state=08S01,code=1)

 

解决办法：修改hive的配置文件，取消该配置注释 exprot HADOOP_HEAPSIZE=1024

cd $HIVE_HOME/conf

cp hive-env.sh.template hive-env.sh

vim hive-env.sh

exprot HADOOP_HEAPSIZE=1024