CentOS6.8编译Hadoop-2.4.1

前面我写过一篇怎样编译hadoop-2.2.0的博客，今天我又试着编译了一下hadoop-2.4.1版本，编译的步骤跟编译hadoop-2.2.0的时候基本一样，编译hadoop-2.2.0的博客地址是：

http://blog.csdn.net/u012453843/article/details/52903324

我这里就只说下我遇到的问题

1.一定要确保服务器能够连网，因为要下载很多东西，如果不能连网，将无法成功编译

2.编译的过程中碰到了如下问题，后来从网上查找答案，发现是hadoop-2.4.1-src/hadoop-hdfs-project/hadoop-hdfs-httpfs/downloads目录下文件没有下载完全，缺少apache-tomcat-6.0.36.tar.gz，我们便下载一个并把它放到这个目录下，然后重新编译。

[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (dist) on project hadoop-hdfs-httpfs: An Ant BuildException has occured: Can't get http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.36/bin/apache-tomcat-6.0.36.tar.gz to /usr/local/hadoop/hadoop-2.4.1-src/hadoop-hdfs-project/hadoop-hdfs-httpfs/downloads/apache-tomcat-6.0.36.tar.gz
[ERROR] around Ant part ...<get dest="downloads/apache-tomcat-6.0.36.tar.gz" skipexisting="true" verbose="true" src="http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.36/bin/apache-tomcat-6.0.36.tar.gz"/>... @ 5:182 in /usr/local/hadoop/hadoop-2.4.1-src/hadoop-hdfs-project/hadoop-hdfs-httpfs/target/antrun/build-main.xml
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors,re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions,please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems,you can resume the build with the command
[ERROR] mvn <goals> -rf :hadoop-hdfs-httpfs
[root@test hadoop-2.4.1-src]# ^C
[root@test hadoop-2.4.1-src]#

经过了一段时间的编译终于编译成功

[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ................................ SUCCESS [5.031s]
[INFO] Apache Hadoop Project POM ......................... SUCCESS [3.855s]
[INFO] Apache Hadoop Annotations ......................... SUCCESS [7.075s]
[INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.569s]
[INFO] Apache Hadoop Project dist POM .................... SUCCESS [3.354s]
[INFO] Apache Hadoop Maven Plugins ....................... SUCCESS [9.588s]
[INFO] Apache Hadoop MiniKDC ............................. SUCCESS [8.882s]
[INFO] Apache Hadoop Auth ................................ SUCCESS [9.300s]
[INFO] Apache Hadoop Auth Examples ....................... SUCCESS [5.570s]
[INFO] Apache Hadoop Common .............................. SUCCESS [3:25.938s]
[INFO] Apache Hadoop NFS ................................. SUCCESS [18.331s]
[INFO] Apache Hadoop Common Project ...................... SUCCESS [0.082s]
[INFO] Apache Hadoop HDFS ................................ SUCCESS [6:12.442s]
[INFO] Apache Hadoop HttpFS .............................. SUCCESS [40.572s]
[INFO] Apache Hadoop HDFS BookKeeper Journal ............. SUCCESS [22.507s]
[INFO] Apache Hadoop HDFS-NFS ............................ SUCCESS [10.303s]
[INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.143s]
[INFO] hadoop-yarn ....................................... SUCCESS [0.134s]
[INFO] hadoop-yarn-api ................................... SUCCESS [3:06.873s]
[INFO] hadoop-yarn-common ................................ SUCCESS [1:13.959s]
[INFO] hadoop-yarn-server ................................ SUCCESS [0.167s]
[INFO] hadoop-yarn-server-common ......................... SUCCESS [26.176s]
[INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [31.386s]
[INFO] hadoop-yarn-server-web-proxy ...................... SUCCESS [8.314s]
[INFO] hadoop-yarn-server-applicationhistoryservice ...... SUCCESS [15.003s]
[INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [34.895s]
[INFO] hadoop-yarn-server-tests .......................... SUCCESS [0.996s]
[INFO] hadoop-yarn-client ................................ SUCCESS [14.592s]
[INFO] hadoop-yarn-applications .......................... SUCCESS [0.129s]
[INFO] hadoop-yarn-applications-distributedshell ......... SUCCESS [7.437s]
[INFO] hadoop-yarn-applications-unmanaged-am-launcher .... SUCCESS [5.101s]
[INFO] hadoop-yarn-site .................................. SUCCESS [0.125s]
[INFO] hadoop-yarn-project ............................... SUCCESS [9.303s]
[INFO] hadoop-mapreduce-client ........................... SUCCESS [0.138s]
[INFO] hadoop-mapreduce-client-core ...................... SUCCESS [53.614s]
[INFO] hadoop-mapreduce-client-common .................... SUCCESS [50.496s]
[INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [7.640s]
[INFO] hadoop-mapreduce-client-app ....................... SUCCESS [23.926s]
[INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [23.072s]
[INFO] hadoop-mapreduce-client-jobclient ................. SUCCESS [2:02.021s]
[INFO] hadoop-mapreduce-client-hs-plugins ................ SUCCESS [6.489s]
[INFO] Apache Hadoop MapReduce Examples .................. SUCCESS [22.299s]
[INFO] hadoop-mapreduce .................................. SUCCESS [10.712s]
[INFO] Apache Hadoop MapReduce Streaming ................. SUCCESS [30.326s]
[INFO] Apache Hadoop distributed copy .................... SUCCESS [58.770s]
[INFO] Apache Hadoop Archives ............................ SUCCESS [6.264s]
[INFO] Apache Hadoop Rumen ............................... SUCCESS [16.981s]
[INFO] Apache Hadoop Gridmix ............................. SUCCESS [11.848s]
[INFO] Apache Hadoop Data Join ........................... SUCCESS [6.313s]
[INFO] Apache Hadoop Extras .............................. SUCCESS [6.905s]
[INFO] Apache Hadoop Pipes ............................... SUCCESS [13.115s]
[INFO] Apache Hadoop OpenStack support ................... SUCCESS [12.148s]
[INFO] Apache Hadoop Client .............................. SUCCESS [11.822s]
[INFO] Apache Hadoop Mini-Cluster ........................ SUCCESS [0.269s]
[INFO] Apache Hadoop Scheduler Load Simulator ............ SUCCESS [36.518s]
[INFO] Apache Hadoop Tools dist .......................... SUCCESS [20.957s]
[INFO] Apache Hadoop Tools ............................... SUCCESS [0.067s]
[INFO] Apache Hadoop distribution ........................ SUCCESS [56.226s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 29:14.596s
[INFO] Finished at: Tue Nov 15 00:22:34 PST 2016
[INFO] Final Memory: 77M/237M
[INFO] ------------------------------------------------------------------------
[root@test hadoop-2.4.1-src]# ls
BUILDING.txt hadoop-client hadoop-hdfs-project hadoop-minicluster hadoop-tools NOTICE.txt
dev-support hadoop-common-project hadoop-mapreduce-project hadoop-project hadoop-yarn-project pom.xml
hadoop-assemblies hadoop-dist hadoop-maven-plugins hadoop-project-dist LICENSE.txt README.txt
[root@test hadoop-2.4.1-src]# cd hadoop-dist/
[root@test hadoop-dist]# ls
pom.xml target
[root@test hadoop-dist]# cd target/
[root@test target]# ls
antrun dist-tar-stitching.sh hadoop-2.4.1.tar.gz hadoop-dist-2.4.1-javadoc.jar maven-archiver
dist-layout-stitching.sh hadoop-2.4.1 hadoop-dist-2.4.1.jar javadoc-bundle-options test-dir [root@test target]#

原文地址：https://www.jb51.cc/centos/379897.html