Hadoop平台搭建说明1.Hadoop节点规划本次安装规划使用三个节点，每个节点都使用centos系统。

三个节点的hostname分别规划为：centoshadoop1、centoshadoop2、centoshadoop3（此处为本教程参数，可根据实际环境情况修改）三个节点的ip地址分别规划为：192.168.65.57、192.168.65.58、192.168.65.59（此处为本教程参数，根据实际环境情况修改）2.平台搭建使用的软件下载如下软件操作系统安装包：Centos6.3_x64Jdk安装包：jdk-6u37-linux-x64.binHadoop安装包：hadoop-1.1.2.tar.gz3.安装centos操作系统安装三个节点的操作系统，安装过程省略。

4.配置centoshadoop1节点4.1.修改节点hostname[root@localhost~]#vi/etc/sysconfig/networkHOSTNAME=centoshadoop1[root@localhost~]#vi/etc/hosts……192.168.65.57 centoshadoop1192.168.65.58 centoshadoop2192.168.65.59 centoshadoop3[root@localhost~]#reboot4.2.关闭iptables防火墙[root@ centoshadoop1~]#service iptables stop注意每次操作系统重启后都要操作4.3.建立无ssh密码登陆生成签名文件[root@ centoshadoop1~]#cd /root[root@ centoshadoop1~]#ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa[root@ centoshadoop1~]#cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys [root@ centoshadoop1~]#测试本地SSH无密码登录[root@ centoshadoop1~]#ssh centoshadoop14.4.安装jdk上传jdk-6u37-linux-x64.bin到/root目录下[root@ centoshadoop1~]#chmod 777 jdk-6u37-linux-x64.bin[root@ centoshadoop1~]#./jdk-6u37-linux-x64.bin[root@ centoshadoop1~]#ll查看生成jdk-6u37-linux-x64目录4.5.安装hadoop软件上传hadoop-1.1.2.tar.gz到/root目录下[root@ centoshadoop1~]#tar -zvxf hadoop-1.1.2.tar.gz[root@ centoshadoop1~]#ll查看生成hadoop-1.1.2目录[root@ centoshadoop1~]#vi /conf/core-site.xml<?xml version="1.0"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!-- Put site-specific property overrides in this file. --><configuration><property><name></name><value>hdfs://192.168.65.57:9000</value></property><property><name>hadoop.tmp.dir</name><value>/root/hadoop-1.1.2/tmp</value></property></configuration>[root@ centoshadoop1~]#vi hdfs-site.xml<?xml version="1.0"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Put site-specific property overrides in this file. --><configuration><property><name>dfs.replication</name><value>1</value></property></configuration>[root@ centoshadoop1~]#vi mapred-site.xml<?xml version="1.0"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Put site-specific property overrides in this file. --><configuration><property><name>mapred.job.tracker</name><value>192.168.65.57:9001</value></property></configuration>4.6.配置hadoop数据节点[root@ centoshadoop1~]#vi /root/hadoop-1.1.2/conf/masters 192.168.65.57 #secondaryNameNode[root@ centoshadoop1~]#vi /root/hadoop-1.1.2/conf/slaves 192.168.65.58 #datanode192.168.65.59 #datanode4.7.添加环境变量[root@ centoshadoop1~]#cd /root[root@ centoshadoop1~]#vi /etc/profileexport JAVA_HOME=/root/jdk1.6.0_37export JRE_HOME=/root/jdk1.6.0_37/jreexport HADOOP_HOME=/root/hadoop-1.1.2export HADOOP_HOME_WARN_SUPPRESS=1export CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/lib:$HADOOP_HOME/lib:$CLASSPATH export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$HADOOP_HOME/bin:$PATH[root@ centoshadoop1~]#vi ~/.bashrcexport JAVA_HOME=/root/jdk1.6.0_37export JRE_HOME=/root/jdk1.6.0_37/jreexport HADOOP_HOME=/root/hadoop-1.1.2export HADOOP_DEV_HOME=/root/hadoop-1.1.2export HADOOP_COMMON_HOME=/root/hadoop-1.1.2export HADOOP_HDFS_HOME=/root/hadoop-1.1.2export HADOOP_CONF_DIR=/root/hadoop-1.1.2/conf[root@ centoshadoop1~]#vi /root/hadoop-1.1.2/conf/hadoop-env.shexport JAVA_HOME=/root/jdk1.6.0_37export HADOOP_HOME_WARN_SUPPRESS=1export HADOOP_CLASSPATH=/root/hadoop-1.1.2:/root/hadoop-1.1.2/lib5.配置centoshadoop2节点5.1.修改节点hostname[root@localhost~]#vi/etc/sysconfig/networkHOSTNAME=centoshadoop2[root@localhost~]#vi/etc/hosts……192.168.65.57 centoshadoop1192.168.65.58 centoshadoop2192.168.65.59 centoshadoop3[root@localhost~]#reboot5.2.关闭iptables防火墙[root@ centoshadoop1~]#service iptables stop注意每次操作系统重启后都要操作5.3.建立无ssh密码登陆将centoshadoop1节点上/root/.ssh目录下的authorized_keys、known_hosts 、id_rsa.pub、id_rsa四个文件拷贝到本机的/root/.ssh目录下5.4.安装jdk上传jdk-6u37-linux-x64.bin到/root目录下[root@ centoshadoop1~]#chmod 777 jdk-6u37-linux-x64.bin[root@ centoshadoop1~]#./jdk-6u37-linux-x64.bin[root@ centoshadoop1~]#ll查看生成jdk-6u37-linux-x64目录5.5.安装hadoop软件上传hadoop-1.1.2.tar.gz到/root目录下[root@ centoshadoop1~]#tar -zvxf hadoop-1.1.2.tar.gz[root@ centoshadoop1~]#ll查看生成hadoop-1.1.2目录[root@ centoshadoop1~]#vi /conf/core-site.xml<?xml version="1.0"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!-- Put site-specific property overrides in this file. --><configuration><property><name></name><value>hdfs://192.168.65.57:9000</value></property><property><name>hadoop.tmp.dir</name><value>/root/hadoop-1.1.2/tmp</value></property></configuration>[root@ centoshadoop1~]#vi hdfs-site.xml<?xml version="1.0"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!-- Put site-specific property overrides in this file. --><configuration><property><name>dfs.replication</name><value>1</value></property></configuration>[root@ centoshadoop1~]#vi mapred-site.xml<?xml version="1.0"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!-- Put site-specific property overrides in this file. --><configuration><property><name>mapred.job.tracker</name><value>192.168.65.57:9001</value></property></configuration>5.6.添加环境变量[root@ centoshadoop1~]#cd /root[root@ centoshadoop1~]#vi /etc/profileexport JAVA_HOME=/root/jdk1.6.0_37export JRE_HOME=/root/jdk1.6.0_37/jreexport HADOOP_HOME=/root/hadoop-1.1.2export HADOOP_HOME_WARN_SUPPRESS=1export CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/lib:$HADOOP_HOME/lib:$CLASSPATH export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$HADOOP_HOME/bin:$PATH[root@ centoshadoop1~]#vi ~/.bashrcexport JAVA_HOME=/root/jdk1.6.0_37export JRE_HOME=/root/jdk1.6.0_37/jreexport HADOOP_HOME=/root/hadoop-1.1.2export HADOOP_DEV_HOME=/root/hadoop-1.1.2export HADOOP_COMMON_HOME=/root/hadoop-1.1.2export HADOOP_HDFS_HOME=/root/hadoop-1.1.2export HADOOP_CONF_DIR=/root/hadoop-1.1.2/conf[root@ centoshadoop1~]#vi /root/hadoop-1.1.2/conf/hadoop-env.shexport JAVA_HOME=/root/jdk1.6.0_37export HADOOP_HOME_WARN_SUPPRESS=1export HADOOP_CLASSPATH=/root/hadoop-1.1.2:/root/hadoop-1.1.2/lib6.配置centoshadoop3节点6.1.修改节点hostname[root@localhost~]#vi/etc/sysconfig/networkHOSTNAME=centoshadoop3[root@localhost~]#vi/etc/hosts……192.168.65.57 centoshadoop1192.168.65.58 centoshadoop2192.168.65.59 centoshadoop3[root@localhost~]#reboot关闭iptables防火墙、建立无ssh密码登陆、安装jdk、安装hadoop软件、添加环境变量等操作跟“配置centoshadoop2节点”完全相同7.启动运行hadoop平台7.1.首先执行格式化[root@ centoshadoop1~]#hadoop namenode –format注意：core-site.xml、hdfs-site.xml、mapred-site.xml等配置文件变化后都需要执行格式化7.2.启动hadoop[root@ centoshadoop1~]#start-all.sh7.3.停止hadoop[root@ centoshadoop1~]#stop-all.sh7.4.查看hadoop运行状态查看hadoop启动的进程[root@ centoshadoop1~]#jps查看hadoop集群状态[root@ centoshadoop1~]#hadoop dfsadmin -report 查看hadoop中的目录情况[root@ centoshadoop1~]#hadoop fs –ls /。