Hadoop装配总结
Hadoop安装总结
安装JDK
1、下载jdk1.6及以上版本，在/usr下安装
chmod u+x jdk-6u26-linux-i586.bin
./ jdk-6u26-linux-i586.bin
2、配置环境变量
vi /etc/profile
找到如下代码：
for i in /etc/profile.d/*.sh ; do
if [ -r "$i" ]; then
. $i
fi
done
在之后加入：
#java config
JAVA_HOME=/usr/jdk1.6.0_26
export JAVA_HOME
PATH=$PATH:$JAVA_HOME/bin
export PATH
CLASSPATH=.:$JAVA_HOME/lib
export CLASSPATH
3、配置软链接:
---删除旧的链接
cd /usr/bin
rm –rf java
rm –rf javac
---配置新的链接
ln –s /usr/jdk1.6.0_26/bin/java java
ln –s /usr/jdk1.6.0_26/bin/javac javac
4、测试是否安装成功: 看是否显示1.6版本
[root@localhost jdk1.6.0_26]# java -version
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) Client VM (build 20.1-b02, mixed mode, sharing)
新建用户
为保证利于管理，最好是新建一个hadoop用户，作为运行环境。
groupadd hadoop ---建立hadoop组
useradd -g hadoop hadoop --建立hadoop用户，加入hadoop组
passwd hadoop --设置密码
为设置ssh，需要将hadoop加入wheel组
usermod –g wheel hadoop
应该还有其他方式，设置hadoop组到可以使用ssh，暂时未研究。
配置SSH
在hadoop用户下：
[hadoop@localhost ~]$ ssh-keygen -t rsa
[hadoop@localhost ~]$ cat id_rsa.pub >> authorized_keys
测试：
ssh localhost
在单机配置伪分布方式可以按照如上执行，如果设置集群，则需要将id_rsa.pub 复制到各子机，然后导入验证密钥。
安装HADOOP
1、安装文件
到hadoop官方网站（http://hadoop.apache.org/）下载hadoop安装包，这里下载的是0.20.203。
上传到hadoop目录下：/home/hadoop
[hadoop@localhost ~]$ tar -zvxf hadoop-0.20.203.0rc1.tar.gz
2、配置环境变量：
[hadoop@localhost ~]$ vi /etc/profile
在java配置下添加如下：
export HADOOP_HOME=/home/hadoop/hadoop-0.20.203.0
export PATH=$PATH:$HADOOP_HOME/bin
注意刷新配置！
3、修改hadoop配置文件：
[hadoop@localhost conf]$ vi /home/hadoop/hadoop-0.20.203.0/conf/hadoop-env.sh
修改JAVA_HOME配置
# export JAVA_HOME=/usr/lib/j2sdk1.5-sun
export JAVA_HOME=/usr/jdk1.6.0_26
4、检查安装：
[hadoop@localhost ~]$ hadoop version
Hadoop 0.20.203.0
Subversion http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20-security-203 -r 1099333
Compiled by oom on Wed May 4 07:57:50 PDT 2011
5、配置伪分布模式配置文件
[hadoop@localhost conf]$ vi core-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost/</value>
</property>
</configuration>
[hadoop@localhost conf]$ vi hdfs-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
[hadoop@localhost conf]$ vi mapred-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>localhost:8021</value>
</property>
</configuration>
可参考：http://hadoop.apache.org/common/docs/current/single_node_setup.html
也可以把配置文件放在任意目录，只需要在启动守护进程时使用—config选项。
运行HADOOP
<!--[if !supportLists]-->1、 <!--[endif]-->格式化HDFS文件系统
[hadoop@localhost bin]$ hadoop namenode –format
如下是执行日志，可以看到运行参数信息，后续需要仔细研究字段含义：
[hadoop@localhost bin]$ hadoop namenode -format
11/08/13 12:52:56 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = localhost.localdomain/127.0.0.1
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 0.20.203.0
STARTUP_MSG: build = http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20-security-203 -r 1099333; compiled by 'oom' on Wed May 4 07:57:50 PDT 2011
************************************************************/
11/08/13 12:52:57 INFO util.GSet: VM type = 32-bit
11/08/13 12:52:57 INFO util.GSet: 2% max memory = 19.33375 MB
11/08/13 12:52:57 INFO util.GSet: capacity = 2^22 = 4194304 entries
11/08/13 12:52:57 INFO util.GSet: recommended=4194304, actual=4194304
11/08/13 12:52:58 INFO namenode.FSNamesystem: fsOwner=hadoop
11/08/13 12:52:59 INFO namenode.FSNamesystem: supergroup=supergroup
11/08/13 12:52:59 INFO namenode.FSNamesystem: isPermissionEnabled=true
11/08/13 12:52:59 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100
11/08/13 12:52:59 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)
11/08/13 12:52:59 INFO namenode.NameNode: Caching file names occuring more than 10 times
11/08/13 12:52:59 INFO common.Storage: Image file of size 112 saved in 0 seconds.
11/08/13 12:52:59 INFO common.Storage: Storage directory /tmp/hadoop-hadoop/dfs/name has been successfully formatted.
11/08/13 12:52:59 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at localhost.localdomain/127.0.0.1
************************************************************/
<!--[if !supportLists]-->2、 <!--[endif]-->启动守护进程
[hadoop@localhost bin]$ start-dfs.sh
starting namenode, logging to /home/hadoop/hadoop-0.20.203.0/bin/../logs/hadoop-hadoop-namenode-localhost.localdomain.out
localhost: starting datanode, logging to /home/hadoop/hadoop-0.20.203.0/bin/../logs/hadoop-hadoop-datanode-localhost.localdomain.out
localhost: starting secondarynamenode, logging to /home/hadoop/hadoop-0.20.203.0/bin/../logs/hadoop-hadoop-secondarynamenode-localhost.localdomain.out
[hadoop@localhost bin]$ start-mapred.sh
starting jobtracker, logging to /home/hadoop/hadoop-0.20.203.0/bin/../logs/hadoop-hadoop-jobtracker-localhost.localdomain.out
localhost: starting tasktracker, logging to /home/hadoop/hadoop-0.20.203.0/bin/../logs/hadoop-hadoop-tasktracker-localhost.localdomain.out
<!--[if !supportLists]-->3、 <!--[endif]-->关闭守护进程
[hadoop@localhost bin]$ stop-dfs.sh
[hadoop@localhost bin]$ stop-mapred.sh
<!--[if !supportLists]-->4、 <!--[endif]-->监控界面
http://192.168.128.133:50070/dfshealth.jsp
-------------------------------------------
作者：CNZQS|JesseZhang 个人博客：CNZQS(http://www.cnzqs.com)
版权声明：除非注明，文章均为原创，可以任意转载，转载时请务必以超链接形式标明文章原始出处和作者信息及本声明
--------------------------------------------