建站
咨詢
售后
建站咨詢
新聞中心
VMware建立虛擬機
配置Java環(huán)境
克隆虛擬機
在修改master的hosts,
創(chuàng)新互聯(lián)建站專注于漣源企業(yè)網(wǎng)站建設,成都響應式網(wǎng)站建設,商城網(wǎng)站建設。漣源網(wǎng)站建設公司,為漣源等地區(qū)提供建站服務。全流程按需制作網(wǎng)站，專業(yè)設計，全程項目跟蹤，創(chuàng)新互聯(lián)建站專業(yè)和態(tài)度為您提供的服務
192.168.197.132 master-01
192.168.197.133 slave-01
192.168.197.134 slave-02然后克隆matser,分別命名為slave1,slave2。
現(xiàn)在有三臺虛擬機
IP 虛擬機名稱 用戶
192.168.197.132 master yang
192.168.197.133 slave1 yang
192.168.197.134 slave2 yang
ssh免登陸
(1)centos默認沒有啟動ssh無密登錄，去掉/etc/ssh/sshd_config其中2行的注釋，每臺服務器都要設置，
#RSAAuthentication yes
#PubkeyAuthentication yes
安裝ssh
在master-01的機器上進入 yang用戶 的 .ssh 目錄
使用 ssh-keygen -t rsa 來生成公鑰和私鑰(連續(xù)回車，不設置密碼)
把公鑰文件復制到要訪問的機器的yang的用戶目錄下的.ssh 目錄
scp ~/.ssh/id_rsa.pub yang@master-01:/home/yang/.ssh/authorized_keys
scp ~/.ssh/id_rsa.pub yang@slave-01:/home/yang/.ssh/authorized_keys
scp ~/.ssh/id_rsa.pub yang@slave-02:/home/yang/.ssh/authorized_keys
檢測是否可以不需要密碼登陸
ssh localhost
ssh yang@master-01
ssh yang@slave-01
ssh yang@slave-02
這里只有master-01是master，如果有多個namenode，或者rm的話則需要打通所有master到其他剩余節(jié)點的免密碼登陸。（將master-01的authorized_keys追加到02和03的authorized_keys）
配置安裝Hadoop 2.7.3
下載Hadoop-2.7.3
下載Hadoop 2.7.3并解壓到/usr/software目錄下，在hadoop-2.7.3目錄下新建hdfs，hdfs/data，hdfs/name，hdfs/temp目錄。
配置core-site.xml
fs.defaultFS
hdfs://master-01:9000
hadoop.tmp.dir
file:/usr/software/hadoop-2.7.3/tmp
io.file.buffer.size
131702
配置mapred-site.xml
mapreduce.framework.name
yarn
mapreduce.jobhistory.address
master-01:10020
mapreduce.jobhistory.webapp.address
master-01:19888
配置yarn-site.xml
yarn.nodemanager.aux-services
mapreduce_shuffle
yarn.nodemanager.auxservices.mapreduce.shuffle.class
org.apache.hadoop.mapred.ShuffleHandler
yarn.resourcemanager.address
master-01:8032
yarn.resourcemanager.scheduler.address
master-01:8030
yarn.resourcemanager.resource-tracker.address
master-01:8031
yarn.resourcemanager.admin.address
master-01:8033
yarn.resourcemanager.webapp.address
master-01:8088
設置slaves
修改hadoop-2.7.3/etc/hadoop下的slaves文件，添加我們之前建立好的兩個slave
slave-01
slave-02網(wǎng)上很多地方說需要設置hadoop-env.sh和yarn-env.sh的Java環(huán)境，我看了這兩個文件的內(nèi)容，已經(jīng)幫我們配置好了，所以不用管。
配置完成
然后分別復制master下面的已經(jīng)配置好的Hadoop-2.7.3到y(tǒng)ang@slave-01和yang@slave02的/usr/software目錄下。
啟動
在Master服務器啟動hadoop，從節(jié)點會自動啟動，進入/usr/software/hadoop-2.7.3目錄
(1)初始化，輸入命令，bin/hdfs namenode -format
(2)啟動sbin/start-dfs.sh,輸出如下內(nèi)容，則成功
Starting namenodes on [master-01]
master-01: starting namenode, logging to /usr/software/hadoop-2.7.3/logs/hadoop-yang-namenode-master-01.out
slave-01: starting datanode, logging to /usr/software/hadoop-2.7.3/logs/hadoop-yang-datanode-slave-01.out
slave-02: starting datanode, logging to /usr/software/hadoop-2.7.3/logs/hadoop-yang-datanode-slave-02.out
Starting secondary namenodes [master-01]
master-01: starting secondarynamenode, logging to /usr/software/hadoop-2.7.3/logs/hadoop-yang-secondarynamenode-master-01.out(3)sbin/start-yarn.sh,如下則成功
[yang@master-01 hadoop-2.7.3]$ ./sbin/start-yarn.sh
starting yarn daemons
starting resourcemanager, logging to /usr/software/hadoop-2.7.3/logs/yarn-yang-resourcemanager-master-01.out
slave-02: starting nodemanager, logging to /usr/software/hadoop-2.7.3/logs/yarn-yang-nodemanager-slave-02.out
slave-01: starting nodemanager, logging to /usr/software/hadoop-2.7.3/logs/yarn-yang-nodemanager-slave-01.out(4)停止的話，輸入命令，sbin/stop-dfs.sh,sbin/stop-yarn.sh
(5)輸入命令，jps，可以看到相關信息
yang@master-01 hadoop-2.7.3]$ jps
6932 SecondaryNameNode
7384 Jps
6729 NameNode
7118 ResourceManager
[yang@master-01 hadoop-2.7.3]$ ./bin/hdfs dfsadmin -report
Configured Capacity: 75404550144 (70.23 GB)
Present Capacity: 54191501312 (50.47 GB)
DFS Remaining: 54191452160 (50.47 GB)
DFS Used: 49152 (48 KB)
DFS Used%: 0.00%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0
Missing blocks (with replication factor 1): 0
-------------------------------------------------
Live datanodes (2):
Name: 192.168.197.133:50010 (slave-01)
Hostname: slave-01
Decommission Status : Normal
Configured Capacity: 37702275072 (35.11 GB)
DFS Used: 24576 (24 KB)
Non DFS Used: 10606755840 (9.88 GB)
DFS Remaining: 27095494656 (25.23 GB)
DFS Used%: 0.00%
DFS Remaining%: 71.87%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Tue Sep 27 17:18:44 CST 2016
Name: 192.168.197.134:50010 (slave-02)
Hostname: slave-02
Decommission Status : Normal
Configured Capacity: 37702275072 (35.11 GB)
DFS Used: 24576 (24 KB)
Non DFS Used: 10606292992 (9.88 GB)
DFS Remaining: 27095957504 (25.24 GB)
DFS Used%: 0.00%
DFS Remaining%: 71.87%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Tue Sep 27 17:18:44 CST 2016Web訪問直接關閉防火墻
(1)瀏覽器打開http://192.168.197.132:8088/
(2)瀏覽器打開http://192.168.197.132:50070/
有如下信息：
Configured Capacity: 35.11 GB
DFS Used: 28 KB (0%)
Non DFS Used: 9.88 GB
DFS Remaining: 25.23 GB (71.87%)
Block Pool Used: 28 KB (0%)
DataNodes usages% (Min/Median/Max/stdDev): 0.00% / 0.00% / 0.00% / 0.00%
Live Nodes 1 (Decommissioned: 0)
Dead Nodes 1 (Decommissioned: 0)
Decommissioning Nodes 0
Total Datanode Volume Failures 0 (0 B)
Number of Under-Replicated Blocks 0
Number of Blocks Pending Deletion 0
Block Deletion Start Time 9/27/2016, 5:15:33 PM
總結
我在啟動的時候總是出現(xiàn)錯誤，提示權限問題，后來發(fā)現(xiàn)我之前的操作是用root用戶，然后hadoop-2.7.3的用戶組也是root，yang這個用戶根本沒有權限，那么問題找到了就修改唄，chrown修改為yang，問題解決。當然配置的過程中出現(xiàn)各種問題，都是參照網(wǎng)上的辦法和logs解決了。就不一一指出了，如果大家按照這個配置，還是有些問題，那么請多多百度，google吧。
當前文章：CentOS6.7安裝Hadoop2.7.3
文章源于：http://m.fisionsoft.com.cn/article/djigshg.html
