This is an old revision of the document!
Table of Contents
HDFS install
Kapcsolódó oldalak:
https://ccp.cloudera.com/display/CDH4DOC/Software+Configuration+for+Quorum-based+Storage
https://ccp.cloudera.com/display/CDHDOC/ZooKeeper+Installation
Cloudera repo csomag letöltés: https://ccp.cloudera.com/display/CDH4DOC/CDH4+Installation#CDH4Installation-DebianPackage
dpkg -i cdh4-repository_1.0_all.deb ; apt-get update
apt-get install openjdk-7-jre-headless:amd64
General configuration
.bashrc:
export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64
export ZK_HOME=/usr/lib/zookeeper
Zookeeper install
A következő 5 gépre kell feltelepíteni: mgmt1, mgmt2, stg1, stg2, stg3
apt-get install zookeeper-server
szerver inicializálás:
zookeeper-server-initialize –force
chown -R zookeeper:zookeeper /var/lib/zookeeper/
service start:
service zookeeper-server start
Zookeeper configuration
/etc/zookeeper/conf/zk-auth.txt:
digest:vodstorage01-zkfcs:<password>
chown root:hdfs /etc/zookeeper/conf/zk-auth.txt
chmod 640 /etc/zookeeper/conf/zk-auth.txt
java -cp $ZK_HOME/lib/*:$ZK_HOME/zookeeper-3.4.5-cdh4.2.0.jar org.apache.zookeeper.server.auth.DigestAuthenticationProvider vodstorage01-zkfcs:<password>
Minta output (a végén a digest jelszó):
...->vodstorage01-zkfcs:Ez4VnDSTGPKnPJHGhzp+2W74=
/etc/zookeeper/conf/zk-acl.txt:
digest:vodstorage01-zkfcs:<digest password in output>:rwcda
chown root:hdfs /etc/zookeeper/conf/zk-acl.txt
chmod 640 /etc/zookeeper/conf/zk-acl.txt
Journal install
apt-get install hadoop-hdfs-journalnode
Hadoop general configuration
/etc/default/hadoop:
export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64
Namenode install
apt-get install hadoop-hdfs-namenode
Namenode configuration
/etc/hadoop/conf/core-site.xml
<property> <name>fs.defaultFS</name> <value>hdfs://vodstorage01</value> </property>
su hdfs -c “hadoop namenode -format”
Datanode install
apt-get install hadoop-hdfs-datanode
HDFS kliens install
apt-get install hadoop-client
HDFS FUSE install
apt-get install hadoop-hdfs-fuse
with all dependency:
apt-get install hadoop-hdfs-fuse hadoop-client hadoop-mapreduce hadoop-0.20-mapreduce libssl0.9.8
libssl0.9.8 is available in squeeze, need to build one for wheezy
FUSE mount:
hadoop-fuse-dfs dfs://<namenode> <mount point> hadoop-fuse-dfs dfs://172.28.28.11:8020 /srv/users/default/content
