建站
咨詢
售后
建站咨詢
新聞中心
集群組成：
創(chuàng)新互聯(lián)專業(yè)為企業(yè)提供普洱網(wǎng)站建設(shè)、普洱做網(wǎng)站、普洱網(wǎng)站設(shè)計、普洱網(wǎng)站制作等企業(yè)網(wǎng)站建設(shè)、網(wǎng)頁設(shè)計與制作、普洱企業(yè)網(wǎng)站模板建站服務(wù)，十多年普洱做網(wǎng)站經(jīng)驗，不只是建網(wǎng)站，更提供有價值的思路和整體網(wǎng)絡(luò)服務(wù)。
一臺主機，一臺從節(jié)點。
系統(tǒng)環(huán)境：
操作系統(tǒng)：centos 7，64位，7.4.1708（/etc/RedHat-release中查看）
CPU：AMD Fx-8300 8核
內(nèi)存：8GB
硬盤：120GB
GNOME：3.22.2
安裝版本：
GPDB：V5.4.1
GPORCA：V2.53.11
前提條件：禁用防火墻（所有節(jié)點和主機都要禁用！?。?/strong>
使用root賬號執(zhí)行下列命令（同時禁用默認(rèn)的防火墻和可能已經(jīng)安裝的iptables，共兩個防火墻程序）：
關(guān)閉默認(rèn)的防火墻
# systemctl stop firewalld
屏蔽默認(rèn)的防火墻（重啟后也不會啟動）
# systemctl mask firewalld
關(guān)閉iptables
# systemctl stop iptables
禁用iptables
# systemctl disable iptables
安裝過程
一）創(chuàng)建專有賬號gpdba，并加入root用戶組。
二）修改所有服務(wù)器的主機名（所有節(jié)點和主機）
1）修改hosts使用命令 vi /etc/hosts 來修改
127.0.0.1 localhost localhost.localdomain
192.168.58.102 Master shsm002
192.168.58.104 Slave1 shsm004
最后，再輸入 source /etc/profile 刷新。
2）修改network文件，輸入命令vi /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=對應(yīng)的主機名稱
3）如果主機名稱與設(shè)備名稱不符，則按照下列格式修改：
127.0.0.1 localhost localhost.localdomain
IP地址 主機名稱 設(shè)備名稱
最后使用ping命令驗證是否可以連通。
三）修改系統(tǒng)文件（所有節(jié)點和主機）
1）修改內(nèi)核配置
vi /etc/sysctl.conf，添加下面內(nèi)容：
kernel.shmmax = 5000000000
kernel.shmmni = 4096
kernel.shmall = 4000000000
kernel.sem = 250 512000 100 2048
kernel.sysrq = 1
kernel.core_uses_pid = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.msgmni = 2048
net.ipv4.tcp_syncookies = 1
net.ipv4.ip_forward = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.conf.all.arp_filter = 1
net.ipv4.ip_local_port_range = 1025 65535
net.core.netdev_max_backlog = 10000
net.core.rmem_max = 2097152
net.core.wmem_max = 2097152
vm.overcommit_memory = 2
執(zhí)行命令 sysctl -p 使修改數(shù)值生效
2）修改限制配置
vi /etc/security/limits.conf
添加下面內(nèi)容：
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072
3）禁用SELINUX
vi /etc/selinux/config，修改SELINUX的值為disabled。修改后，如下：
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
# targeted - Targeted processes are protected,
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
三）安裝優(yōu)化器GPORCA的依賴項（所有節(jié)點和主機）
1）安裝cmake（3.10.2）
下載：
$ wget http://www.cmake.org/files/v3.10/cmake-3.10.2.tar.gz
解壓：
$ tar xzf cmake-3.10.2.tar.gz
定位到解壓后的目錄中：
$ cd cmake-3.10.2
關(guān)于configure命令：
如果要查看詳細(xì)的配置選項，使用下面命令：
$ ./configure --help
執(zhí)行配置命令（安裝到目錄/usr/cmake）：
$ ./configure --prefix=/usr/cmake
編譯：
$ make
安裝：
# make install
最后進(jìn)行驗證：
$ /usr/cmake/bin/cmake -version
輸出類似下面內(nèi)容顯示出版本號：
cmake version 3.10.2
編輯修改/etc/profile文件，將cmake添加到環(huán)境變量定義中，添加下面內(nèi)容：
### CMAKE 3.10 ###
export PATH=/usr/cmake/bin:$PATH
2）安裝gp-xerces
使用gpdba賬號解壓源碼文件壓縮包，進(jìn)入解壓目錄，執(zhí)行下面命令。
mkdir build
cd build
../configure --prefix=/usr/local ##安裝到/usr/local目錄下
（注意：如果出錯，則使用root賬號執(zhí)行下面的make命令）
make
make install
3）安裝re2c（1.0.3）
進(jìn)入 http://re2c.org/install/install.html 頁面下載自己需要的版本
安裝re2c是由于配置ninja時需要
$ ./configure --prefix=/usr/local
（注意：使用root賬號執(zhí)行下面的make命令；如果用戶沒有在root用戶組中時）
$ make
$ make install
4）安裝Ninja
可以使用git下載：https://github.com/ninja-build/ninja.git
下載后進(jìn)入ninja目錄執(zhí)行如下命令:
./configure.py --bootstrap
由于最終結(jié)果只是一個二進(jìn)制文件ninja，之后使用root賬號拷貝ninja文件到/usr/bin目錄即可（/usr/bin目錄已經(jīng)在環(huán)境變量PATH中配置定義了）
Installation is not necessary because the only required file is the resulting ninja binary. However, to enable features like Bash completion and Emacs and Vim editing modes, some files in misc/ must be copied to appropriate locations.
特別說明：先在主機上安裝所有依賴項的程序，然后通過scp命令遠(yuǎn)程復(fù)制安裝包或壓縮包到其他節(jié)點上逐個執(zhí)行安裝。
四）安裝GPORCA
下載地址：https://github.com/greenplum-db/gporca
安裝GPORCA（GPDB-5.4.1對應(yīng)的依賴版本，2.53.11）
使用gpdba賬號解壓源碼文件壓縮包，進(jìn)入解壓目錄，執(zhí)行下面命令。
cmake -GNinja -H. -Bbuild
ninja install -C build
查看GPDB依賴的ORCA的版本信息：/gpdb-5.4.1/depends/conanfile_orca.txt文件
[requires]
orca/v2.53.11@gpdb/stable
安裝完成后，進(jìn)入/gporca/build目錄，執(zhí)行ctest命令進(jìn)行檢查
如果最后輸出類似如下結(jié)果：
100% tests passed, 0 tests failed out of 119
Total Test time (real) = 195.48 sec
這說明編譯成功了。
【刪除舊版的GPORCA】
進(jìn)入源文件目錄下，執(zhí)行命令
rm -rf build/*
rm -rf /usr/local/include/naucrates
rm -rf /usr/local/include/gpdbcost
rm -rf /usr/local/include/gpopt
rm -rf /usr/local/include/gpos
rm -rf /usr/local/lib/libnaucrates.so*
rm -rf /usr/local/lib/libgpdbcost.so*
rm -rf /usr/local/lib/libgpopt.so*
rm -rf /usr/local/lib/libgpos.so*
五）安裝GPDB（選擇版本5.4.1）
1）使用root賬號安裝依賴項
sudo yum install -y epel-release
sudo yum install -y apr-devel bison bzip2-devel cmake3 flex gcc gcc-c++ krb5-devel libcurl-devel libevent-devel libkadm5 libyaml-devel libxml2-devel perl-ExtUtils-Embed Python-devel python-paramiko python-pip python-psutil python-setuptools readline-devel xerces-c-devel zlib-devel
# Install lockfile with pip because the yum package `python-pip` is too old (0.8).
sudo pip install lockfile conan
2）下載源代碼文件，解壓后編譯安裝。
使用gpdba賬號進(jìn)入下載解壓的源文件目錄下，執(zhí)行命令（prefix后面的路徑/usr/gpdb是安裝目錄）
./configure --with-perl --with-python --with-libxml --with-gssapi --prefix=/usr/gpdb
如果沒有安裝ORCA，則可以使用：./configure --with-perl --with-python --with-libxml --with-gssapi --disable-orca --prefix=/usr/gpdb
然后執(zhí)行make
make -j8
最后執(zhí)行安裝
make -j8 install
3）分發(fā)
首先，創(chuàng)建服務(wù)器之間的ssh免密連接。
創(chuàng)建目錄/usr/gpdb-conf，在該目錄中創(chuàng)建主機清單文件hostlist，文件內(nèi)容如下：
Master
Salve1
然后繼續(xù)在gpdb-conf目錄中創(chuàng)建seg_hosts，文件內(nèi)容如下：
Slave1
刷新greenplum_path的配置
source /usr/gpdb/greenplum_path.sh
gpssh交換密鑰
gpssh-exkeys -f /usr/gpdb-conf/hostlist
最后，將安裝成功的文件夾壓縮打包
gtar -cvf /home/gpdba/gpdb-install-binary-5.4.1.tar /usr/gpdb
使用gpscp命令復(fù)制到其他節(jié)點上（或者先ssh后scp也可以）
gpscp -f /usr/gpdb-conf/seg_hosts /home/gpdba/gpdb-install-binary-5.4.1.tar =:/usr
使用gpssh連接主機與從節(jié)點，解壓tar文件，安裝路徑同主機的安裝路徑保持一致。
gpssh -f /usr/gpdb-conf/hostlist
master 節(jié)點連接 slave 節(jié)點之后，執(zhí)行所有命令都應(yīng)該有n份輸出才表示正常。
解壓文件
gtar -xvf gpdb-install-binary-5.4.1.tar
創(chuàng)建數(shù)據(jù)庫工作目錄
cd /home/gpdba/gpdata
mkdir gpdatap1 gpdatap2 gpdatam1 gpdatam2 gpmaster
4）初始化數(shù)據(jù)庫（在master主機）
配置bash_profile環(huán)境變量
vi .bash_profile
修改如下：
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/.local/bin:$HOME/bin
export PATH
## Greenplum Database
source /usr/gpdb/greenplum_path.sh
export MASTER_DATA_DIRECTORY=/home/gpdba/gpdata/gpmaster/gpseg-1
export PGPORT=2346
export PGDATABASE=testDB
保存后，刷新生效：
. ~/.bash_profile
配置數(shù)據(jù)庫的啟動參數(shù)
將/usr/gpdb/docs/cli_help/gpconfigs/gpinitsystem_config 文件 復(fù)制到 /usr/gpdb-conf 目錄下然后編輯，保留如下內(nèi)容：
# FILE NAME: gpinitsystem_config
# Configuration file needed by the gpinitsystem
################################################
#### REQUIRED PARAMETERS
################################################
#### Name of this Greenplum system enclosed in quotes.
ARRAY_NAME="Greenplum Data Platform"
#### Naming convention for utility-generated data directories.
SEG_PREFIX=gpseg
#### Base number by which primary segment port numbers
#### are calculated.
PORT_BASE=40000
#### File system location(s) where primary segment data directories
#### will be created. The number of locations in the list dictate
#### the number of primary segments that will get created per
#### physical host (if multiple addresses for a host are listed in
#### the hostfile, the number of segments will be spread evenly across
#### the specified interface addresses).
declare -a DATA_DIRECTORY=(/data1/primary /data1/primary /data1/primary /data2/primary /data2/primary /data2/primary)
#### OS-configured hostname or IP address of the master host.
MASTER_HOSTNAME=mdw
#### File system location where the master data directory
#### will be created.
MASTER_DIRECTORY=/data/master
#### Port number for the master instance.
MASTER_PORT=5432
#### Shell utility used to connect to remote hosts.
TRUSTED_SHELL=ssh
#### Maximum log file segments between automatic WAL checkpoints.
CHECK_POINT_SEGMENTS=8
#### Default server-side character set encoding.
ENCODING=UNICODE
################################################
#### OPTIONAL MIRROR PARAMETERS
################################################
#### Base number by which mirror segment port numbers
#### are calculated.
#MIRROR_PORT_BASE=50000
#### Base number by which primary file replication port
#### numbers are calculated.
#REPLICATION_PORT_BASE=41000
#### Base number by which mirror file replication port
#### numbers are calculated.
#MIRROR_REPLICATION_PORT_BASE=51000
#### File system location(s) where mirror segment data directories
#### will be created. The number of mirror locations must equal the
#### number of primary locations as specified in the
#### DATA_DIRECTORY parameter.
#declare -a MIRROR_DATA_DIRECTORY=(/data1/mirror /data1/mirror /data1/mirror /data2/mirror /data2/mirror /data2/mirror)
################################################
#### OTHER OPTIONAL PARAMETERS
################################################
#### Create a database of this name after initialization.
#DATABASE_NAME=name_of_database
#### Specify the location of the host address file here instead of
#### with the the -h option of gpinitsystem.
#MACHINE_LIST_FILE=/home/gpadmin/gpconfigs/hostfile_gpinitsystem
最后，執(zhí)行命令開始初始化：
gpinitsystem -c /usr/gpdb-conf/gpinitsystem_config -a
查詢并關(guān)閉配置指定端口的postgres進(jìn)程
刪除生成的未完成的數(shù)據(jù)庫文件（可能是所有節(jié)點服務(wù)器），/home/gpdba/gpdata/gpmaster/gpseg-1文件夾。
六）錯誤解決
錯誤：
[gpdba@shsm002 ~]$ gpssh-exkeys -f /usr/gpdb-conf/hostlist
Error: unable to import module: version conflict: '/usr/lib64/python2.7/site-packages/psutil/_psutil_linux.so' C extension module was built for another version of psutil (different than 2.2.1)
解決：重新安裝psutil。sudo pip install psutil==2.2.1
錯誤：
20180129:23:40:43:gpinitsystem:shsm002:gpdba-[FATAL]:-Found indication of postmaster process on port 2345 on Master host Script Exiting!
解決：關(guān)閉殺死占用端口2345的進(jìn)程。
先查詢進(jìn)程
$ lsof -i:2345
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
postgres 10738 gpadmin 3u IPv4 264510 0t0 TCP *:postgres (LISTEN)
postgres 10738 gpadmin 4u IPv6 264511 0t0 TCP *:postgres (LISTEN)
然后殺死進(jìn)程
$ kill -9 10738
錯誤：
20180207:00:14:09:005166 gpinitsystem:shsm002:gpdba-[INFO]:-Building the Master instance database, please wait...
20180207:00:14:17:005166 gpinitsystem:shsm002:gpdba-[INFO]:-Starting the Master in admin mode
20180207:00:14:23:gpinitsystem:shsm002:gpdba-[FATAL]:-Unknown host shsm004 Script Exiting!
20180207:00:14:23:005166 gpinitsystem:shsm002:gpdba-[WARN]:-Script has left Greenplum Database in an incomplete state
原因：hostname與用戶賬號的@后面的主機名稱不一致，hosts定義中也沒有shsm004，添加進(jìn)去即可。
解決：修改hosts文件，每行記錄為：IP地址 主機名 域名。將hostname數(shù)值shsm004放到域名字段保存即可。使用ping命令可以ping通。
錯誤:
20180207:00:05:00:003516 gpinitsystem:shsm002:gpdba-[INFO]:-Checking Master host
20180207:00:05:00:003516 gpinitsystem:shsm002:gpdba-[WARN]:-Have lock file /tmp/.s.PGSQL.2346.lock but no process running on port 2346
20180207:00:05:00:gpinitsystem:shsm002:gpdba-[FATAL]:-Found indication of postmaster process on port 2346 on Master host Script Exiting!
解決：刪除文件，rm /tmp/.s.PGSQL.2346.lock。
錯誤：
[gpdba@shsm002 ~]$ /bin/bash /home/gpdba/gpAdminLogs/backout_gpinitsystem_gpdba_20180207_225128
[FATAL]:-Not on original master host Master, backout script exiting!
解決：不使用這個腳本清理中間數(shù)據(jù)，直接刪除gpdata目錄下的未完成的數(shù)據(jù)庫文件即可。
錯誤：
20180207:23:39:31:028691 gpcreateseg.sh:shsm002:gpdba-[INFO][1]:-Start Function PROCESS_QE
20180207:23:39:31:028691 gpcreateseg.sh:shsm002:gpdba-[INFO][1]:-Processing segment Slave1
/usr/gpdb/bin/postgres: error while loading shared libraries: libgpopt.so.3: cannot open shared object file: No such file or directory
no data was returned by command ""/usr/gpdb/bin/postgres" -V"
The program "postgres" is needed by initdb but was either not found in the same directory as "/usr/gpdb/bin/initdb" or failed unexpectedly.
Check your installation; "postgres -V" may have more information.
/usr/gpdb/bin/postgres: error while loading shared libraries: libgpopt.so.3: cannot open shared object file: No such file or directory
no data was returned by command ""/usr/gpdb/bin/postgres" -V"
The program "postgres" is needed by initdb but was either not found in the same directory as "/usr/gpdb/bin/initdb" or failed unexpectedly.
Check your installation; "postgres -V" may have more information.
cat: /home/gpdba/gpdata/gpdatap1/gpseg0.initdb: No such file or directory
cat: /home/gpdba/gpdata/gpdatap2/gpseg1.initdb: No such file or directory
解決：修改/usr/gpdb/greenplum_path.sh文件，添加libgpopt.so.3文件所在目錄到環(huán)境變量LD_LIBRARY_PATH定義中，然后執(zhí)行source命令刷新（在重啟電腦之前，可能每次打開終端命令行時都需要手動刷新一下）。修改后的文件內(nèi)容如下：
GPHOME=/usr/gpdb
# Replace with symlink path if it is present and correct
if [ -h ${GPHOME}/../greenplum-db ]; then
GPHOME_BY_SYMLINK=`(cd ${GPHOME}/../greenplum-db/ && pwd -P)`
if [ x"${GPHOME_BY_SYMLINK}" = x"${GPHOME}" ]; then
GPHOME=`(cd ${GPHOME}/../greenplum-db/ && pwd -L)`/.
fi
unset GPHOME_BY_SYMLINK
fi
#setup PYTHONHOME
if [ -x $GPHOME/ext/python/bin/python ]; then
PYTHONHOME="$GPHOME/ext/python"
fi
PYTHONPATH=$GPHOME/lib/python
PATH=$GPHOME/bin:$PYTHONHOME/bin:$PATH
LD_LIBRARY_PATH=$GPHOME/lib:/usr/local/lib:${LD_LIBRARY_PATH-}
export LD_LIBRARY_PATH
OPENSSL_CONF=$GPHOME/etc/openssl.cnf
export GPHOME
export PATH
export PYTHONPATH
export PYTHONHOME
export OPENSSL_CONF
當(dāng)前文章：CentOS7下Greenplum源碼安裝教程
URL鏈接：http://m.fisionsoft.com.cn/article/cdjijco.html
