LVS FULLNAT 模式安装 By 清风徐来 605612253@1 部署规划依照淘宝开源的 FULLNAT 模式 LVS，规划使用版本信息： Linux 内核：2.6.32-220.23.1.el6 LVS 版本：version 1.2.1 Keepalived 版本：v1.2.2 序号 主机 IP 域名 作用 备注 1 10.142.67.121 TEST-DEV-121 Master 编译内核+LVS+keepalived 2 10.142.67.122 TEST-DEV-122 Backup 编译内核+LVS+keepalived 3 10.142.78.74 TEST-BDD-074 Hiveserver2 编译内核+hiveserver2 4 10.142.78.76 TEST-BDD-076 Hiveserver2 编译内核+hiveserver22 LVS 安装2.1 内核编译内核编译需要在 master 和 backup 节点都执行，以下以 master 节点为例 安装脚本：compilekerna-LVSmaster.sh由于不能上外网，所以提前把对应的安装包下好 下载的暂时放在家目录 [op@TEST-DEV-121 ~]$ ls compilekerna-LVSmaster.sh kernel-2.6.32-220.23.1.el6.src.rpm Lvs-fullnat-synproxy.tar.gz 安装 此处由于是编译内核，使用 root 用户来执行，以免遇到各种权限问题 [op@TEST-DEV-121 ~]$ sudo chmod +x compilekerna-LVSmaster.sh[op@TEST-DEV-121 ~]$ sudo su – [root@TEST-DEV-121 ~]# cd /home/op/ [root@TEST-DEV-121 op]# uname -r 2.6.32-431.el6.x86_64 # 此处是编译前看一下内核版本号 [root@TEST-DEV-121 op]# ./compilekerna-LVSmaster.sh 安装依赖包的时候，发现默认 yum 源没有这些东西 Error Downloading Packages: newt-devel-0.52.11-3.el6.x86_64: failure: Packages/newt-devel-0.52.11-3.el6.x86_64.rpm from base: [Errno 256] No more mirrors to try. slang-devel-2.2.1-1.el6.x86_64: failure: Packages/slang-devel-2.2.1-1.el6.x86_64.rpm from base: [Errno 256] No more mirrors to try. asciidoc-8.4.5-4.1.el6.noarch: failure: Packages/asciidoc-8.4.5-4.1.el6.noarch.rpm from base: [Errno 256] No more mirrors to try. 所以，要自己下newt-devel-0.52.11-3.el6.x86_64.rpm slang-devel-2.2.1-1.el6.x86_64.rpmasciidoc-8.4.5-4.1.el6.noarch.rpm[root@TEST-DEV-121 op]# rpm -ivh slang-devel-2.2.1-1.el6.x86_64.rpm warning: slang-devel-2.2.1-1.el6.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 192a7d7d: NOKEY Preparing... ########################################### [100%] 1:slang-devel ########################################### [100%] [root@TEST-DEV-121 op]# rpm -ivh newt-devel-0.52.11-3.el6.x86_64.rpmwarning: newt-devel-0.52.11-3.el6.x86_64.rpm: Header V3 RSA/SHA1 Signature, key ID c105b9de: NOKEY Preparing... ########################################### [100%] 1:newt-devel ########################################### [100%] [root@TEST-DEV-121 op]# rpm -ivh asciidoc-8.4.5-4.1.el6.noarch.rpm warning: asciidoc-8.4.5-4.1.el6.noarch.rpm: Header V4 DSA/SHA1 Signature, key ID 192a7d7d: NOKEY Preparing... ########################################### [100%] 1:asciidoc ########################################### [100%] 再次运行即可 编译结束后： DEPMOD 2.6.32sh /home/op/rpms/BUILD/kernel-2.6.32-220.23.1.el6/linux-2.6.32-22 0.23.1.el6.x86_64/arch/x86/boot/install.sh 2.6.32 arch/x86/boot/bzImage \ System.map "/boot" ERROR: modinfo: could not find module xen_procfs ERROR: modinfo: could not find module xen_scsifront ERROR: modinfo: could not find module xen_hcall 感觉像是虚拟机是 xen 做的底层支持。
但是 kernel 里面没有 xen 支持模块，所以 会报错。
所以应该在时提及上没问题。
之后修改启动顺序 [root@TEST-DEV-121 ~]# vim /boot/grub/grub.conf default=0 之后重启查看是否内核变成了 2.6 [root@TEST-DEV-121 ~]# sudo reboot 之后[op@TEST-DEV-121 ~]$ uname -r 2.6.322.2 Keepalived 安装注意，一定要先安装 keepalived 服务，再安装 LVS 服务，否则会报错；同时两 个节点都需要安装。
2.2.1 安装包/images/a/a5/Lvs-fullnat-synproxy.tar.gz 里面有个 tools 压缩包就是了2.2.2 部署安装解压 [op@TEST-DEV-121 ~]$ tar -zxvf lvs-tools.tar.gz 安装依赖 [op@TEST-DEV-121 keepalived]$ sudo yum install -y openssl openssl-devel popt-devel kernel-devel popt* 报错，发现没有这个包 Error Downloading Packages: popt-static-1.13-7.el6.x86_64: failure: Packages/popt-static-1.13-7.el6.x86_64.rpm from base: [Errno 256] No more mirrors to try. 下载对应的包即可popt-static-1.13-7.el6.x86_64.rpm[op@TEST-DEV-121 ~]$ sudo rpm -ivh popt-static-1.13-7.el6.x86_64.rpm warning: popt-static-1.13-7.el6.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID c105b9de: NOKEY Preparing... ########################################### [100%]1:popt-static ########################################### [100%] 编译安装 keepalived [op@TEST-DEV-121 ~]$ cd tools/ [op@TEST-DEV-121 tools]$ cd keepalived/ [op@TEST-DEV-121 keepalived]$ ./configure --with-kernel-dir="/lib/modules/2.6.32/build" 检查结果 Keepalived configuration -----------------------Keepalived version Compiler Compiler flags Extra Lib Use IPVS Framework : 1.2.2 : gcc : -g -O2 : -lpopt -lssl -lcrypto : YesIPVS sync daemon support : Yes IPVS use libnl Use VRRP Framework Use Debug flags 三个 YES 即可。
[op@TEST-DEV-121 keepalived]$ make [op@TEST-DEV-121 keepalived]$ sudo make install 添加系统管理 [op@TEST-DEV-121 keepalived]$ sudo cp /usr/local/etc/rc.d/init.d/keepalived /etc/init.d/ [op@TEST-DEV-121 keepalived]$ sudo cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/ [op@TEST-DEV-121 keepalived]$ sudo mkdir /etc/keepalived [op@TEST-DEV-121 keepalived]$ sudo cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/ : No : Yes : No[op@TEST-DEV-121 keepalived]$ sudo cp /usr/local/sbin/keepalived /usr/sbin/ 由于默认 keepalived 的日志是在 message 里面，现在指定文件接受 keepalived 日 志 [op@TEST-DEV-121 keepalived]$ sudo vim /etc/sysconfig/keepalived #KEEPALIVED_OPTIONS="-D" KEEPALIVED_OPTIONS="-D -d -S 0"然后在 rsyslog 中添加一行 [op@TEST-DEV-121 keepalived]$ sudo vim /etc/rsyslog.conf #keepalived log local0.* /var/log/keepalived.log修改配置文件后启动 [op@TEST-DEV-121 ~]$ sudo /etc/init.d/rsyslog restart 关闭系统日志记录器： 启动系统日志记录器： 启动 [op@TEST-DEV-121 keepalived]$ sudo service keepalived start [确定] [确定]2.3 Lvs 安装安装包同上，都在 tools 目录下 编译安装 [op@TEST-DEV-121 ~]$ cd tools/ipvsadm/ [op@TEST-DEV-121 ipvsadm]$ sudo make [op@TEST-DEV-121 ipvsadm]$ sudo make install 验证 [op@TEST-DEV-121 ipvsadm]$ sudo ipvsadm IP Virtual Server version 1.2.1 (size=4194304)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port InActConn Forward Weight ActiveConn[op@TEST-DEV-121 ~]$ lsmod | grep ip_vs ip_vs ipv6 161155 323428 0 283 ip_vs3 RealServer 端安装3.1 源码编译下载地址： /images/3/34/Linux-2.6.32-220.23.1.el6.x86_64.rs.src.t ar.gz 解压 [op@TEST-BDD-064 ding]$ ls Linux-2.6.32-220.23.1.el6.x86_64.rs.src.tar.gz [op@TEST-BDD-064 ding]$ tar -zxvf Linux-2.6.32-220.23.1.el6.x86_64.rs.src.tar.gz 修改.config 文件 安装几个依赖包 yum install -y openssl openssl-devel popt-devel kernel-devel [op@TEST-BDD-074 ~]$ cd ding/ [op@TEST-BDD-074 ding]$ cd linux-2.6.32-220.23.1.el6.x86_64.rs/ [op@TEST-BDD-074 linux-2.6.32-220.23.1.el6.x86_64.rs]$ [op@TEST-BDD-074 linux-2.6.32-220.23.1.el6.x86_64.rs]$ vim .config 将 CONFIG_IPV6=M 改成 CONFIG_IPV6=y 编辑 makefile，可以加一下后缀，这样 uname –r 的时候可以显示出来，当然 不加也可以。