LVS相关原理

LVS的组成:

  • IPVS(kernel内核空间):运行在内核空间(LVS核心组件)
  • IPVSADM(User用户空间):运行在用户空间,管理集群服务的命令工具

LVS的原理:根据用户请求的套接字判断,分流至真是服务器的工作模块

LVS工作方式

工作模式

LVS – DR 模式

工作逻辑图

模式特点

  • 集群节点,必须在一个网络中
  • 真是服务器网关指向路由器
  • RIP既可以是私网地址,又可以是公网地址
  • 负载调度器只负责入站请求
  • 大大减轻负载调度器压力,支持更多的服务器节点

LVS —NAT 模式

模式特点

  • 集群节点,必须在一个网络中
  • 真是服务器必须将网关指向负载调度器
  • RIP通常都是私有IP,仅用于各个集群节点通信
  • 负载调度器必须位于客户端和真是服务器之间,充当网关
  • 支持端口映射
  • 负载调度器操作系统必须是Linux,真是服务器可以使用任意系统

LVS –TUN模式

工作逻辑图

模式特点

  • 集群节点不必位于同一个物理网络单必须都拥有公网ip(或都可以被路由)
  • 真实服务器不能将网关指向负载调度器
  • RIP必须是公网地址
  • 负载调度器只负责入站请求
  • 不支持端口映射功能
  • 发送方和接收方必须支持隧道功能

LVS集群构建

LVS-DR模式集群构建

实验架构图

C本机 10.10.10.240
Centos6.9Diectory10.10.10.11 10.10.10.100
Centos6.9RS110.10.10.12 vip:10.10.10.100
Centos6.9RS210.10.10.13 vip:10.10.10.100

每台机器都是两块网卡

基础环境搭建

三台服务器都需要配置

[root@localhost ~]# service iptables stop
[root@localhost ~]# setenforce 0
[root@localhost ~]# mkdir /media/cdrom/
[root@localhost ~]# mount /dev/cdrom /media/cdrom/
[root@localhost ~]# mount
/dev/mapper/VolGroup-lv_root on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw,rootcontext="system_u:object_r:tmpfs_t:s0")
/dev/sda1 on /boot type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
/dev/sr0 on /media/cdrom type iso9660 (ro)
[root@localhost ~]# cd /etc/yum.repos.d/
[root@localhost yum.repos.d]# mkdir a
[root@localhost yum.repos.d]# mv *.repo a
[root@localhost yum.repos.d]# mv ./a/CentOS-Media.repo  .
[root@localhost yum.repos.d]# ls
a  CentOS-Media.repo
[root@localhost yum.repos.d]# vim CentOS-Media.repo 
[root@localhost yum.repos.d]# cat CentOS-Media.repo 
[c6-media]
name=CentOS-$releasever - Media
baseurl=file:///media/cdrom/
gpgcheck=0
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
[root@localhost yum.repos.d]# yum clean all
已加载插件:fastestmirror, refresh-packagekit, security
Cleaning repos: c6-media
清理一切
Cleaning up list of fastest mirrors
[root@localhost yum.repos.d]# yum -y install gcc gcc-c++ lrzsz
[root@localhost yum.repos.d]# shutdown -h now
关机拍快照

重启操作
Service NetworkManager stop 					关闭网卡守护进程(如果系统没有安装图形化界面不用敲)

负载调度器配置

[root@localhost ~]# cd /etc/sysconfig/network-scripts/
[root@localhost network-scripts]# cp ifcfg-eth0 ifcfg-eth0:0
[root@localhost network-scripts]# vim !$
vim ifcfg-eth0:0
[root@localhost network-scripts]# cat ifcfg-eth0:0
DEVICE=eth0:0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=static
USERCTL=no
IPADDR=10.10.10.100
NETWASK=255.255.255.0
[root@localhost network-scripts]# ifup ifcfg-eth0:0
[root@localhost network-scripts]# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0C:29:83:47:26  
          inet addr:10.10.10.11  Bcast:10.255.255.255  Mask:255.0.0.0
          inet6 addr: fe80::20c:29ff:fe83:4726/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1867 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1101 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:167678 (163.7 KiB)  TX bytes:133877 (130.7 KiB)

eth0:0    Link encap:Ethernet  HWaddr 00:0C:29:83:47:26  
          inet addr:10.10.10.100  Bcast:10.255.255.255  Mask:255.0.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:808 (808.0 b)  TX bytes:808 (808.0 b)
[root@localhost ~]# vim /etc/sysctl.conf							//关闭网卡重定向
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0						//加在最下面
net.ipv4.conf.eth0.send_redirects = 0
[root@localhost ~]# sysctl -p
[root@localhost ~]# modprobe ip_vs						//重载ipvs模块
WARNING: Deprecated config file /etc/modprobe.conf, all config files belong into /etc/modprobe.d/.
(警告:配置文件/etc已弃用/modprobe.conf文件,所有配置文件都属于/etc/modprobe.d/。)
[root@localhost ~]# mv /etc/modprobe.conf /etc/modprobe.d/oprofile.conf			//解决方法
[root@localhost ~]# modprobe ip_vs							//重新输入
[root@localhost ~]# yum -y install ipvsadm						//安装ipvsadm

Ipvsadm -v //查看当前ipvs集群内容
Ipvsadm -A -t 虚拟IP:80 -s rr //添加ipvsTCP集群rr轮询
Ipvsadm -a -t 虚拟ip:80 -r 网站1:80 -g //添加ipvsadm 集群子节点
Ipvsadm -a -t 虚拟ip:80 -r 网站1:80 -g
Ipvsadm	-Ln
Service ipvsadm save		//保存ipvs集群内容至文件,进行持久化存储
Chkconfig ipvsadm on	//设置开机自启
root@localhost ~]# service ipvsadm start
ipvsadm: Saving IPVS table to /etc/sysconfig/ipvsadm:                                                    [确定]
ipvsadm: Clearing the current IPVS table:           [确定]
ipvsadm: Applying IPVS configuration:               [确定]
[root@localhost ~]# ipvsadm -v
ipvsadm v1.26 2008/5/15 (compiled with popt and IPVS v1.2.1)
[root@localhost ~]# ipvsadm -A -t 10.10.10.100:80 -s rr
[root@localhost ~]# ipvsadm -A -t 10.10.10.100:80 -s rr
[root@localhost ~]# ipvsadm -a -t 10.10.10.100:80 -r 10.10.10.12:80 -g
[root@localhost ~]# ipvsadm -a -t 10.10.10.100:80 -r 10.10.10.13:80 -g
[root@localhost ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.10.10.100:80 rr
  -> 10.10.10.12:80               Route   1      0          0         
  -> 10.10.10.13:80               Route   1      0          0     

[root@localhost ~]# service ipvsadm save
ipvsadm: Saving IPVS table to /etc/sysconfig/ipvsadm:      [确定]
[root@localhost ~]# chkconfig ipvsadm on

真是服务器配置

[root@localhost ~]# cd /etc/sysconfig/network-scripts/
[root@localhost network-scripts]# cp ifcfg-lo ifcfg-lo:0
[root@localhost network-scripts]# vim !$
[root@localhost network-scripts]# cat ifcfg-lo:0
DEVICE=lo:0
IPADDR=10.10.10.100
NETMASK=255.255.255.255
NETWORK=127.0.0.0
# If you're having problems with gated making 127.0.0.0/8 a martian,
# you can change this to something else (255.255.255.255, for example)
BROADCAST=127.255.255.255
ONBOOT=yes
NAME=loopback
[root@localhost network-scripts]# ifup ifcfg-lo:0
[root@localhost network-scripts]# ifcfonfig
-bash: ifcfonfig: command not found
[root@localhost network-scripts]# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0C:29:94:BF:BA  
          inet addr:10.10.10.12  Bcast:10.255.255.255  Mask:255.0.0.0
          inet6 addr: fe80::20c:29ff:fe94:bfba/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:595 errors:0 dropped:0 overruns:0 frame:0
          TX packets:369 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:54345 (53.0 KiB)  TX bytes:43419 (42.4 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

lo:0      Link encap:Local Loopback  
          inet addr:10.10.10.100  Mask:255.255.255.255
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
[root@localhost ~]# vim /etc/sysctl.conf			//关闭对应ARP响应功能
net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2
net.ipv4.conf.default.arp_ignore=1
net.ipv4.conf.default.arp_announce=2
net.ipv4.conf.lo.arp_ignore=1
net.ipv4.conf.lo.arp_announce=2
[root@localhost ~]# sysctl -p
[root@localhost ~]# route add -host 10.10.10.100 dev lo:0				//添加路由记录访问VIP交给lo:0网卡接受
[root@localhost ~]# service httpd start
正在启动 httpd:httpd: Could not reliably determine the server's fully qualified domain name, using localhost.localdomain for ServerName
                                                        [确定]
[root@localhost ~]# echo "11111" > /var/www/html/index.html		
[root@localhost ~]# curl localhost
11111
[root@localhost etc]# curl localhost		(R2 改成”2222”)
222222

访问10.10.10.100

NAT-模式集群构建

架构图

网络环境搭建

设配器更改

网络是适配器 > 打开VMware Network Adapter VMnet1 > 属性ipv4 > 高级添加20.20.20.22

负载配置

[root@localhost network-scripts]# cat ifcfg-eth0
DEVICE=eth0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=static
USERCTL=no
IPADDR=20.20.20.11
NETWASK=255.255.255.0
[root@localhost network-scripts]# cat ifcfg-eth1
DEVICE=eth1
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=static
USERCTL=no
IPADDR=10.10.10.11
NETWASK=255.255.255.0

[root@localhost network-scripts]# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0C:29:83:47:26  
          inet addr:20.20.20.11  Bcast:20.255.255.255  Mask:255.0.0.0
          inet6 addr: fe80::20c:29ff:fe83:4726/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4038 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2410 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:367452 (358.8 KiB)  TX bytes:305533 (298.3 KiB)

eth1      Link encap:Ethernet  HWaddr 00:0C:29:83:47:30  
          inet addr:10.10.10.11  Bcast:10.255.255.255  Mask:255.0.0.0
          inet6 addr: fe80::20c:29ff:fe83:4730/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:259 errors:0 dropped:0 overruns:0 frame:0
          TX packets:193 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:23538 (22.9 KiB)  TX bytes:29920 (29.2 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:808 (808.0 b)  TX bytes:808 (808.0 b)
[root@localhost ~]# service NetworkManager stop				//R1 R2 都需要操作
[root@localhost ~]# chkconfig NetworkManager off				//R1 R2 都需要操作

安装ipvsadm
[root@localhost ~]# mount /dev/cdrom /media/cdrom/
[root@localhost ~]# yum -y install ipvsadm
[root@localhost ~]# vim /etc/sysctl.conf
net.ipv4.ip_forward = 1						//forward=1 开启路由转发功能
[root@localhost ~]# sysctl  -p
[root@localhost ~]# service iptables start		//开启防火墙
[root@localhost ~]# chkconfig iptables on		//开机启动
[root@localhost ~]# iptables -F				//清空防火墙策略
[root@localhost ~]# iptables -t nat -A POSTROUTING -s 10.10.10.0/24 -o eth0 -j SNAT --to-source 20.20.20.11		//添加防火墙记录,当源地址是内网网段,并且出口网卡为eth0的时候进行snat的转换,转换源地址为外网卡地址。
[root@localhost ~]# iptables -t nat -L 				// 查看记录是否保存成功
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
SNAT       all  --  10.10.10.0/24        anywhere            to:20.20.20.11 

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

[root@localhost ~]# ipvsadm -A -t 20.20.20.11:80 -s rr //添加ipvsadm TCP集群
[root@localhost ~]# ipvsadm -a -t 20.20.20.11:80 -r 10.10.10.12:80 -m	//添加ipvsadm 节点
[root@localhost ~]# ipvsadm -a -t 20.20.20.11:80 -r 10.10.10.13:8080 -m 
[root@localhost ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  20.20.20.11:80 rr
  -> 10.10.10.12:80               Masq    1      0          0         
  -> 10.10.10.13:80               Masq    1      0          0         
[root@localhost ~]# service ipvsadm save		//保存ipvs集群设置到文件进行持久化
ipvsadm: Saving IPVS table to /etc/sysconfig/ipvsadm:                            [确定]
[root@localhost ~]# chkconfig ipvsadm on	

真是服务器:两台基本相似

在这里把服务器的网关指向负载调度器 10.10.10.11/24

[root@localhost ~]# echo "GATEWAY=10.10.10.11" >> /etc/sysconfig/network-scripts/ifcfg-eth0
[root@localhost ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0 
DEVICE=eth0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=static
USERCTL=no
IPADDR=10.10.10.12
NETWASK=255.255.255.0
GATEWAY=10.10.10.11
[root@localhost ~]# route -n						//这里查看去任何地方都交给10.10.10.11
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
169.254.0.0     0.0.0.0         255.255.0.0     U     1002   0        0 eth0
10.0.0.0        0.0.0.0         255.0.0.0       U     0      0        0 eth0
0.0.0.0         10.10.10.11     0.0.0.0         UG    0      0        0 eth0
[root@localhost ~]# service httpd start
[root@localhost ~]# echo "111111111" >> /var/www/html/index.html
[root@localhost ~]# vim /etc/httpd/conf/httpd.conf		
Listen 8080			//R2服务器操作端口改为8080
[root@localhost ~]# service httpd restart		//重启httpd
[root@localhost ~]# curl localhost				//R1操作
111111111
[root@localhost ~]# curl localhost:8080		//R2操作
222222222222222

验证实验是否成功

访问20.20.20.11

[root@localhost ~]# ipvsadm -Ln --stats
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port               Conns   InPkts  OutPkts  InBytes OutBytes
  -> RemoteAddress:Port
TCP  20.20.20.11:80                     36      158      150    17311    12657
  -> 10.10.10.12:80                     12       62       56     7498     5692
  -> 10.10.10.13:8080  

LVS-NAT模式负责出站和入站


该文章采用「CC 协议」,转载必须注明作者和本文链接.
分类: Linux