Nginx配合keepalived實現(xiàn)雙主負載均衡
作者：loveganen
由于之前一直使用的是主從高可用加后端負載，隨著業(yè)務(wù)量的增大，前端一臺服務(wù)在高峰時期有些吃力，所有對之前架構(gòu)進行了一點改造，把主從升級成了雙主，可以更充分的利用現(xiàn)有服務(wù)器資源，由于實驗環(huán)境有限，后端的測試僅僅使用了一個靜態(tài)頁面，沒有搭建動態(tài)環(huán)境，也沒用使用數(shù)據(jù)庫，如有需要可以參考其他文檔。
前言：由于之前一直使用的是主從高可用加后端負載，隨著業(yè)務(wù)量的增大，前端一臺服務(wù)在高峰時期有些吃力，所有對之前架構(gòu)進行了一點改造，把主從升級成了雙主，可以更充分的利用現(xiàn)有服務(wù)器資源，由于實驗環(huán)境有限，后端的測試僅僅使用了一個靜態(tài)頁面，沒有搭建動態(tài)環(huán)境，也沒用使用數(shù)據(jù)庫，如有需要可以參考其他文檔。
一、架構(gòu)規(guī)劃
1、服務(wù)器IP地址規(guī)劃
VIP1：192.168.1.149
VIP2：192.168.1.150
Keepalived1:192.168.1.151
Keepalived2:192.168.1.152
WebServer1:192.168.1.201
WebServer2:192.168.1.202
2、服務(wù)器操作系統(tǒng)
所使用的操作系統(tǒng)均為CentOS release 6.6 (Final) x86_64，最小化安裝。
3、網(wǎng)絡(luò)拓撲圖
#p#
二、配置Nginx代理服務(wù)器
此部分Node1與Node2的配置完全相同。
1，準備編譯環(huán)境
|
1
|
# yum –y install gccgcc-c++ pcre-devel openssl openssl-devel wget |
2，編譯安裝nginx
- # ./configure \
- --prefix=/usr/local/nginx \
- --sbin-path=/usr/local/nginx/sbin/nginx \
- --conf-path=/etc/nginx/nginx.conf \
- --error-log-path=/var/log/nginx/error.log \
- --http-log-path=/var/log/nginx/access.log \
- --pid-path=/var/run/nginx/nginx.pid \
- --lock-path=/var/lock/nginx.lock \
- --user=nginx \
- --group=nginx \
- --with-http_ssl_module \
- --with-pcre
- # make && make install
3,為nginx提供SysV init腳本:
# vi /etc/rc.d/init.d/nginx
添加如下內(nèi)容
- #!/bin/sh
- #
- # nginx - this script starts and stopsthe nginx daemon
- #
- # chkconfig: - 85 15
- # description: Nginx is an HTTP(S) server, HTTP(S) reverse \
- # proxy and IMAP/POP3 proxy server
- # processname: nginx
- # config: /etc/nginx/nginx.conf
- # config: /etc/sysconfig/nginx
- # pidfile: /var/run/nginx.pid
- # Source function library.
- . /etc/rc.d/init.d/functions
- # Source networking configuration.
- . /etc/sysconfig/network
- # Check that networking is up.
- [ "$NETWORKING" = "no"] && exit 0
- nginx="/usr/local/nginx/sbin/nginx"
- prog=$(basename $nginx)
- NGINX_CONF_FILE="/etc/nginx/nginx.conf"
- [ -f /etc/sysconfig/nginx ] && ./etc/sysconfig/nginx
- lockfile=/var/lock/subsys/nginx
- make_dirs() {
- # make required directories
- user=`nginx -V 2>&1 | grep "configure arguments:" | sed's/[^*]*--user=\([^ ]*\).*/\1/g' -`
- options=`$nginx -V 2>&1 | grep 'configure arguments:'`
- for opt in $options; do
- if [ `echo $opt | grep '.*-temp-path'` ]; then
- value=`echo $opt | cut -d"=" -f 2`
- if [ ! -d "$value" ]; then
- # echo "creating"$value
- mkdir -p $value && chown-R $user $value
- fi
- fi
- done
- }
- start() {
- [ -x $nginx ] || exit 5
- [ -f $NGINX_CONF_FILE ] || exit 6
- make_dirs
- echo -n $"Starting $prog: "
- daemon $nginx -c $NGINX_CONF_FILE
- retval=$?
- echo
- [ $retval -eq 0 ] && touch $lockfile
- return $retval
- }
- stop() {
- echo -n $"Stopping $prog: "
- killproc $prog -QUIT
- retval=$?
- echo
- [ $retval -eq 0 ] && rm -f $lockfile
- return $retval
- }
- restart() {
- configtest || return $?
- stop
- sleep 1
- start
- }
- reload() {
- configtest || return $?
- echo -n $"Reloading $prog: "
- killproc $nginx -HUP
- RETVAL=$?
- echo
- }
- force_reload() {
- restart
- }
- configtest() {
- $nginx -t -c $NGINX_CONF_FILE
- }
- rh_status() {
- status $prog
- }
- rh_status_q() {
- rh_status >/dev/null 2>&1
- }
- case "$1" in
- start)
- rh_status_q && exit 0
- $1
- ;;
- stop)
- rh_status_q || exit 0
- $1
- ;;
- restart|configtest)
- $1
- ;;
- reload)
- rh_status_q || exit 7
- $1
- ;;
- force-reload)
- force_reload
- ;;
- status)
- rh_status
- ;;
- condrestart|try-restart)
- rh_status_q || exit 0
- ;;
- *)
- echo $"Usage: $0{start|stop|status|restart|condrestart|try-restart|reload|force-reload|configtest}"
- exit 2
- esac
而后為此腳本賦予執(zhí)行權(quán)限：
# chmod +x /etc/rc.d/init.d/nginx
添加至服務(wù)管理列表，并讓其開機自動啟動：
# chkconfig --add nginx
# chkconfig nginx on
4，配置Nginx代理
# cat nginx.conf
- user nginx nginx;
- worker_processes 1;
- pid /var/run/nginx/nginx.pid;
- worker_rlimit_nofile 51200;
- events
- {
- use epoll;
- worker_connections 51200;
- }
- http{
- include mime.types;
- default_type application/octet-stream;
- server_names_hash_bucket_size 128;
- client_header_buffer_size 32k;
- large_client_header_buffers 4 32k;
- client_max_body_size 8m;
- sendfile on;
- tcp_nopush on;
- keepalive_timeout 60;
- tcp_nodelay on;
- fastcgi_connect_timeout 300;
- fastcgi_send_timeout 300;
- fastcgi_read_timeout 300;
- fastcgi_buffer_size 64k;
- fastcgi_buffers 4 64k;
- fastcgi_busy_buffers_size 128k;
- fastcgi_temp_file_write_size 128k;
- gzip on;
- gzip_min_length 1k;
- gzip_buffers 4 16k;
- gzip_http_version 1.0;
- gzip_comp_level 2;
- gzip_types text/plain application/x-javascript text/css application/xml;
- gzip_vary on;
- upstream backend
- {
- ip_hash;
- server 192.168.1.201:80;
- server 192.168.1.202:80;
- }
- log_format access '$remote_addr - $remote_user [$time_local] "$request" '
- '$status $body_bytes_sent "$http_referer" '
- '"$http_user_agent" $http_x_forwarded_for';
- access_log /var/log/nginx/access.log access;
- server {
- listen 80;
- server_name www.test.com;
- location / {
- root /var/www/html ;
- index index.php index.htm index.html;
- proxy_redirect off;
- proxy_set_header Host $host;
- proxy_set_header X-Real-IP $remote_addr;
- proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
- proxy_pass http://backend;
- }
- location /nginx {
- access_log off;
- auth_basic "NginxStatus";
- }
- }
- }
#p#
三、安裝與配置keepalived
- # wgethttp://www.keepalived.org/software/keepalived-1.2.16.tar.gz
- # yum -y install libnl-devel
- # ./configure --prefix=/usr/local/keepalived
- # make && make install
- # cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
- # cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
- # cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
- # mkdir /etc/keepalived
- # cd /etc/keepalived/
- # vim keepalived.conf
以上步驟在兩臺keepalived機器上都需要進行
下面分別是兩節(jié)點的配置文件
節(jié)點一
- ! Configuration File for keepalived
- global_defs {
- notification_email {
- ganen2008@126.com #接收警報的email地址,可以添加多個
- }
- notification_email_from ganen201405@126.com #發(fā)件人地址
- smtp_connect_timeout 3 #超時時間
- smtp_server 127.0.0.1 #發(fā)送郵件的服務(wù)器
- router_id LVS_DEVEL #load balancer的標識ID,用于email警報
- }
- vrrp_instance VI_1 {
- state MASTER
- interface eth0
- virtual_router_id 51
- priority 100 # 權(quán)值要比 back 高
- advert_int 1
- authentication {
- auth_type PASS
- auth_pass 123456
- }
- virtual_ipaddress {
- 192.168.1.149 #vip的地址
- }
- }
- vrrp_instance VI_2 {
- state BACKUP
- interface eth0
- virtual_router_id 52
- priority 90
- advert_int 1
- authentication {
- auth_type PASS
- auth_pass 123456
- }
- virtual_ipaddress {
- 192.168.1.150
- }
- }
節(jié)點二
- ! Configuration File for keepalived
- global_defs {
- notification_email {
- ganen2008@126.com
- }
- notification_email_from ganen201405@126.com
- smtp_connect_timeout 3
- smtp_server 127.0.0.1
- router_id LVS_DEVEL
- }
- vrrp_instance VI_1 {
- state BACKUP
- interface eth1
- virtual_router_id 51
- priority 90
- advert_int 1
- authentication {
- auth_type PASS
- auth_pass 123456
- }
- virtual_ipaddress {
- 192.168.1.149
- }
- }
- vrrp_instance VI_2 {
- state MASTER
- interface eth1
- virtual_router_id 52
- priority 100
- advert_int 1
- authentication {
- auth_type PASS
- auth_pass 123456
- }
- virtual_ipaddress {
- 192.168.1.150
- }
- }
為兩節(jié)點添加nginx狀態(tài)監(jiān)控腳本，由于keepalived本身不能檢測到nginx的存活狀態(tài)，需要借助于第三方腳本來實現(xiàn)，下面是出自余洪春前輩的一個檢測腳本，在這里借用一下。
# vim /home/nginx_chk.sh
# chmod +x /home/nginx_chk.sh
- #!/bin/bash
- while :
- do
- nginxpid=`ps -C nginx --no-header | wc -l`
- if [ $nginxpid -eq 0 ];then
- /usr/local/nginx/sbin/nginx
- sleep 5
- nginxpid=`ps -C nginx --no-header | wc -l`
- echo $nginxpid
- if [ $nginxpid -eq 0 ];then
- /etc/init.d/keepalived stop
- fi
- fi
- sleep 5
- done
# chmod +x /home/nginx_chk.sh
后臺執(zhí)行該腳本
# nohup sh /home/nginx_chk.sh &
#p#
四、安裝web server
這里為了測試我直接使用yum安裝兩臺服務(wù)器上的nginx服務(wù)。
增加額外資源庫
- # yum -y install yum-priorities
- # rpm -Uvh http://mirrors.yun-idc.com/epel/6/x86_64/epel-release-6-8.noarch.rpm
- # rpm -Uvh http://rpms.famillecollet.com/enterprise/remi-release-6.rpm
- # yum -y install nginx
- # /etc/init.d/nginx start
五、啟動測試
首先測試兩臺WebServer的可用性
可以看到兩臺WebServer都運行正常
查看兩臺keepalived節(jié)點的IP情況
使用兩個VIP進行訪問
停掉一臺keepalived服務(wù)，查看IP
可以看到VIP已經(jīng)成功流轉(zhuǎn)到另一節(jié)點上，再使用VIP進行訪問，依然可以正常訪問。
這時重新啟動節(jié)點一上的keepalived服務(wù)，可以看到屬于節(jié)點一的VIP又重新回到了節(jié)點一上。
基本配置到這里就完成了。
責任編輯：藍雨淚
來源：
51CTO博客
