OpenStack L3網(wǎng)絡(luò)QOS的實(shí)現(xiàn)思路
在L版的ReleaseNotes里面我們看到Neutron已經(jīng)支持了 ML2-QOS 特性, 這個特性可以讓我們針對L2-Port和L2-Net做流量QOS設(shè)置, 這是一個非常實(shí)用的特性,在之前的版本中,對于虛機(jī)網(wǎng)絡(luò)流量QOS,我們只能通過Flavor來去設(shè)置QOS, Flavor-QOS不是很靈活,這會讓我去維護(hù)很多個Flavor來保證不同的QOS需求。
但是ML2-QOS只能做二層網(wǎng)絡(luò)的流量限制,對于floating-IP級別的并不能生效,而且ML2-QOS依托于OpenVswitch本身的特性, 且OpenVswitch的QOS只能設(shè)置單向流量(虛機(jī)上行流量)。
我們還是需要一個L3-QOS,來實(shí)現(xiàn)針對floating-IP和router設(shè)置QOS,且能夠設(shè)置上行和下行流量。
TC是基礎(chǔ)
在Linux系統(tǒng)上,無論去使用怎樣的工具,本質(zhì)上都要使用TC來做流量控制。 如上面所說,針對floating-IP和router的QOS設(shè)置,我們必須讓TC規(guī)則作用在對應(yīng)的Net-namespace上:
使用HTB來控制虛機(jī)的上行流量,使用Ingress來控制虛機(jī)的下行流量:
此處需要注意的是,TC的版本不能太低,否則TC不能支持在namespace下設(shè)置規(guī)則,如果你是ubuntu系統(tǒng), 建議至少使用這個版本IProute2_4.1.1-1ubuntu1_amd64。
實(shí)現(xiàn)原理請參考這個腳本:
- #!/bin/bash
- #
- # kb or k: Kilobytes
- # mb or m: Megabytes
- # mbit: Megabits
- # kbit: Kilobits
- # To get the byte figure from bits, divide the number by 8 bit
- #
- # ex: tc_htb.sh restart qg-f5f0ed64-84 qrouter-d5b5b595-9ae2-49e9-8fd9-6ddbf6195702
- #
- TC=/sbin/tc
- if [ -n "$3" ]; then
- TC="/sbin/tc -n $3" #tc utility, IProute2-ss150706
- fi
- IF=$2 # Interface
- UPLD=1mbit # VM UPLOAD Limit
- DEF_UPLD=2mbit # VM UPLOAD Limit
- DNLD=10mbit # VM DOWNLOAD Limit
- DEF_DNLD=20mbit # VM DOWNLOAD Limit
- IP=192.168.252.168 # VM IP
- GW_IP=192.168.252.141 # Router GW IP
- HTB_U32="$TC filter add dev $IF protocol all parent 1: prio 1 u32"
- INGRESS_U32="$TC filter add dev $IF protocol all parent ffff:"
- start() {
- # UPLOAD TRAFFIC
- $TC qdisc add dev $IF root handle 1: htb default 1000
- $TC class add dev $IF parent 1: classid 1:1000 htb rate $DEF_UPLD #burst 100kb
- $TC qdisc add dev $IF parent 1:1000 handle 1001: sfq perturb 10
- $TC class add dev $IF parent 1: classid 1:1 htb rate $UPLD #burst 100kb
- $TC qdisc add dev $IF parent 1:1 handle 2: sfq perturb 10
- $HTB_U32 match IP src $IP/32 flowid 1:1
- # DOWNLOAD TRAFFIC
- $TC qdisc add dev $IF ingress
- $INGRESS_U32 prio 1 u32 match IP dst $GW_IP/32 police rate $DEF_DNLD burst 512kb mtu 64kb drop flowid :1
- $INGRESS_U32 prio 2 u32 match IP dst $IP/32 police rate $DNLD burst 256kb mtu 64kb drop flowid :1
- }
- rm() {
- $TC qdisc del dev $IF parent 1:1
- $TC filter del dev $IF protocol all parent 1: prio 1
- $TC class del dev $IF parent 1: classid 1:1
- $TC filter del dev $IF protocol all parent ffff: prio 2
- }
- stop() {
- $TC qdisc del dev $IF root
- $TC qdisc del dev $IF ingress
- }
- restart() {
- stop
- sleep 1
- start
- }
- show() {
- echo '=================================='
- $TC -s -d -p qdisc show dev $IF
- echo '=================================='
- echo -e '\n'
- echo '=================================='
- $TC class show dev $IF
- echo '=================================='
- echo -e '\n'
- echo '=================================='
- $TC -s -d -p filter show dev $IF
- echo '----------------------------------'
- $TC filter show dev $IF parent ffff:
- echo '=================================='
- }
- case "$1" in
- start)
- echo -n "Starting bandwidth shaping: "
- start
- echo "done"
- ;;
- stop)
- echo -n "Stopping bandwidth shaping: "
- stop
- echo "done"
- ;;
- restart)
- echo -n "Restarting bandwidth shaping: "
- restart
- echo "done"
- ;;
- rm)
- echo -n "Rm bandwidth shaping: "
- rm
- echo "done"
- ;;
- show)
- echo "Bandwidth shaping status for $IF:"
- show
- echo ""
- ;;
- *)
- pwd=$(pwd)
- echo "Usage: $(/usr/bin/dirname $pwd)/tc.bash {start|stop|restart|show}"
- ;;
- esac
- exit 0
將上面的實(shí)現(xiàn)原理整合到自己維護(hù)的Neutron源碼中。
使用案例
除了在底層支持L3-QOS外,還需要在Neutron-api層面支持設(shè)置具體屬性值, 比如創(chuàng)建一個具有帶寬限制的router或者floatingIP:
- $ Neutron router-create aaaa --ha True --ingress_rate 14mbit --ingress_burst 512kb --egress_rate 12mbit --egress_burst 512kb
- Created a new router:
- +-----------------------+--------------------------------------------------------------------------------------------------------+
- | Field | Value |
- +-----------------------+--------------------------------------------------------------------------------------------------------+
- | admin_state_up | True |
- | bandwidth | {"ingress_rate": "14mbit", "egress_rate": "12mbit", "egress_burst": "512kb", "ingress_burst": "512kb"} |
- | distributed | False |
- | external_gateway_info | |
- | ha | True |
- | id | d2e8813a-6619-4784-a08a-2a2725fe2fb5 |
- | name | aaaa |
- | routes | |
- | status | ACTIVE |
- | tenant_id | 5e30ecc3c3434f24aa56801467ef1688 |
- +-----------------------+--------------------------------------------------------------------------------------------------------+
- $ Neutron floatingIP-create --egress_rate 5mbit --egress_burst 150kb
- Created a new floatingIP:
- +---------------------+---------------------------------------------------+
- | Field | Value |
- +---------------------+---------------------------------------------------+
- | bandwidth | {"egress_rate": "5mbit", "egress_burst": "150kb"} |
- | fixed_IP_address | |
- | floating_IP_address | 192.168.252.161 |
- | floating_Network_id | 9946adcc-041e-4c0f-91b6-96a9fb6254c0 |
- | id | 432f7dd6-b171-4de1-8e43-8069c42ee5a5 |
- | Port_id | |
- | router_id | |
- | status | DOWN |
- | tenant_id | 5e30ecc3c3434f24aa56801467ef1688 |
- +---------------------+---------------------------------------------------+
修改已有router的帶寬規(guī)則,并讓其生效:
- $ Neutron router-update --admin-state-up False --ingress_rate 8mbit --ingress_burst 256kb
- $ Neutron router-update --admin-state-up True
注意這個過程中,連到這個router的所有機(jī)器都會斷網(wǎng)
修改已有floatingIP的帶寬規(guī)則,并讓其生效:
- $ Neutron floatingIP-disassociate --ingress_rate 7mbit --ingress_burst 256kb
- $ Neutron floatingIP-associate
注意這個過程中,floatingIP是從解綁到綁定
本文源自O(shè)penStack中國社區(qū)與云頭條、ZDNet\Solidot、CSDN、51CTO、infoQ、科技茱比、特大號、IDC圈等Top合作媒體聯(lián)合舉辦的“我要直播奧斯汀”(http://www.openstack.cn/?p=4396)活動
保護(hù)期(2016.1.19-2016.1.26)內(nèi)非合作媒體不得轉(zhuǎn)載。