MySQL讀寫分離那些事,用Docker輕輕松松搞定
一個應(yīng)用系統(tǒng)是否高可用,整個系統(tǒng)的效率是否滿足預(yù)期,往往受到多方面的制約因素,例如:運行環(huán)境,網(wǎng)絡(luò)環(huán)境以及數(shù)據(jù)的吞吐量等;作為一般的大型應(yīng)用系統(tǒng)來說,我們想要提升整個系統(tǒng)的效率,我們大多數(shù)情況下會從我們的后端數(shù)據(jù)庫做優(yōu)化,從而提高我們整個系統(tǒng)的數(shù)據(jù)吞吐量,如果我們后端采用的是關(guān)系型數(shù)據(jù),我們可能會想到兩種解決方案,一種是更換非關(guān)系型數(shù)據(jù)庫,這種方案代價比較昂貴,它會涉及到數(shù)據(jù)的遷移,以及程序代碼的修改;另一種是通過數(shù)據(jù)庫集群的方式,來橫向和縱向擴展我們的數(shù)據(jù)庫,這種方案容易實現(xiàn)并且程序代碼修改量比較小;
根據(jù)MySQL的官方文檔介紹,MySQL支持讀寫分離的集群配置,并且MySQL提供兩種類型的讀寫分離數(shù)據(jù)復(fù)制類型;一種是二進制日志文件方式的數(shù)據(jù)復(fù)制,另一種是Global Transaction Identifiers (GTIDs)。
下面我們通過一個小實驗來看看MySQL數(shù)據(jù)基于二進制日志數(shù)據(jù)復(fù)制方式的主從集群是如何實現(xiàn)的。
OS環(huán)境:CentOS7
軟件環(huán)境:Docker(最新版),MySQL:latest鏡像
我們通過Docker容器快速的構(gòu)建兩個MySQL數(shù)據(jù)庫服務(wù)器
- docker pull mysql
- [root@dev01 ~]# docker images
- REPOSITORY TAG IMAGE ID CREATED SIZE
- mysql latest 62a9f311b99c 4 weeks ago 445MB
可以看到我們已經(jīng)拉取到最新的MySQL Docker容器,為了在本地環(huán)境中啟動兩個不同的MySQL Docker容器,我們需要通過修改容器的配置文件的方式來改變MySQL的配置;
首先,我們修改MySQL Master(主數(shù)據(jù)庫)的容器配置文件:
一,創(chuàng)建Master數(shù)據(jù)庫的配置文件
- vi master.cnf
- [mysqld]
- # master server id
- server-id = 1
- # bin log
- log_bin = mysql-master-bin
- server-id,在MySQL集群數(shù)據(jù)庫中,這個參數(shù)必須唯一
- log_bin,MySQL采用二進制日志文件復(fù)制的文件名
二,將該配置文件拷貝到MySQL Docker容器中
- docker run --name mysql -e MYSQL_ROOT_PASSWORD=root -d mysql
- docker cp master.cnf fdb98bbd52b6:/etc/mysql/conf.d
三,提交修改后的Docker容器
- docker commit -m "add master configure file" fdb98bbd52b6 mysql:master
- [root@dev01 ~]# docker images
- REPOSITORY TAG IMAGE ID CREATED SIZE
- mysql master 345465966cb5 3 hours ago 445MB
- commit -m是修改容器后提交的信息,類似Git提交;
- fdb98bbd52b6是剛才修改的容器;
- mysql:master是我們?yōu)樾薷暮蟮娜萜鞔蛏蟭ag標簽master
接下來我們修改MySQL Slave(從數(shù)據(jù)庫)的容器配置文件
一,創(chuàng)建Slave數(shù)據(jù)庫的配置文件
- vi slave.cnf
- [mysqld]
- # slave server id
- server-id = 2
- # bin log
- log_bin = mysql-slave-bin
- relay_log = mysql-relay-bin
- log_slave_updates = 1
- read_only = 1
- server-id為從數(shù)據(jù)庫的ID,該參數(shù)在MySQL集群中必須保持唯一性;
- log_bin如果slave為其它slave的master,必須設(shè)置bin_log,在這里我們暫時開啟;
- relay_log配置中繼日志
- log_slave_updates表示slave將復(fù)制事件寫進自己的二進制日志(后面會看到它的用處);
- read_only盡量使用read_only,它防止改變數(shù)據(jù)(除了特殊的線程);
二,將配置文件拷貝到容器中
- docker run --name mysql -e MYSQL_ROOT_PASSWORD=root -d mysql
- docker cp slave.cnf 8ee82abb2e91:/etc/mysql/conf.d
三,提交修改后的Docker容器
- docker commit -m "add slave configure file" 8ee82abb2e91 mysql:slave
- [root@dev01 ~]# docker images
- REPOSITORY TAG IMAGE ID CREATED SIZE
- mysql slave 3a53cd39ee45 4 hours ago 445MB
到此,所需要的兩個MySQL容器已經(jīng)修改完畢,并且保存在我們本地的容器倉庫中,接下來我們開始啟動剛才修改的兩個容器進行后續(xù)的配置
一,啟動Master數(shù)據(jù)庫
- docker run --name master -e MYSQL_ROOT_PASSWORD=root -d mysql:master
- --name master是我們?yōu)閱拥娜萜髅?
- mysql:master是我們剛才修改后并且提交到本地的Docker鏡像
- 默認數(shù)據(jù)庫root的密碼設(shè)置為root
二,啟動Slave數(shù)據(jù)庫
- docker run --link master:master --name slave -e MYSQL_ROOT_PASSWORD=root -d mysql:slave
- 為了master容器與slave容器的網(wǎng)絡(luò)互通,我們添加了--line選項來連接到我們剛才啟動的名為master容器;
- --name slave是我們?yōu)閱拥娜萜髅?
- mysql:slave是我們剛才修改后并且提交到本地的Docker鏡像;
- 默認數(shù)據(jù)庫root的密碼設(shè)置為root
三,進入master容器中通過mysql命令進入數(shù)據(jù)庫
- docker exec -it master /bin/bash
- mysql -u root -proot
四,在master數(shù)據(jù)庫中創(chuàng)建用于復(fù)制數(shù)據(jù)的賬號,并且給該賬號相應(yīng)的權(quán)限
- create user 'repl'@'%' identified by 'repl-pwd';
- grant replication slave on *.* to 'repl'@'%';
- flush privileges;
五,查看master數(shù)據(jù)庫的狀態(tài)
- mysql> show master status;
- +-------------------------+----------+--------------+------------------+-------------------+
- | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
- +-------------------------+----------+--------------+------------------+-------------------+
- | mysql-master-bin.000003 | 2743 | | | |
- +-------------------------+----------+--------------+------------------+-------------------+
- 1 row in set (0.00 sec)
六,進入slave容器中通過mysql命令進入數(shù)據(jù)庫
- docker exec -it slave /bin/bash
- mysql -u root -proot
七,配置slave,將master數(shù)據(jù)庫指向剛才配置好的master數(shù)據(jù)庫節(jié)點,并且啟動slave
- change master to master_host='master', master_user='repl', master_password='repl-pwd', master_log_file='mysql-master-bin.000003', master_log_pos=0;
- start slave;
八,查看slave數(shù)據(jù)庫狀態(tài)
- mysql> show slave status\G
- *************************** 1. row ***************************
- Slave_IO_State:
- Master_Host: master
- Master_User: repl
- Master_Port: 3306
- Connect_Retry: 60
- Master_Log_File: mysql-master-bin.000003
- Read_Master_Log_Pos: 4
- Relay_Log_File: mysql-relay-bin.000001
- Relay_Log_Pos: 4
- Relay_Master_Log_File: mysql-master-bin.000003
- Slave_IO_Running: No
- Slave_SQL_Running: No
- Replicate_Do_DB:
- Replicate_Ignore_DB:
- Replicate_Do_Table:
- Replicate_Ignore_Table:
- Replicate_Wild_Do_Table:
- Replicate_Wild_Ignore_Table:
- Last_Errno: 0
- Last_Error:
- Skip_Counter: 0
- Exec_Master_Log_Pos: 4
- Relay_Log_Space: 155
- Until_Condition: None
- Until_Log_File:
- Until_Log_Pos: 0
- Master_SSL_Allowed: No
- Master_SSL_CA_File:
- Master_SSL_CA_Path:
- Master_SSL_Cert:
- Master_SSL_Cipher:
- Master_SSL_Key:
- Seconds_Behind_Master: NULL
- Master_SSL_Verify_Server_Cert: No
- Last_IO_Errno: 2061
- Last_IO_Error: error connecting to master 'repl@master:3306' - retry-time: 60 retries: 1 message: Authentication plugin 'caching_sha2_password' reported error: Authentication requires secure connection.
- Last_SQL_Errno: 0
- Last_SQL_Error:
- Replicate_Ignore_Server_Ids:
- Master_Server_Id: 0
- Master_UUID:
- Master_Info_File: mysql.slave_master_info
- SQL_Delay: 0
- SQL_Remaining_Delay: NULL
- Slave_SQL_Running_State:
- Master_Retry_Count: 86400
- Master_Bind:
- Last_IO_Error_Timestamp: 190912 06:06:14
- Last_SQL_Error_Timestamp:
- Master_SSL_Crl:
- Master_SSL_Crlpath:
- Retrieved_Gtid_Set:
- Executed_Gtid_Set:
- Auto_Position: 0
- Replicate_Rewrite_DB:
- Channel_Name:
- Master_TLS_Version:
- Master_public_key_path:
- Get_master_public_key: 0
- Network_Namespace:
- 1 row in set (0.00 sec)
由于最新版的MySQL為了提升安全性更改了密碼校驗插件,在這里我們還是采用之前的密碼校驗插件,我們在master數(shù)據(jù)庫上用如下的命令來修改剛才在master數(shù)據(jù)庫創(chuàng)建的用戶
- alter user 'repl'@'%' identified by 'repl-pwd' password expire never;
- alter user 'repl'@'%' identified with mysql_native_password by 'repl-pwd';
- flush privileges;
九,再次查看slave狀態(tài)
- mysql> show slave status\G
- *************************** 1. row ***************************
- Slave_IO_State: Waiting for master to send event
- Master_Host: master
- Master_User: repl
- Master_Port: 3306
- Connect_Retry: 60
- Master_Log_File: mysql-master-bin.000003
- Read_Master_Log_Pos: 2743
- Relay_Log_File: mysql-relay-bin.000002
- Relay_Log_Pos: 2971
- Relay_Master_Log_File: mysql-master-bin.000003
- Slave_IO_Running: Yes
- Slave_SQL_Running: Yes
- Replicate_Do_DB:
- Replicate_Ignore_DB:
- Replicate_Do_Table:
- Replicate_Ignore_Table:
- Replicate_Wild_Do_Table:
- Replicate_Wild_Ignore_Table:
- Last_Errno: 0
- Last_Error:
- Skip_Counter: 0
- Exec_Master_Log_Pos: 2743
- Relay_Log_Space: 3179
- Until_Condition: None
- Until_Log_File:
- Until_Log_Pos: 0
- Master_SSL_Allowed: No
- Master_SSL_CA_File:
- Master_SSL_CA_Path:
- Master_SSL_Cert:
- Master_SSL_Cipher:
- Master_SSL_Key:
- Seconds_Behind_Master: 0
- Master_SSL_Verify_Server_Cert: No
- Last_IO_Errno: 0
- Last_IO_Error:
- Last_SQL_Errno: 0
- Last_SQL_Error:
- Replicate_Ignore_Server_Ids:
- Master_Server_Id: 1
- Master_UUID: f6e8062e-d521-11e9-9009-0242ac110008
- Master_Info_File: mysql.slave_master_info
- SQL_Delay: 0
- SQL_Remaining_Delay: NULL
- Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
- Master_Retry_Count: 86400
- Master_Bind:
- Last_IO_Error_Timestamp:
- Last_SQL_Error_Timestamp:
- Master_SSL_Crl:
- Master_SSL_Crlpath:
- Retrieved_Gtid_Set:
- Executed_Gtid_Set:
- Auto_Position: 0
- Replicate_Rewrite_DB:
- Channel_Name:
- Master_TLS_Version:
- Master_public_key_path:
- Get_master_public_key: 0
- Network_Namespace:
- 1 row in set (0.00 sec)
我們主要查看Slave_IO_Running和Slave_SQL_Running,表示我們的slave數(shù)據(jù)庫節(jié)點已經(jīng)成功的連接到了我們的master數(shù)據(jù)庫節(jié)點。
十,驗證,我們在master數(shù)據(jù)庫節(jié)點上創(chuàng)建一個空的數(shù)據(jù)庫
- mysql> create database data;
- Query OK, 1 row affected (0.10 sec)
- mysql> show databases;
- +--------------------+
- | Database |
- +--------------------+
- | data |
- | information_schema |
- | mysql |
- | performance_schema |
- | sys |
- +--------------------+
- 5 rows in set (0.01 sec)
十一,驗證slave,查看剛才在master數(shù)據(jù)庫節(jié)點上創(chuàng)建的數(shù)據(jù)庫是否同步到slave數(shù)據(jù)節(jié)點
- mysql> show databases;
- +--------------------+
- | Database |
- +--------------------+
- | data |
- | information_schema |
- | mysql |
- | performance_schema |
- | sys |
- +--------------------+
- 5 rows in set (0.01 sec)
可以看到,我們剛才在master數(shù)據(jù)庫節(jié)點上創(chuàng)建的數(shù)據(jù)庫已經(jīng)同步到我們的slave數(shù)據(jù)庫節(jié)點,master和slave的數(shù)據(jù)庫數(shù)據(jù)已經(jīng)保持一致。
以上實驗我們選取了兩個數(shù)據(jù)庫(一主一從)的方式,在實際的應(yīng)用中我們會根據(jù)我們的應(yīng)用場景,slave數(shù)據(jù)庫節(jié)點也有可能被選擇作為主數(shù)據(jù)庫,這個時候slave數(shù)據(jù)庫節(jié)點就有可能是1到N個。
參考:
https://dev.mysql.com/doc/refman/5.7/en/replication.html