巧用 Ansible 實(shí)現(xiàn) MySQL 備份,運(yùn)維看過來
本文以容器形式部署了開源自動化運(yùn)維工具 Ansible,基于自帶的 MySQL 管理模塊編排了 playbook 配置文件,最終實(shí)現(xiàn) MySQL 數(shù)據(jù)庫備份的目標(biāo)。選擇容器而非直接安裝的部署形式,可以避免對系統(tǒng)環(huán)境的污染,使運(yùn)維工作開展更加高效和靈活。
MySQL 數(shù)據(jù)庫備份技術(shù)和相關(guān)方案已經(jīng)非常成熟,本文不做贅述和展開。在實(shí)際場景中,數(shù)據(jù)庫不可能脫離業(yè)務(wù)單獨(dú)存在;因此對于備份等運(yùn)維操作來說,應(yīng)當(dāng)在運(yùn)維平臺統(tǒng)一的調(diào)度下發(fā)起或?qū)嵤?,Ansible 作為近年來流行的自動化運(yùn)維工具,可以定位于運(yùn)維平臺的核心來使用。
Ansible簡介
關(guān)于 Ansible 的介紹,公開渠道可以查閱到大量資料,公眾號此前也有文章專門介紹,本文不再浪費(fèi)篇幅鋪開。簡單來說,Ansible 是一個基于Python語言開發(fā)的運(yùn)維工具,由于沒有客戶端依賴,在管理成本方面較其他產(chǎn)品有顯著優(yōu)勢;Ansible 基于模塊工作,通過系統(tǒng)自帶、客戶編寫、和第三方模塊,可以滿足各種管理任務(wù),本文使用 mysql_db 數(shù)據(jù)庫模塊實(shí)現(xiàn)了備份功能。
環(huán)境準(zhǔn)備
筆者對紅帽系統(tǒng)比較熟悉,原本想直接通過 yum 命令安裝 ansible,但是從實(shí)際工作角度出發(fā),一方面要維護(hù)非聯(lián)網(wǎng)環(huán)境中 yum 源,另一方面 ansible 需要連帶安裝大量依賴包,易對系統(tǒng)造成“污染”,因此并不推薦 yum 直接安裝。根據(jù) Ansible 的官方文檔,使用 mysql_db 模塊需要安裝 MySQL 客戶端和其他一些工具,這些對于操作系統(tǒng)本身也非必要。基于以上考慮,筆者最終采用構(gòu)建自定義 docker 鏡像的方式部署 Ansible。除了克服以上弊端外,鏡像制作完成后可以方便地移植到任意安裝 docker 的環(huán)境中,無需兼顧環(huán)境因素。本文使用 debian11 的官方鏡像作為底座和中科大的軟件源,Dockerfile 文件定義如下:
FROM debian:latest
RUN sed -i 's/deb.debian.org/mirrors.ustc.edu.cn/g' /etc/apt/sources.list \
&& apt-get update -y \
&& apt-get upgrade -y \
&& apt-get install ansible -y \
&& apt-get install sshpass -y \
&& apt-get install pip -y \
&& apt-get install libmysql++-dev -y \
&& apt-get install default-mysql-client -y \
&& apt-get autoclean
RUN pip3 install mysqlclient
RUN mkdir -p /etc/ansible/
CMD ["/bin/bash"]
使用 docker 編譯后,導(dǎo)出的鏡像約 900MB,安裝的 ansible 版本為 2.10.8。使用該鏡像部署的容器啟動后檢查輸出如下圖。
mysql_db模塊
該模塊用于實(shí)現(xiàn) MySQL 庫級別的管理,提供 CREATE、DROP、DUMP、IMPORT 四種功能。本文通過把 state 設(shè)置為 dump,實(shí)現(xiàn)調(diào)用 mysqldump 工具完成備份,mysql_db 模塊備份功能可能用到的主要輸入?yún)?shù)說明如下:
connect_timeout integer | The connection timeout when connecting to the MySQL server. Default: 30 |
dump_extra_args string added in 0.1.0 of community.mysql | Provide additional arguments for mysqldump. Used when state=dump only, ignored otherwise. |
encoding string | Encoding mode to use, examples include utf8 or latin1_swedish_ci, at creation of database, dump or importation of sql script. Default: “” |
force boolean added in 0.1.0 of community.mysql | Continue dump or import even if we get an SQL error. Used only when state is dump or import. Choices:
|
hex_blob boolean added in 0.1.0 of community.mysql | Dump binary columns using hexadecimal notation. Choices:
|
ignore_tables list / elements=string | A list of table names that will be ignored in the dump of the form database_name.table_name. Default: [] |
login_host string | Host running the database. In some cases for local connections the login_unix_socket=/path/to/mysqld/socket, that is usually /var/run/mysqld/mysqld.sock, needs to be used instead of login_host=localhost. Default: “l(fā)ocalhost” |
login_password string | The password used to authenticate with. |
login_port integer | Port of the MySQL server. Requires login_host be defined as other than localhost if login_port is used. Default: 3306 |
login_user string | The username used to authenticate with. |
master_data integer added in 0.1.0 of community.mysql | Option to dump a master replication server to produce a dump file that can be used to set up another server as a slave of the master. 0 to not include master data. 1 to generate a ‘CHANGE MASTER TO’ statement required on the slave to start the replication process. 2 to generate a commented ‘CHANGE MASTER TO’. Can be used when state=dump. Choices:
Default: 0 |
name aliases: db list / elements=string / required | Name of the database to add or remove. name=all may only be provided if state is dump or import. List of databases is provided with state=dump, state=present and state=absent. If name=all it works like –all-databases option for mysqldump (Added in 2.0). |
quick boolean | Option used for dumping large tables. Choices:
|
restrict_config_file boolean added in 0.1.0 of community.mysql | Read only passed config_file. When state is dump or import, by default the module passes config_file parameter using --defaults-extra-file command-line argument to mysql/mysqldump utilities under the hood that read named option file in addition to usual option files. If this behavior is undesirable, use yes to read only named option file. Choices:
|
single_transaction boolean | Execute the dump in a single transaction. Choices:
|
skip_lock_tables boolean added in 0.1.0 of community.mysql | Skip locking tables for read. Used when state=dump, ignored otherwise. Choices:
|
state string | The database state. Choices:
|
target path | Location, on the remote host, of the dump file to read from or write to. Uncompressed SQL files (.sql) as well as bzip2 (.bz2), gzip (.gz) and xz (Added in 2.0) compressed files are supported. |
playbook編排
playbook 是 Ansible 用于配置、部署、和管理被控節(jié)點(diǎn)的劇本,給被控節(jié)點(diǎn)列出的一系列 to-do-list。劇本在執(zhí)行過程中按照編排定義,執(zhí)行一個或多個 task,實(shí)現(xiàn)目標(biāo)主機(jī)完成指定任務(wù),達(dá)到預(yù)期的狀態(tài)。筆者編寫了一個簡單的 playbook,配置了一個task調(diào)用 mysql_db 模塊實(shí)現(xiàn)備份目標(biāo),需要注意的是 hosts 建議設(shè)定為127.0.0.1,表示 ansible 所在容器本身,yml 文件具體如下:
---
- hosts: 127.0.0.1
tasks:
- name: "mysql dump test"
mysql_db:
login_host: 192.168.43.51
login_user: root
login_password: ******
state: dump
name: test
target: /tmp/test_{{ ansible_date_time.date }}.gz
執(zhí)行該 playbook 的過程和結(jié)果如下圖所示:
PLAYBOOK: bak.yml *********************************************************
1 plays in bak.yml
PLAY [127.0.0.1] *********************************************************
TASK [Gathering Facts] *********************************************************
task path: /bak.yml:2
ok: [127.0.0.1]
META: ran handlers
TASK [mysql dump test] *********************************************************
task path: /bak.yml:4
redirecting (type: modules) ansible.builtin.mysql_db to community.mysql.mysql_db
[WARNING]: The value "********" (type int) was converted to "'********'" (type string). If this does not look like what you expect, quote the entire value to
ensure it does not change.
changed: [127.0.0.1] => {"changed": true, "db": "test", "db_list": ["test"], "executed_commands": ["/usr/bin/mysqldump --user=root --password=******** --host=192.168.43.51 --port=3306 test --quick | /bin/gzip > /tmp/test_2022-05-06.gz"], "msg": ""}
META: ran handlers
META: ran handlers
PLAY RECAP *********************************************************
127.0.0.1 : ok=2 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
總結(jié)和展望
本文使用容器形式實(shí)現(xiàn) Ansible 輕量化安裝部署的嘗試,旨在更好地發(fā)揮 Ansible 在運(yùn)維管理中的積極作用,Ansible 模塊化的屬性可以幫助運(yùn)維人員擺脫復(fù)雜的技術(shù)而更好地專注于運(yùn)維場景本身。筆者僅實(shí)現(xiàn)了 MySQL 備份一個場景,在企業(yè)級規(guī)模運(yùn)維管理中,要實(shí)現(xiàn)更復(fù)雜的運(yùn)維場景,做好模塊管理還是有必要部署 Ansible Tower。筆者也會持續(xù)開展 Ansible Tower對應(yīng)的開源產(chǎn)品AWX應(yīng)用研究。
作者簡介:曹杰,中國結(jié)算上海分公司高級經(jīng)理,從事系統(tǒng)運(yùn)維管理工作。