Kong和Konga在K8s部署真的很難嗎？
本文轉(zhuǎn)載自微信公眾號(hào)「運(yùn)維開發(fā)故事」，作者小姜。轉(zhuǎn)載本文請(qǐng)聯(lián)系運(yùn)維開發(fā)故事公眾號(hào)。
背景
隨著k8s的成熟，在k8s上使用ingress-nginx、traefik、apisix等各類ingress的人也越來(lái)越多。ingress-kong也可以作為k8s的ingress使用，當(dāng)然kong也可以單獨(dú)作為微服務(wù)的gateway網(wǎng)關(guān)來(lái)使用。
如果kong是作為ingress來(lái)使用，那么需要使用kong鏡像來(lái)部署一個(gè)ingress-kong和一個(gè)kong/kubernetes-ingress-controller鏡像來(lái)部署一個(gè)kong的ingress-controller。還需要部署sa賬號(hào)和rbac，還有CRD資源。
如果kong只是作為一個(gè)gateway來(lái)替換springcloud中的gateway來(lái)使用，那么只需要部署sa賬號(hào)和rbac，然后用kong鏡像部署一個(gè)pod即可當(dāng)做gateway使用，crd也就不需要了。
kong和konga支持MySQL、MongoDB、PostgresSQL三種數(shù)據(jù)庫(kù)。而我這里選擇的是PostgresSQL。因?yàn)楣径际莂ll in docker，all in kubernetes，因此這里也就選擇了在k8s中部署kong和konga。下面記錄了一下部署kong和konga過(guò)程中遇到的一些報(bào)錯(cuò)和排查解決方案思路手段。
部署konga
- $ cat kong-ui-pre.yaml
- ---
- apiVersion: apps/v1
- kind: Deployment
- metadata:
- labels:
- appName: konga-ui-aggre-sit
- appEnv: sit
- name: konga-ui-aggre-sit
- namespace: kong
- spec:
- replicas: 1
- selector:
- matchLabels:
- appName: konga-ui-aggre-sit
- appEnv: sit
- template:
- metadata:
- labels:
- appName: konga-ui-aggre-sit
- appEnv: sit
- spec:
- imagePullSecrets:
- - name: registry-auth
- containers:
- - env:
- - name: NODE_ENV
- value: "production"
- - name: DB_ADAPTER
- value: "postgres"
- - name: DB_HOST
- value: "你自己的pgsql地址"
- - name: DB_PORT
- value: "你自己的pgsql端口"
- - name: DB_USER
- value: "kong"
- - name: DB_PASSWORD
- value: "你自己的pgsql密碼"
- - name: DB_DATABASE
- value: "konga"
- - name: TOKEN_SECRET
- value: "自己生成隨機(jī)字符串"
- - name: NO_AUTH
- value: "false"
- - name: NODE_TLS_REJECT_UNAUTHORIZED
- value: "0"
- image: registry.ayunw.cn/kube-system/pantsel/konga:0.14.9
- imagePullPolicy: IfNotPresent
- livenessProbe:
- failureThreshold: 3
- httpGet:
- path: /
- port: 1337
- scheme: HTTP
- initialDelaySeconds: 5
- periodSeconds: 10
- successThreshold: 1
- timeoutSeconds: 1
- name: konga-ui-aggre-sit
- ports:
- - containerPort: 1337
- name: kong-ui
- protocol: TCP
- readinessProbe:
- failureThreshold: 3
- httpGet:
- path: /
- port: 1337
- scheme: HTTP
- initialDelaySeconds: 5
- periodSeconds: 10
- successThreshold: 1
- timeoutSeconds: 1
- serviceAccountName: kong-serviceaccount
我部署kong和konga的時(shí)候使用的是pgsql，根據(jù)GitHub上的文檔上說(shuō)明得知，在使用NODE_ENV為production的時(shí)候需要手動(dòng)去初始化pgsql?？戳艘幌耴ong官網(wǎng)的文檔來(lái)初始化，官網(wǎng)文檔：https://docs.konghq.com/install/docker/
初始化pgsql報(bào)錯(cuò)
- $ docker run --rm --network=kong-net -e "KONG_DATABASE=postgres" \
- > -e "KONG_PG_HOST=pgsql地址" \
- > -e "KONG_PG_USER=konga" \
- > -e "KONG_PG_PASSWORD=pgsql密碼" \
- > -e "KONG_PG_DATABASE=konga" \
- > -e "KONG_PG_PORT=3306" \
- > registry.ayunw.cn/kong/kong-custom:382-ffaf4d50 kong migrations up
- Unable to find image 'registry.ayunw.cn/kong/kong-custom:382-ffaf4d50' locally
- 382-ffaf4d50: Pulling from kong/kong-custom
- 0a6724ff3fcd: Pull complete
- 274efec6805c: Pull complete
- 4bb58967a4ce: Pull complete
- 3f59fb9af44b: Pull complete
- 7e3ec18b9226: Pull complete
- ce8acfac03f7: Pull complete
- Digest: sha256:453dea194d4e39275ea771d062262f0868e29120fc529702dea10278677413c2
- Status: Downloaded newer image for registry.ayunw.cn/kong/kong-custom:382-ffaf4d50
- Error: [PostgreSQL error] failed to retrieve PostgreSQL server_version_num: temporary failure in name resolution
- Run with --v (verbose) or --vv (debug) for more details
想了一下，他這個(gè)完全是docker部署kong而不是k8s中部署kong，因此他會(huì)需要?jiǎng)?chuàng)建一個(gè)docker的network為kong-net，然后保證所有kong相關(guān)的服務(wù)都跑在同一個(gè)網(wǎng)絡(luò)中。那我這里是部署在k8s中，且pgsql也是買的云的服務(wù)，所以我這里其實(shí)是不應(yīng)該這樣子來(lái)進(jìn)行初始化的。
然后上github，找到https://github.com/pantsel/konga/tree/master文檔上的Production Docker Image步驟中的Prepare the database來(lái)操作。
但是里面可能有一個(gè)參數(shù)解釋的不是很清楚，我自己一開始也是比較模糊的，就是https://github.com/pantsel/konga上Production Docker Image步驟下的Prepare the database步驟中的connection-uri參數(shù)，你會(huì)發(fā)現(xiàn)他似乎沒(méi)給出connection-uri的這個(gè)uri的例子是什么樣子的。但是其實(shí)你會(huì)發(fā)現(xiàn)在這個(gè)github中的Production步驟有寫一條url和這個(gè)比較類似的，告訴你在使用MySQL或者PostgresSQL的時(shí)候應(yīng)該怎么初始化，命令是：node ./bin/konga.js prepare --adapter postgres --uri postgresql://localhost:5432/konga。但是注意，這個(gè)命令他應(yīng)該不是在你將konga部署在k8s中來(lái)使用的。但是這里就有一條url：postgresql://localhost:5432/konga，可以用來(lái)參考，那么也就是說(shuō)差不多就是這個(gè)樣子的url了，但是應(yīng)該還需要帶上用戶名和密碼。
其實(shí)這里我還犯了一個(gè)錯(cuò)誤，就是其實(shí)一開始我初始化的pgsql命令中pgsql的url寫錯(cuò)了，看了https://github.com/pantsel/konga/tree/master中的文檔后，手敲命令，將postgresql://localhost:5432/konga敲成了postgre://localhost:5432/konga，所以初始化的時(shí)候也報(bào)錯(cuò)了。
更改后重新初始化
- $ docker run registry.ayunw.cn/kube-system/pantsel/konga:0.14.9 -c prepare -a postgres -u postgresql://konga:pgsql密碼@pgsql地址:pgsql端口/konga
- debug: Preparing database...
- Using postgres DB Adapter.
- Failed to connect to DB Error: getaddrinfo EAI_AGAIN konga
- at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:64:26) {
- errno: 'EAI_AGAIN',
- code: 'EAI_AGAIN',
- syscall: 'getaddrinfo',
- hostname: 'konga'
- }
然后想到了因?yàn)?ldquo;#”導(dǎo)致的問(wèn)題。甚至嘗試了將密碼用單引號(hào)或者雙引號(hào)括起來(lái)嘗試，仍然報(bào)錯(cuò)，然后將整個(gè)連接pgsql的url全部用單引號(hào)或者雙引號(hào)括起來(lái)，也都是沒(méi)有用。再查了一下google發(fā)現(xiàn)需要將“#”符號(hào)改成%23。
再次初始化
- $ docker run registry.ayunw.cn/kube-system/pantsel/konga:0.14.9 -c prepare -a postgres -u postgresql://konga:pgsql密碼@pgsql地址:pgsql端口/konga
- debug: Preparing database...
- Using postgres DB Adapter.
- Database exists. Continue...
- debug: Hook:api_health_checks:process() called
- debug: Hook:health_checks:process() called
- debug: Hook:start-scheduled-snapshots:process() called
- debug: Hook:upstream_health_checks:process() called
- debug: Hook:user_events_hook:process() called
- debug: Seeding User...
- debug: User seed planted
- debug: Seeding Kongnode...
- debug: Kongnode seed planted
- debug: Seeding Emailtransport...
- debug: Emailtransport seed planted
- debug: Database migrations completed!
初始化成功
總結(jié)部署konga遇到的問(wèn)題
1、文檔中寫的connection-uri的話必須不能寫錯(cuò)，postgresql不要寫成postgres
2、密碼中有符號(hào)“#”要改成%23
部署kong
- ---
- # Source: kong-custom-pre-master/templates/deployment.yaml
- apiVersion: apps/v1
- kind: Deployment
- metadata:
- name: kong-custom-pre-master
- namespace: kong
- labels:
- appEnv: pre
- appName: kong-custom
- appGroup: kong
- spec:
- replicas: 1
- progressDeadlineSeconds: 1800
- minReadySeconds: 5
- strategy:
- type: RollingUpdate
- rollingUpdate:
- maxUnavailable: 50%
- maxSurge: 50%
- selector:
- matchLabels:
- appEnv: pre
- appName: kong-custom
- appGroup: kong
- template:
- metadata:
- labels:
- appEnv: pre
- appName: kong-custom
- appGroup: kong
- spec:
- dnsPolicy: ClusterFirst
- terminationGracePeriodSeconds: 10
- serviceAccountName: kong-serviceaccount
- imagePullSecrets:
- - name: registry-auth-kong-pro
- initContainers:
- - name: wait-for-migrations
- image: "registry.ayunw.cn/kong/kong-custom:398-c44f9085"
- command:
- - /bin/sh
- - -c
- - while true; do kong migrations bootstrap; if [[ 0 -eq 0 ]]; then exit 0; fi; sleep 2; done;
- env:
- # 注意這里KONG_DATABASE的value是錯(cuò)誤的，應(yīng)該將Kong改成postgres
- - name: KONG_DATABASE
- value: "kong"
- - name: KONG_PG_USER
- value: "kong"
- - name: KONG_PG_PORT
- value: "pgsql端口"
- - name: KONG_PG_PASSWORD
- value: "pgsql密碼"
- - name: KONG_PG_HOST
- value: "pgsql地址"
- containers:
- - name: kong-custom-pre-master
- image: "registry.ayunw.cn/kong/kong-custom:398-c44f9085"
- ports:
- - name: proxy
- containerPort: 8000
- protocol: TCP
- - name: proxy-ssl
- containerPort: 9443
- protocol: TCP
- - name: metrics
- containerPort: 8100
- protocol: TCP
- - name: admin-url
- containerPort: 8444
- protocol: TCP
- resources:
- limits:
- cpu: "5000m"
- memory: "1024Mi"
- requests:
- cpu: "100m"
- memory: "512Mi"
- lifecycle:
- preStop:
- exec:
- command:
- - /bin/sh
- - -c
- - kong quit
- env:
- - name: "KONG_PROXY_LISTEN"
- value: "0.0.0.0:8000, 0.0.0.0:9443 ssl http2"
- - name: "KONG_PORT_MAPS"
- value: "80:8000, 443:8443"
- - name: "KONG_ADMIN_LISTEN"
- value: "0.0.0.0:8444 ssl"
- - name: "KONG_STATUS_LISTEN"
- value: "0.0.0.0:8100"
- - name: "KONG_NGINX_WORKER_PROCESSES"
- value: "2"
- - name: "KONG_ADMIN_ACCESS_LOG"
- value: "/dev/stdout"
- - name: "KONG_ADMIN_ERROR_LOG"
- value: "/dev/stderr"
- - name: "KONG_PROXY_ERROR_LOG"
- value: "/dev/stderr"
- # 注意這里KONG_DATABASE的value是錯(cuò)誤的，應(yīng)該將Kong改成postgres
- - name: KONG_DATABASE
- value: "kong"
- - name: KONG_PG_USER
- value: "kong"
- - name: KONG_PG_PORT
- value: "pgsql端口"
- - name: KONG_PG_PASSWORD
- value: "pgsql密碼"
- - name: KONG_PG_HOST
- value: "pgsql地址"
查看kong的日志報(bào)錯(cuò)
- $ kubectl logs -f --tail=20 -n kong kong-custom-sit-9c5cf7b69-4q29l
- stack traceback:
- [C]: in function 'error'
- /usr/local/share/lua/5.1/kong/cmd/utils/migrations.lua:16: in function 'check_state'
- /usr/local/share/lua/5.1/kong/init.lua:455: in function 'init'
- init_by_lua:3: in main chunk
- nginx: [error] init_by_lua error: /usr/local/share/lua/5.1/kong/cmd/utils/migrations.lua:16: Database needs bootstrapping or is older than Kong 1.0.
- To start a new installation from scratch, run 'kong migrations bootstrap'.
- To migrate from a version older than 1.0, migrated to Kong 1.5.0 first.
- If you still have 'apis' entities, you can convert them to Routes and Services
- using the 'kong migrations migrate-apis' command in Kong 1.5.0.
- stack traceback:
- [C]: in function 'error'
- /usr/local/share/lua/5.1/kong/cmd/utils/migrations.lua:16: in function 'check_state'
- /usr/local/share/lua/5.1/kong/init.lua:455: in function 'init'
- init_by_lua:3: in main chunk
很明顯是因?yàn)闆](méi)有執(zhí)行kong migrations的命令。但是明明在deployment.yaml中的initContainers中已經(jīng)寫了，為何沒(méi)執(zhí)行?
原因
是因?yàn)閗ong的deployment.yaml中的initContainers下的env環(huán)境變量中KONG_DATABASE寫錯(cuò)了，我寫成了kong。而根據(jù)github：https://github.com/pantsel/konga/tree/master上README.md文檔中的Environment variables中所知，KONG_DATABASE這個(gè)變量是要指明使用postgres或者M(jìn)ySQL或者off三個(gè)選項(xiàng)其中的一個(gè)。因?yàn)槲矣玫氖莗gsql，因此這里需要指定postgres，那么初始化就回去pgsql中創(chuàng)建庫(kù)、表、創(chuàng)建數(shù)據(jù)等操作，這樣才會(huì)成功。
更改后，kong啟動(dòng)正常。
寫在最后
我在konga的github上沒(méi)找到manifests直接可以用的yaml清單文件，只找到了chart包，因此我是渲染出來(lái)以后然后根據(jù)自己的實(shí)際環(huán)境進(jìn)行了更改。初次使用難免會(huì)遇到雜七雜八的問(wèn)題。其實(shí)也是由于我個(gè)人的不細(xì)心，沒(méi)仔細(xì)多看幾遍文檔導(dǎo)致的。有讀者如果找到了konga的manifests清單文件也可下方留言交流。也提醒各位小伙伴們，平常也需要注意一下習(xí)慣，減少和我犯同樣的錯(cuò)誤!
