一 前言
随着发展 k8s对于有状态应用的适应已经越来越好 下面就针对mongo db的部署为大家分享下k8s sts控制器的使用
二 部署
2.1 环境准备
数据库这种核心组件 首先需要准备持久化的数据卷 这里就使用sc动态存储来作为底层存储
yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: nfs-mongo
provisioner: nfs-provisioner-mongo
reclaimPolicy: Retain
volumeBindingMode: Immediate
创建ns
yaml
apiVersion: v1
kind: Namespace
metadata:
name: mongo
配置cm
yaml
kind: ConfigMap
apiVersion: v1
metadata:
name: mongo-keyfile
namespace: mongo
annotations:
kubesphere.io/creator: admin
data:
key.txt: |-
nX24aaT7/WOQKOfchxP66kulNNdjSn5LpsE/RbDL7gwMIqZ424WQVCOJzRV5YMw9
fQ0pz591I+U+G2/iP8PEPF6po1/rLKi1LotoKV+ubh336N/TCMzi86OH3aibNVyC
ydhbw6hTrtYxaZatxyKHZ7C8K39su05T0cDh1iqCYDBEkxUh5zYUafwfxa5GC5Ar
+NGiDhrZJXj+fjGBhUOBA8yl0fG7V5lx56PIJ7u9M+EaFoJcoYembSZ40A/JRcmO
wV76/5sng1HQ0A016xh3WPFrUkSrie3RqmpjTP/gx7sej3fiheOxNmjjFYcR4Fof
Y3VH+MXs0qJm4HIcJUIEnYInunFlsbpyrSFu77sgfXuL/zYkozK3CPaK5+kZZCfY
FqsTxo9xWI9D8QVpXlSU7Pbj4g8DatHBjAxQWgDgEFxABhypILz56RsDybQ8aNtX
OWuEhJJUvjAMEYutTOSM80ATYSFh+XbB19n5SthDfcz9RvJ+XMdn6biBZwS46vqb
oKWaM2U7lpNC2ZvhEAfGYsHwPfs98ak4O1n1iYQVeohC+TkQdrJQk5sppMp2bu5b
hcG0qrYOWgjXS3EayMMAwYKBAcyb0sCugFIY4KeCl3RMzJpnSVL916LwXuOaHRTb
SkFrbgEapveUaGB+NCARJq/NUi2sA16gpGVoLVYctJCHnJVNhTQ1bH32I7PULiL8
6fae3gLs6yTbC8Isdn4CqkJP4yB7ae8z9VOYpTouP7YblJaGfcQ/pGD0xWVCj4BL
IXhsEygotZj+3RySpFFwedX/Ku5JDMlA7FDbwNguUBPUtO6a8osiw5jSb8wplFXZ
ei8LgkyGx6onVrIgzuQv0CDm9tUpdgNuf8d7CYYWLyZBpTuBFrbJdnWQ1ViigRq5
wlKzb2yL76KxiIch4JwL61XTC/6+k5K2opGimAP8Ebft9tfB0M38cFfrdD7PJCL5
x7c9cfDxO4O56DNzpPimyW6pJ6nw
2.2 配置sts
yaml
kind: StatefulSet
apiVersion: apps/v1
metadata:
name: mongo
namespace: mongo
annotations:
kubesphere.io/creator: admin
spec:
replicas: 3
selector:
matchLabels:
environment: test
role: mongo
template:
metadata:
creationTimestamp: null
labels:
environment: test
role: mongo
annotations:
kubesphere.io/restartedAt: '2022-03-05T07:52:33.099Z'
spec:
volumes:
- name: mongo-keyfile
configMap:
name: mongo-keyfile
items:
- key: key.txt
path: key.txt
defaultMode: 420
initContainers:
- name: fix-permissions
image: harbor.huhuhahei.cn/huhuhahei/busybox
command:
- sh
- '-c'
- >-
cp /data/key.txt /data/db/key && chown 999:999 /data/db/key &&
chmod 400 /data/db/key && ls -l /data/
resources: {}
volumeMounts:
- name: mongo
mountPath: /data/db
- name: mongo-keyfile
mountPath: /data/key.txt
subPath: key.txt
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: Always
securityContext:
privileged: true
containers:
- name: mongo
image: harbor.huhuhahei.cn/huhuhahei/mongo:4.2.8
command:
- docker-entrypoint.sh
- '--replSet'
- rs0
- '--bind_ip'
- 0.0.0.0
- '--auth'
- '--clusterAuthMode'
- keyFile
- '--keyFile'
- /data/db/key
ports:
- containerPort: 27017
protocol: TCP
env:
- name: MONGO_INITDB_ROOT_USERNAME
value: root
- name: MONGO_INITDB_ROOT_PASSWORD
value: dSJN52PuSqn
resources: {}
volumeMounts:
- name: mongo
mountPath: /data/db
- name: mongo-keyfile
mountPath: /data/key.txt
subPath: key.txt
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: Always
- name: mongo-sidecar
image: harbor.huhuhahei.cn/huhuhahei/k8s-mongo-sidecar
env:
- name: KUBERNETES_POD_LABELS
value: 'role=mongo,environment=test'
- name: KUBERNETES_NAMESPACE
value: mongo
- name: MONGO_USERNAME
value: root
- name: MONGO_PASSWORD
value: dSJN52PuSqn
- name: MONGO_DATABASE
value: admin
- name: KUBERNETES_SERVICE_NAME
value: mongo
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: IfNotPresent
restartPolicy: Always
terminationGracePeriodSeconds: 10
dnsPolicy: ClusterFirst
securityContext: {}
schedulerName: default-scheduler
volumeClaimTemplates:
- kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: mongo
creationTimestamp: null
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
storageClassName: nfs-mongo
volumeMode: Filesystem
status:
phase: Pending
serviceName: mongo
podManagementPolicy: OrderedReady
updateStrategy:
type: RollingUpdate
rollingUpdate:
partition: 0
revisionHistoryLimit: 10
因为mongo-sidecar需要监控mongo状态所以需要创建ClusterRoleBinding
yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: mongo-default-view
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: view
subjects:
- kind: ServiceAccount
name: default
namespace: default
2.3 配置svc
yaml
kind: Service
apiVersion: v1
metadata:
name: mongo
namespace: mongo
labels:
name: mongo
annotations:
kubesphere.io/creator: admin
spec:
ports:
- protocol: TCP
port: 27017
targetPort: 27017
selector:
role: mongo
clusterIP: None
type: ClusterIP
sessionAffinity: None
2.4 交付k8s
yaml
kubectl apply -f .
三 验证
3.1 连接测试
bash
# mongo -uroot -p dSJN52PuSqn
MongoDB shell version v4.2.8
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("8f142a74-c99c-48df-8c37-5107f846e4a5") }
MongoDB server version: 4.2.8
Server has startup warnings:
2022-03-05T07:53:05.774+0000 I CONTROL [initandlisten]
2022-03-05T07:53:05.774+0000 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2022-03-05T07:53:05.774+0000 I CONTROL [initandlisten] ** We suggest setting it to 'never'
2022-03-05T07:53:05.774+0000 I CONTROL [initandlisten]
2022-03-05T07:53:05.774+0000 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2022-03-05T07:53:05.774+0000 I CONTROL [initandlisten] ** We suggest setting it to 'never'
2022-03-05T07:53:05.774+0000 I CONTROL [initandlisten]
---
Enable MongoDB's free cloud-based monitoring service, which will then receive and display
metrics about your deployment (disk utilization, CPU, operation statistics, etc).
The monitoring data will be available on a MongoDB website with a unique URL accessible to you
and anyone you share the URL with. MongoDB may use this information to make product
improvements and to suggest MongoDB products and deployment options to you.
To enable free monitoring, run the following command: db.enableFreeMonitoring()
To permanently disable this reminder, run the following command: db.disableFreeMonitoring()
---
rs0:PRIMARY> rs.status
function() {
return db._adminCommand("replSetGetStatus");
}
rs0:PRIMARY> rs.status()
{
"set" : "rs0",
"date" : ISODate("2022-03-05T08:31:10.117Z"),
"myState" : 1,
"term" : NumberLong(5),
"syncingTo" : "",
"syncSourceHost" : "",
"syncSourceId" : -1,
"heartbeatIntervalMillis" : NumberLong(2000),
"majorityVoteCount" : 2,
"writeMajorityCount" : 2,
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1646469067, 1),
"t" : NumberLong(5)
},
"lastCommittedWallTime" : ISODate("2022-03-05T08:31:07.713Z"),
"readConcernMajorityOpTime" : {
"ts" : Timestamp(1646469067, 1),
"t" : NumberLong(5)
},
"readConcernMajorityWallTime" : ISODate("2022-03-05T08:31:07.713Z"),
"appliedOpTime" : {
"ts" : Timestamp(1646469067, 1),
"t" : NumberLong(5)
},
"durableOpTime" : {
"ts" : Timestamp(1646469067, 1),
"t" : NumberLong(5)
},
"lastAppliedWallTime" : ISODate("2022-03-05T08:31:07.713Z"),
"lastDurableWallTime" : ISODate("2022-03-05T08:31:07.713Z")
},
"lastStableRecoveryTimestamp" : Timestamp(1646469057, 1),
"lastStableCheckpointTimestamp" : Timestamp(1646469057, 1),
"electionCandidateMetrics" : {
"lastElectionReason" : "electionTimeout",
"lastElectionDate" : ISODate("2022-03-05T07:53:17.358Z"),
"electionTerm" : NumberLong(5),
"lastCommittedOpTimeAtElection" : {
"ts" : Timestamp(1646466756, 1),
"t" : NumberLong(2)
},
"lastSeenOpTimeAtElection" : {
"ts" : Timestamp(1646466769, 2),
"t" : NumberLong(4)
},
"numVotesNeeded" : 2,
"priorityAtElection" : 1,
"electionTimeoutMillis" : NumberLong(10000),
"numCatchUpOps" : NumberLong(0),
"newTermStartDate" : ISODate("2022-03-05T07:53:17.393Z"),
"wMajorityWriteAvailabilityDate" : ISODate("2022-03-05T07:53:17.729Z")
},
"members" : [
{
"_id" : 1,
"name" : "mongo-0.mongo.mongo.svc.cluster.local:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 2287,
"optime" : {
"ts" : Timestamp(1646469067, 1),
"t" : NumberLong(5)
},
"optimeDate" : ISODate("2022-03-05T08:31:07Z"),
"syncingTo" : "",
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "",
"electionTime" : Timestamp(1646466797, 1),
"electionDate" : ISODate("2022-03-05T07:53:17Z"),
"configVersion" : 6,
"self" : true,
"lastHeartbeatMessage" : ""
},
{
"_id" : 2,
"name" : "mongo-2.mongo.mongo.svc.cluster.local:27017",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 2284,
"optime" : {
"ts" : Timestamp(1646469067, 1),
"t" : NumberLong(5)
},
"optimeDurable" : {
"ts" : Timestamp(1646469067, 1),
"t" : NumberLong(5)
},
"optimeDate" : ISODate("2022-03-05T08:31:07Z"),
"optimeDurableDate" : ISODate("2022-03-05T08:31:07Z"),
"lastHeartbeat" : ISODate("2022-03-05T08:31:09.817Z"),
"lastHeartbeatRecv" : ISODate("2022-03-05T08:31:09.807Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncingTo" : "mongo-0.mongo.mongo.svc.cluster.local:27017",
"syncSourceHost" : "mongo-0.mongo.mongo.svc.cluster.local:27017",
"syncSourceId" : 1,
"infoMessage" : "",
"configVersion" : 6
},
{
"_id" : 3,
"name" : "mongo-1.mongo.mongo.svc.cluster.local:27017",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 2260,
"optime" : {
"ts" : Timestamp(1646469067, 1),
"t" : NumberLong(5)
},
"optimeDurable" : {
"ts" : Timestamp(1646469067, 1),
"t" : NumberLong(5)
},
"optimeDate" : ISODate("2022-03-05T08:31:07Z"),
"optimeDurableDate" : ISODate("2022-03-05T08:31:07Z"),
"lastHeartbeat" : ISODate("2022-03-05T08:31:09.222Z"),
"lastHeartbeatRecv" : ISODate("2022-03-05T08:31:09.430Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncingTo" : "mongo-2.mongo.mongo.svc.cluster.local:27017",
"syncSourceHost" : "mongo-2.mongo.mongo.svc.cluster.local:27017",
"syncSourceId" : 2,
"infoMessage" : "",
"configVersion" : 6
}
],
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1646469067, 1),
"signature" : {
"hash" : BinData(0,"LxAPsQBDq0Bx33yC0PzRJWFMbbM="),
"keyId" : NumberLong("7071495762592399364")
}
},
"operationTime" : Timestamp(1646469067, 1)
}
创建数据库测试
bash
rs0:PRIMARY> use test
switched to db test
rs0:PRIMARY> db.users.insertMany( [{
... name: "bob", age: 42, status: "A", },{
... name: "ahn", age: 22, status: "A", },{
... name: "xi", age: 34, status: "D", }])
{
"acknowledged" : true,
"insertedIds" : [
ObjectId("62232018adef03bcf644f4d6"),
ObjectId("62232018adef03bcf644f4d7"),
ObjectId("62232018adef03bcf644f4d8")
]
}
rs0:PRIMARY> show collections
users
rs0:PRIMARY> db.user.find()
rs0:PRIMARY> db.users.find()
{ "_id" : ObjectId("62232018adef03bcf644f4d6"), "name" : "bob", "age" : 42, "status" : "A" }
{ "_id" : ObjectId("62232018adef03bcf644f4d7"), "name" : "ahn", "age" : 22, "status" : "A" }
{ "_id" : ObjectId("62232018adef03bcf644f4d8"), "name" : "xi", "age" : 34, "status" : "D" }
为数据库创建用户
yaml
rs0:PRIMARY> use test
switched to db test
rs0:PRIMARY> db.createUser({user: "jim", pwd: "123456", roles: [{ role: "dbOwner", db: "test" }]})
Successfully added user: {
"user" : "jim",
"roles" : [
{
"role" : "dbOwner",
"db" : "test"
}
]
}