加载中...

K8s配置Mongo集群


一 前言

随着发展 k8s对于有状态应用的适应已经越来越好 下面就针对mongo db的部署为大家分享下k8s sts控制器的使用


二 部署

2.1 环境准备

数据库这种核心组件 首先需要准备持久化的数据卷 这里就使用sc动态存储来作为底层存储

yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: nfs-mongo
provisioner: nfs-provisioner-mongo
reclaimPolicy: Retain
volumeBindingMode: Immediate

创建ns

yaml
apiVersion: v1
kind: Namespace
metadata:
  name: mongo

配置cm

yaml
kind: ConfigMap
apiVersion: v1
metadata:
  name: mongo-keyfile
  namespace: mongo
  annotations:
    kubesphere.io/creator: admin
data:
  key.txt: |-
    nX24aaT7/WOQKOfchxP66kulNNdjSn5LpsE/RbDL7gwMIqZ424WQVCOJzRV5YMw9
    fQ0pz591I+U+G2/iP8PEPF6po1/rLKi1LotoKV+ubh336N/TCMzi86OH3aibNVyC
    ydhbw6hTrtYxaZatxyKHZ7C8K39su05T0cDh1iqCYDBEkxUh5zYUafwfxa5GC5Ar
    +NGiDhrZJXj+fjGBhUOBA8yl0fG7V5lx56PIJ7u9M+EaFoJcoYembSZ40A/JRcmO
    wV76/5sng1HQ0A016xh3WPFrUkSrie3RqmpjTP/gx7sej3fiheOxNmjjFYcR4Fof
    Y3VH+MXs0qJm4HIcJUIEnYInunFlsbpyrSFu77sgfXuL/zYkozK3CPaK5+kZZCfY
    FqsTxo9xWI9D8QVpXlSU7Pbj4g8DatHBjAxQWgDgEFxABhypILz56RsDybQ8aNtX
    OWuEhJJUvjAMEYutTOSM80ATYSFh+XbB19n5SthDfcz9RvJ+XMdn6biBZwS46vqb
    oKWaM2U7lpNC2ZvhEAfGYsHwPfs98ak4O1n1iYQVeohC+TkQdrJQk5sppMp2bu5b
    hcG0qrYOWgjXS3EayMMAwYKBAcyb0sCugFIY4KeCl3RMzJpnSVL916LwXuOaHRTb
    SkFrbgEapveUaGB+NCARJq/NUi2sA16gpGVoLVYctJCHnJVNhTQ1bH32I7PULiL8
    6fae3gLs6yTbC8Isdn4CqkJP4yB7ae8z9VOYpTouP7YblJaGfcQ/pGD0xWVCj4BL
    IXhsEygotZj+3RySpFFwedX/Ku5JDMlA7FDbwNguUBPUtO6a8osiw5jSb8wplFXZ
    ei8LgkyGx6onVrIgzuQv0CDm9tUpdgNuf8d7CYYWLyZBpTuBFrbJdnWQ1ViigRq5
    wlKzb2yL76KxiIch4JwL61XTC/6+k5K2opGimAP8Ebft9tfB0M38cFfrdD7PJCL5
    x7c9cfDxO4O56DNzpPimyW6pJ6nw

2.2 配置sts

yaml
kind: StatefulSet
apiVersion: apps/v1
metadata:
  name: mongo
  namespace: mongo
  annotations:
    kubesphere.io/creator: admin
spec:
  replicas: 3
  selector:
    matchLabels:
      environment: test
      role: mongo
  template:
    metadata:
      creationTimestamp: null
      labels:
        environment: test
        role: mongo
      annotations:
        kubesphere.io/restartedAt: '2022-03-05T07:52:33.099Z'
    spec:
      volumes:
        - name: mongo-keyfile
          configMap:
            name: mongo-keyfile
            items:
              - key: key.txt
                path: key.txt
            defaultMode: 420
      initContainers:
        - name: fix-permissions
          image: harbor.huhuhahei.cn/huhuhahei/busybox
          command:
            - sh
            - '-c'
            - >-
              cp /data/key.txt /data/db/key && chown 999:999 /data/db/key &&
              chmod 400 /data/db/key  && ls -l /data/
          resources: {}
          volumeMounts:
            - name: mongo
              mountPath: /data/db
            - name: mongo-keyfile
              mountPath: /data/key.txt
              subPath: key.txt
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: Always
          securityContext:
            privileged: true
      containers:
        - name: mongo
          image: harbor.huhuhahei.cn/huhuhahei/mongo:4.2.8
          command:
            - docker-entrypoint.sh
            - '--replSet'
            - rs0
            - '--bind_ip'
            - 0.0.0.0
            - '--auth'
            - '--clusterAuthMode'
            - keyFile
            - '--keyFile'
            - /data/db/key
          ports:
            - containerPort: 27017
              protocol: TCP
          env:
            - name: MONGO_INITDB_ROOT_USERNAME
              value: root
            - name: MONGO_INITDB_ROOT_PASSWORD
              value: dSJN52PuSqn
          resources: {}
          volumeMounts:
            - name: mongo
              mountPath: /data/db
            - name: mongo-keyfile
              mountPath: /data/key.txt
              subPath: key.txt
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: Always
        - name: mongo-sidecar
          image: harbor.huhuhahei.cn/huhuhahei/k8s-mongo-sidecar
          env:
            - name: KUBERNETES_POD_LABELS
              value: 'role=mongo,environment=test'
            - name: KUBERNETES_NAMESPACE
              value: mongo
            - name: MONGO_USERNAME
              value: root
            - name: MONGO_PASSWORD
              value: dSJN52PuSqn
            - name: MONGO_DATABASE
              value: admin
            - name: KUBERNETES_SERVICE_NAME
              value: mongo
          resources: {}
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: IfNotPresent
      restartPolicy: Always
      terminationGracePeriodSeconds: 10
      dnsPolicy: ClusterFirst
      securityContext: {}
      schedulerName: default-scheduler
  volumeClaimTemplates:
    - kind: PersistentVolumeClaim
      apiVersion: v1
      metadata:
        name: mongo
        creationTimestamp: null
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 20Gi
        storageClassName: nfs-mongo
        volumeMode: Filesystem
      status:
        phase: Pending
  serviceName: mongo
  podManagementPolicy: OrderedReady
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      partition: 0
  revisionHistoryLimit: 10

因为mongo-sidecar需要监控mongo状态所以需要创建ClusterRoleBinding

yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: mongo-default-view
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: view
subjects:

- kind: ServiceAccount
  name: default
  namespace: default

2.3 配置svc

yaml
kind: Service
apiVersion: v1
metadata:
  name: mongo
  namespace: mongo
  labels:
    name: mongo
  annotations:
    kubesphere.io/creator: admin
spec:
  ports:
    - protocol: TCP
      port: 27017
      targetPort: 27017
  selector:
    role: mongo
  clusterIP: None
  type: ClusterIP
  sessionAffinity: None

2.4 交付k8s

yaml
kubectl apply -f .

三 验证

3.1 连接测试

bash
# mongo -uroot -p dSJN52PuSqn
MongoDB shell version v4.2.8
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("8f142a74-c99c-48df-8c37-5107f846e4a5") }
MongoDB server version: 4.2.8
Server has startup warnings:
2022-03-05T07:53:05.774+0000 I  CONTROL  [initandlisten]
2022-03-05T07:53:05.774+0000 I  CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2022-03-05T07:53:05.774+0000 I  CONTROL  [initandlisten] **        We suggest setting it to 'never'
2022-03-05T07:53:05.774+0000 I  CONTROL  [initandlisten]
2022-03-05T07:53:05.774+0000 I  CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2022-03-05T07:53:05.774+0000 I  CONTROL  [initandlisten] **        We suggest setting it to 'never'
2022-03-05T07:53:05.774+0000 I  CONTROL  [initandlisten]
---
Enable MongoDB's free cloud-based monitoring service, which will then receive and display
metrics about your deployment (disk utilization, CPU, operation statistics, etc).

The monitoring data will be available on a MongoDB website with a unique URL accessible to you
and anyone you share the URL with. MongoDB may use this information to make product
improvements and to suggest MongoDB products and deployment options to you.

To enable free monitoring, run the following command: db.enableFreeMonitoring()
To permanently disable this reminder, run the following command: db.disableFreeMonitoring()
---

rs0:PRIMARY> rs.status
function() {
    return db._adminCommand("replSetGetStatus");
}
rs0:PRIMARY> rs.status()
{
        "set" : "rs0",
        "date" : ISODate("2022-03-05T08:31:10.117Z"),
        "myState" : 1,
        "term" : NumberLong(5),
        "syncingTo" : "",
        "syncSourceHost" : "",
        "syncSourceId" : -1,
        "heartbeatIntervalMillis" : NumberLong(2000),
        "majorityVoteCount" : 2,
        "writeMajorityCount" : 2,
        "optimes" : {
                "lastCommittedOpTime" : {
                        "ts" : Timestamp(1646469067, 1),
                        "t" : NumberLong(5)
                },
                "lastCommittedWallTime" : ISODate("2022-03-05T08:31:07.713Z"),
                "readConcernMajorityOpTime" : {
                        "ts" : Timestamp(1646469067, 1),
                        "t" : NumberLong(5)
                },
                "readConcernMajorityWallTime" : ISODate("2022-03-05T08:31:07.713Z"),
                "appliedOpTime" : {
                        "ts" : Timestamp(1646469067, 1),
                        "t" : NumberLong(5)
                },
                "durableOpTime" : {
                        "ts" : Timestamp(1646469067, 1),
                        "t" : NumberLong(5)
                },
                "lastAppliedWallTime" : ISODate("2022-03-05T08:31:07.713Z"),
                "lastDurableWallTime" : ISODate("2022-03-05T08:31:07.713Z")
        },
        "lastStableRecoveryTimestamp" : Timestamp(1646469057, 1),
        "lastStableCheckpointTimestamp" : Timestamp(1646469057, 1),
        "electionCandidateMetrics" : {
                "lastElectionReason" : "electionTimeout",
                "lastElectionDate" : ISODate("2022-03-05T07:53:17.358Z"),
                "electionTerm" : NumberLong(5),
                "lastCommittedOpTimeAtElection" : {
                        "ts" : Timestamp(1646466756, 1),
                        "t" : NumberLong(2)
                },
                "lastSeenOpTimeAtElection" : {
                        "ts" : Timestamp(1646466769, 2),
                        "t" : NumberLong(4)
                },
                "numVotesNeeded" : 2,
                "priorityAtElection" : 1,
                "electionTimeoutMillis" : NumberLong(10000),
                "numCatchUpOps" : NumberLong(0),
                "newTermStartDate" : ISODate("2022-03-05T07:53:17.393Z"),
                "wMajorityWriteAvailabilityDate" : ISODate("2022-03-05T07:53:17.729Z")
        },
        "members" : [
                {
                        "_id" : 1,
                        "name" : "mongo-0.mongo.mongo.svc.cluster.local:27017",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 2287,
                        "optime" : {
                                "ts" : Timestamp(1646469067, 1),
                                "t" : NumberLong(5)
                        },
                        "optimeDate" : ISODate("2022-03-05T08:31:07Z"),
                        "syncingTo" : "",
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "electionTime" : Timestamp(1646466797, 1),
                        "electionDate" : ISODate("2022-03-05T07:53:17Z"),
                        "configVersion" : 6,
                        "self" : true,
                        "lastHeartbeatMessage" : ""
                },
                {
                        "_id" : 2,
                        "name" : "mongo-2.mongo.mongo.svc.cluster.local:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 2284,
                        "optime" : {
                                "ts" : Timestamp(1646469067, 1),
                                "t" : NumberLong(5)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(1646469067, 1),
                                "t" : NumberLong(5)
                        },
                        "optimeDate" : ISODate("2022-03-05T08:31:07Z"),
                        "optimeDurableDate" : ISODate("2022-03-05T08:31:07Z"),
                        "lastHeartbeat" : ISODate("2022-03-05T08:31:09.817Z"),
                        "lastHeartbeatRecv" : ISODate("2022-03-05T08:31:09.807Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "",
                        "syncingTo" : "mongo-0.mongo.mongo.svc.cluster.local:27017",
                        "syncSourceHost" : "mongo-0.mongo.mongo.svc.cluster.local:27017",
                        "syncSourceId" : 1,
                        "infoMessage" : "",
                        "configVersion" : 6
                },
                {
                        "_id" : 3,
                        "name" : "mongo-1.mongo.mongo.svc.cluster.local:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 2260,
                        "optime" : {
                                "ts" : Timestamp(1646469067, 1),
                                "t" : NumberLong(5)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(1646469067, 1),
                                "t" : NumberLong(5)
                        },
                        "optimeDate" : ISODate("2022-03-05T08:31:07Z"),
                        "optimeDurableDate" : ISODate("2022-03-05T08:31:07Z"),
                        "lastHeartbeat" : ISODate("2022-03-05T08:31:09.222Z"),
                        "lastHeartbeatRecv" : ISODate("2022-03-05T08:31:09.430Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "",
                        "syncingTo" : "mongo-2.mongo.mongo.svc.cluster.local:27017",
                        "syncSourceHost" : "mongo-2.mongo.mongo.svc.cluster.local:27017",
                        "syncSourceId" : 2,
                        "infoMessage" : "",
                        "configVersion" : 6
                }
        ],
        "ok" : 1,
        "$clusterTime" : {
                "clusterTime" : Timestamp(1646469067, 1),
                "signature" : {
                        "hash" : BinData(0,"LxAPsQBDq0Bx33yC0PzRJWFMbbM="),
                        "keyId" : NumberLong("7071495762592399364")
                }
        },
        "operationTime" : Timestamp(1646469067, 1)
}

创建数据库测试

bash
rs0:PRIMARY> use test
switched to db test
rs0:PRIMARY> db.users.insertMany( [{
...      name: "bob", age: 42, status: "A", },{
...      name: "ahn", age: 22, status: "A", },{
...      name: "xi", age: 34, status: "D", }])
{
        "acknowledged" : true,
        "insertedIds" : [
                ObjectId("62232018adef03bcf644f4d6"),
                ObjectId("62232018adef03bcf644f4d7"),
                ObjectId("62232018adef03bcf644f4d8")
        ]
}
rs0:PRIMARY> show collections
users
rs0:PRIMARY> db.user.find()
rs0:PRIMARY> db.users.find()
{ "_id" : ObjectId("62232018adef03bcf644f4d6"), "name" : "bob", "age" : 42, "status" : "A" }
{ "_id" : ObjectId("62232018adef03bcf644f4d7"), "name" : "ahn", "age" : 22, "status" : "A" }
{ "_id" : ObjectId("62232018adef03bcf644f4d8"), "name" : "xi", "age" : 34, "status" : "D" }

为数据库创建用户

yaml
rs0:PRIMARY> use test
switched to db test
rs0:PRIMARY> db.createUser({user: "jim", pwd: "123456", roles: [{ role: "dbOwner", db: "test" }]})
Successfully added user: {
        "user" : "jim",
        "roles" : [
                {
                        "role" : "dbOwner",
                        "db" : "test"
                }
        ]
}

文章作者: huhuhahei
版权声明: 本博客所有文章除特別声明外,均采用 CC BY 4.0 许可协议。转载请注明来源 huhuhahei !
评论
No comment yet.
Powered By Valine
v1.4.18
晚上使用黑夜模式阅读能够减轻视觉疲劳。