当前位置:网站首页>Implementation of k8s redis one master multi slave dynamic capacity expansion
Implementation of k8s redis one master multi slave dynamic capacity expansion
2022-04-23 17:55:00 【Wangzi_ milk】
framework
be based on statefulset Realize one master and many slaves redis, Dynamic expansion and contraction redis Slave Library , And based on storageclass Realize dynamic storage , Ensure the persistent storage of data
Host name | IP |
---|---|
k8s-master-1 | 192.168.0.10 |
k8s-node-1 | 192.168.0.11 |
k8s-nfs | 192.168.0.55 |
nfs-provider
# k8s-nfs nfs Configuration information
[[email protected] ~]# cat /etc/exports
/data 192.168.0.0/24(rw,all_squash)
[[email protected]-master-1 1master-Nslave]# cat nfs-provider.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: redis
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: nfs-client-provisioner-runner
rules:
- apiGroups: [""]
resources: ["nodes"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: run-nfs-client-provisioner
subjects:
- kind: ServiceAccount
name: nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: redis
roleRef:
kind: ClusterRole
name: nfs-client-provisioner-runner
apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: leader-locking-nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: redis
rules:
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: leader-locking-nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: redis
subjects:
- kind: ServiceAccount
name: nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: redis
roleRef:
kind: Role
name: leader-locking-nfs-client-provisioner
apiGroup: rbac.authorization.k8s.io
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nfs-client-provisioner
labels:
app: nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: redis
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app: nfs-client-provisioner
template:
metadata:
labels:
app: nfs-client-provisioner
spec:
nodeName: k8s-master-1 # Set in the master Node operation
tolerations: # Set tolerance master Node stains
- key: node-role.kubernetes.io/master
operator: Equal
value: "true"
serviceAccountName: nfs-client-provisioner
containers:
- name: nfs-client-provisioner
image: registry.cn-hangzhou.aliyuncs.com/jiayu-kubernetes/nfs-subdir-external-provisioner:v4.0.0
imagePullPolicy: IfNotPresent
volumeMounts:
- name: nfs-client-root
mountPath: /persistentvolumes
env:
- name: PROVISIONER_NAME
value: k8s/nfs-subdir-external-provisioner
- name: NFS_SERVER
value: 192.168.0.55
- name: NFS_PATH
value: /data
volumes:
- name: nfs-client-root
nfs:
server: 192.168.0.55 # NFS SERVER_IP
path: /data
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: redis-nfs
namespace: redis
annotations:
storageclass.kubernetes.io/is-default-class: "false" # Whether to set as the default storageclass
provisioner: k8s/nfs-subdir-external-provisioner # or choose another name, must match deployment's env PROVISIONER_NAME'
allowVolumeExpansion: true
reclaimPolicy: Delete
parameters:
archiveOnDelete: "false"
redis.conf
[[email protected]-master-1 1master-Nslave]# cat configmap.yaml
apiVersion: v1
kind: Namespace
metadata:
name: redis
---
apiVersion: v1
kind: ConfigMap
metadata:
name: redis
namespace: redis
data:
redis.conf: |-
daemonize no
protected-mode no
bind 0.0.0.0
port 6379
timeout 300
databases 16
save 900 1
save 300 10
save 60 10000
loglevel notice
logfile /data/redis.log
################### RDB ####################
dir /data
stop-writes-on-bgsave-error yes
rdbcompression yes
dbfilename dump.rdb
################## REPL ###################
slave-read-only yes
repl-diskless-sync no
repl-timeout 120
################## SECURITY ###################
requirepass 123456
masterauth 123456
################## AOF ####################
appendonly no
appendfilename "appendonly.aof"
appendfsync no
statefulset
[[email protected]-master-1 1master-Nslave]# cat statefulset.yaml
apiVersion: v1
kind: Service
metadata:
name: redis
namespace: redis
spec:
selector:
app: redis
ports:
- name: redis
port: 6379
targetPort: 6379
protocol: TCP
clusterIP: None
type: ClusterIP
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: redis
namespace: redis
spec:
replicas: 2 # The default is one master and one slave
selector:
matchLabels:
app: redis
serviceName: redis
template:
metadata:
labels:
app: redis
spec:
containers:
- name: redis
image: redis:5
imagePullPolicy: IfNotPresent
command:
- /bin/sh
- -c
- | set -e hostname=$(echo `hostname` | grep -oE '\-([0-9]+)$') if [ "$hostname" != "-0" ]; then # If it's not the first node , Synchronize to the first node redis-server /etc/redis.conf slaveof redis-0.redis.redis.svc.cluster.local 6379 else redis-server /etc/redis.conf fi
volumeMounts:
- name: redis-data
mountPath: /data
- name: redis-conf
mountPath: /etc/redis.conf
subPath: redis.conf
volumes:
- name: redis-conf
configMap:
name: redis
volumeClaimTemplates:
- metadata:
name: redis-data
namespace: redis
spec:
accessModes: ["ReadWriteMany"]
storageClassName: redis-nfs
resources:
requests:
storage: 500Mi
Deploy
# Deploy configmap
[[email protected] 1master-Nslave]# kubectl apply -f configmap.yaml
namespace/redis created
configmap/redis created
# Deploy nfs-provider
[[email protected] 1master-Nslave]# kubectl apply -f nfs-provider.yaml
serviceaccount/nfs-client-provisioner created
clusterrole.rbac.authorization.k8s.io/nfs-client-provisioner-runner created
clusterrolebinding.rbac.authorization.k8s.io/run-nfs-client-provisioner created
role.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner created
rolebinding.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner created
deployment.apps/nfs-client-provisioner created
storageclass.storage.k8s.io/redis-nfs created
# Deploy statefulset
[[email protected] 1master-Nslave]# kubectl apply -f statefulset.yaml
service/redis created
statefulset.apps/redis created
# see pod
[[email protected] 1master-Nslave]# kubectl get pods -n redis
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-7f6758d967-l9b5p 1/1 Running 0 2m10s
redis-0 1/1 Running 0 100s
redis-1 1/1 Running 0 53s
# see PV,PVC
[[email protected] 1master-Nslave]# kubectl get pvc -n redis
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
redis-data-redis-0 Bound pvc-e0642424-f6b3-405f-9d12-d53b277be042 500Mi RWX redis-nfs 2m21s
redis-data-redis-1 Bound pvc-5925df68-31da-41c9-9655-7e9bd4c9bfb6 500Mi RWX redis-nfs 94s
[[email protected] 1master-Nslave]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-5925df68-31da-41c9-9655-7e9bd4c9bfb6 500Mi RWX Delete Bound redis/redis-data-redis-1 redis-nfs 96s
pvc-e0642424-f6b3-405f-9d12-d53b277be042 500Mi RWX Delete Bound redis/redis-data-redis-0 redis-nfs 2m23s
# k8s-nfs Node view
[[email protected] ~]# ls /data
redis-redis-data-redis-0-pvc-e0642424-f6b3-405f-9d12-d53b277be042 redis-redis-data-redis-1-pvc-5925df68-31da-41c9-9655-7e9bd4c9bfb6
# see redis-1 Replication
[[email protected] 1master-Nslave]# kubectl exec redis-1 -n redis -- redis-cli -a 123456 info replication
# Replication
role:slave # Slave Library
master_host:redis-0.redis.redis.svc.cluster.local # Main library information
master_port:6379
master_link_status:up
master_last_io_seconds_ago:7
master_sync_in_progress:0
slave_repl_offset:252
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:03c3dd18042139cbe116946f6c46c640fb91681b
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:252
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:252
Capacity expansion
Expansion from redis-0->redis-N
# Capacity expansion
[[email protected] 1master-Nslave]# kubectl scale statefulset redis --replicas=4 -n redis
statefulset.apps/redis scaled
# see pod
[[email protected] 1master-Nslave]# kubectl get pods -n redis
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-7f6758d967-l9b5p 1/1 Running 0 6m52s
redis-0 1/1 Running 0 6m22s
redis-1 1/1 Running 0 5m35s
redis-2 1/1 Running 0 16s
redis-3 1/1 Running 0 11s
# so pvc,pv It's all dynamically generated
[[email protected] 1master-Nslave]# kubectl get pvc -n redis
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
redis-data-redis-0 Bound pvc-e0642424-f6b3-405f-9d12-d53b277be042 500Mi RWX redis-nfs 7m
redis-data-redis-1 Bound pvc-5925df68-31da-41c9-9655-7e9bd4c9bfb6 500Mi RWX redis-nfs 6m13s
redis-data-redis-2 Bound pvc-521d77ec-64df-44e1-8a49-22d032046b86 500Mi RWX redis-nfs 54s
redis-data-redis-3 Bound pvc-365be101-2ced-4800-9db0-be68fdedd64d 500Mi RWX redis-nfs 49s
[[email protected] 1master-Nslave]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-365be101-2ced-4800-9db0-be68fdedd64d 500Mi RWX Delete Bound redis/redis-data-redis-3 redis-nfs 55s
pvc-521d77ec-64df-44e1-8a49-22d032046b86 500Mi RWX Delete Bound redis/redis-data-redis-2 redis-nfs 60s
pvc-5925df68-31da-41c9-9655-7e9bd4c9bfb6 500Mi RWX Delete Bound redis/redis-data-redis-1 redis-nfs 6m19s
pvc-e0642424-f6b3-405f-9d12-d53b277be042 500Mi RWX Delete Bound redis/redis-data-redis-0 redis-nfs 7m6s
# View the copied information , It can be seen that the synchronization is normal
[[email protected] 1master-Nslave]# kubectl exec redis-3 -n redis -- redis-cli -a 123456 info replication
# Replication
role:slave
master_host:redis-0.redis.redis.svc.cluster.local
master_port:6379
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_repl_offset:630
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:03c3dd18042139cbe116946f6c46c640fb91681b
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:630
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:449
repl_backlog_histlen:182
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
Shrinkage capacity
Shrink volume from redis-N->redis-0
# Shrinkage capacity
[[email protected] 1master-Nslave]# kubectl scale statefulset redis --replicas=1 -n redis
statefulset.apps/redis scaled
# see pod
[[email protected] 1master-Nslave]# kubectl get pods -n redis
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-7f6758d967-l9b5p 1/1 Running 0 11m
redis-0 1/1 Running 0 11m
# see pv,pvc, It can be seen that even if pod Been deleted pvc and pv Have not been deleted , Later, when we expand the capacity, the corresponding pod Will still use the previous pv,
[[email protected] 1master-Nslave]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-365be101-2ced-4800-9db0-be68fdedd64d 500Mi RWX Delete Bound redis/redis-data-redis-3 redis-nfs 6m29s
pvc-521d77ec-64df-44e1-8a49-22d032046b86 500Mi RWX Delete Bound redis/redis-data-redis-2 redis-nfs 6m34s
pvc-5925df68-31da-41c9-9655-7e9bd4c9bfb6 500Mi RWX Delete Bound redis/redis-data-redis-1 redis-nfs 11m
pvc-e0642424-f6b3-405f-9d12-d53b277be042 500Mi RWX Delete Bound redis/redis-data-redis-0 redis-nfs 12m
[[email protected] 1master-Nslave]# kubectl get pvc -n redis
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
redis-data-redis-0 Bound pvc-e0642424-f6b3-405f-9d12-d53b277be042 500Mi RWX redis-nfs 12m
redis-data-redis-1 Bound pvc-5925df68-31da-41c9-9655-7e9bd4c9bfb6 500Mi RWX redis-nfs 11m
redis-data-redis-2 Bound pvc-521d77ec-64df-44e1-8a49-22d032046b86 500Mi RWX redis-nfs 6m39s
redis-data-redis-3 Bound pvc-365be101-2ced-4800-9db0-be68fdedd64d 500Mi RWX redis-nfs 6m34s
版权声明
本文为[Wangzi_ milk]所创,转载请带上原文链接,感谢
https://yzsam.com/2022/04/202204231754544066.html
边栏推荐
- Write a regular
- Logic regression principle and code implementation
- 48. 旋转图像
- Kubernetes service discovery monitoring endpoints
- 122. The best time to buy and sell stocks II - one-time traversal
- Encapsulate a timestamp to date method on string prototype
- Add drag and drop function to El dialog
- Cloud native Virtualization: building edge computing instances based on kubevirt
- 440. The k-th small number of dictionary order (difficult) - dictionary tree - number node - byte skipping high-frequency question
- .105Location
猜你喜欢
Tell the truth of TS
394. 字符串解码-辅助栈
Nat Commun|在生物科学领域应用深度学习的当前进展和开放挑战
MySQL进阶之索引【分类,性能分析,使用,设计原则】
48. Rotate image
The JS timestamp of wechat applet is converted to / 1000 seconds. After six hours and one day, this Friday option calculates the time
92. 反转链表 II-字节跳动高频题
Examination question bank and online simulation examination of the third batch (main person in charge) of special operation certificate of safety officer a certificate in Guangdong Province in 2022
958. 二叉树的完全性检验
MySQL 中的字符串函数
随机推荐
2021 Great Wall Cup WP
JS implementation private attribute
MySQL_01_简单数据检索
2022年广东省安全员A证第三批(主要负责人)特种作业证考试题库及在线模拟考试
440. The k-th small number of dictionary order (difficult) - dictionary tree - number node - byte skipping high-frequency question
高德地图搜索、拖拽 查询地址
2022制冷与空调设备运行操作判断题及答案
394. String decoding - auxiliary stack
How to read literature
587. 安装栅栏 / 剑指 Offer II 014. 字符串中的变位词
SystemVerilog(六)-变量
Theory and practice of laser slam in dark blue College - Chapter 2 (odometer calibration)
[appium] write scripts by designing Keyword Driven files
Land cover / use data product download
_ FindText error
122. The best time to buy and sell stocks II - one-time traversal
958. 二叉树的完全性检验
JS parsing and execution process
列表的使用-增删改查
Sword finger offer 22 The penultimate node in the linked list - speed pointer