Catalog

OpenEBS

brief introduction

framework

Data plane

Control plane

Reference resources

---

OpenEBS

OpenEBS It's a utility model Go Container based block storage open source software written in .OpenEBS Make it more reliable to run critical tasks in containers and loads that need data persistence .

OpenEBS from CloudByte Research and development , This is a company specializing in container storage ,OpenEBS It is an open source product ,CloudByte Put his experience in enterprise container storage into the project . The vision of the project is also simple , Is to enable the storage services in the workload requiring persistent storage to be directly integrated into the environment , Storage services can automatically manage , Hide the stored details , Just as the storage system is another infrastructure .

We know AWS Provided in EBS（Elastic Block Storage）, Apply to Amazon EC2 Persistent block storage , It can meet the functional and performance requirements of the most demanding applications ,OpenEBS That is, its open source implementation .

brief introduction

Use OpenEBS, You can put containers with persistent data , Treat... Like any other ordinary container .OpenEBS It is also deployed through containers , Support Kubernetes、Swarm、Mesos、Rancher Arrange and schedule , Storage services can be assigned to each pod、 Applications 、 Cluster or container level , Include ：

• Cross node data persistence
• Data synchronization across availability zones and cloud vendors
• Use commercial hardware and container engines to provide highly scalable block storage
• Integration with container orchestration engine , Developers' applications can be configured automatically OpenEBS
• be based on CloudByte stay BSD Experience in containerization , Provide users with OpenEBS Of QoS Guarantee

framework

OpenEBS The storage controller itself runs in the container .OpenEBS Volume It consists of one or more containers running as microservices . This storage controller function is based on the micro service architecture —— Each volume's data is provided by its own set of containers , Instead of having a unified system that provides control for multiple volumes at the same time , Monomers （monolithic） Storage controller to provide . This is it. OpenEBS Essential difference from traditional storage devices .

OpenEBS The architecture of can be divided into data plane （Data Plane） And the control plane （Control Plane） Two parts ：

• Data plane ： Provide data storage for applications
• Control plane ： management OpenEBS Volume container , This usually uses the functions of container choreography Software

Data plane

The picture below is OpenEBS Corresponding to Kubernetes The architecture diagram of the deployment on the cluster . among , The yellow part is OpenEBS Persistent storage volumes , adopt Kubernetes Of PV To create , Use iSCSI To achieve , Data saved in node Volumes on nodes or in the cloud （ Such as EBS、GPD etc. ）, It depends on where your cluster is deployed .OpenEBS Volume management is completely independent of the user's application lifecycle , This is also Kuberentes Medium PV The basic idea of .

OpenEBS Volumes provide persistent storage for containers , It has flexibility for system failure , Faster access to storage , Snapshot and backup capabilities . Besides , It also provides monitoring of usage and execution QoS The mechanism of strategy .

The disk on which data is stored is called the storage backend , It can be a host Directory , Additional block devices or remote disks . Every OpenEBS The volume contains a iSCSI Target container （ In the figure above, it is shown as openebs-vol1） And one or more replica containers （openebs-vol1-R1 and openebs-vol1-R2）.

Applications pod adopt iSCSI The target container accesses the storage ,iSCSI The destination container replicates data to all its replicas . In the event of a node failure ,iSCSI The target container will start from one of the remaining online nodes , And provide data by connecting to the available replica container .

Source code

The implementation of this part includes two containers ：

• openebs/jiva： Storage control function , Including replication logic
• openebs/gotgt： from openebs/jiva The use of iSCSI Target function

Control plane

OpenEBS The control plane is also called storage arrangement or maya. The purpose is to create hyperfused OpenEBS, Mount it to, for example Kubernetes、Swarm、Nomad Wait for the container scheduling engine , It is used to extend the storage function provided by a specific container choreography system .

OpenEBS The control plane is also based on microservices , Its services can be divided into the following parts ：

• Container orchestration plug-in , Used to add the function of strong container orchestration framework ：
  • Kubernetes Dynamic configuration ：openebs-provisioner
  • Kubernetes-dashboard：openebs-dashboard
  • Extended schema： be based on Kubernetes Of CRD（ Custom resource type ）, Storage OpenEBS Relevant configuration data
• The cluster service , Provide OpenEBS Specific storage intelligence , Such as ：
  • maya-apiserver： Contains... For performing volume operations API, Convert orchestrated requests to specific operating system containers
  • maya-mulebot： Use the collected information to suggest optimized layout and event handling tips
  • maya-connect： Allow uploading monitoring data to maya-cloud, In order to further analyze the storage access mode
• Node services , Provide OpenEBS Specific random kubelet Storage intelligence running together , Such as ：
  • maya-agent： Including storage management functions

By using prometheus、heapster、grafana and jaegar Perform the above services , Monitoring and tracking functions can be added .

Source code

• openebs/maya： All specific binaries （ Non plug in ） All stored in this warehouse , such as maya-apiserver、maya-agent、maya-mulebot、maya-connect、mayactl wait .
• openebs-dashboard：kubernetes-dashboard The branch of the project , Expanded storage capabilities .
• openebs-provisioner： come from Kubernetes Of the incubator project OpenEBS K8s Provisioner.

Reference resources

• https://www.openebs.io/
• https://github.com/openebs/openebs
• Data Scientists adopting tools and solutions that allow them to focus more on Data Science and less on the infrastructure around them