Cloud native Virtualization: building edge computing instances based on kubevirt

With Kubernetes The popularity of , More and more businesses are running on containers , However, there are still some business forms that are more suitable for running on virtual machines , How to control virtual machines and containers at the same time has gradually become the mainstream demand in the cloud native era , Kubevirt The perfect solution .

1. Kubevirt  Introduce
2. Virtual machine management
3. summary

In the age of cloud Nativity , More and more businesses are gradually migrating to containers , Container has also become an indispensable form of resource publishing and management , The lightweight advantage of container is more obvious in the scene of edge computing , therefore , When building a cloud platform at the edge, we must consider the running form of compatible containers .

Kubernetes At present, it is the most common and popular platform for container scheduling and scheduling , At the same time, more and more cloud vendors choose to use Kubernetes To manage both containers and virtual machines . stay Kubernetes Build a platform for managing virtual machines , There have also been many excellent projects in the industry , for example Kubevirt、Virtlet etc. , among Kubevirt With its architectural design and functional advantages, it has become one of the best solutions to solve the compatibility problem between virtual machine and container . This article will be Kubevirt Based on , How to share details based on Kubevirt Build an edge calculation example .

01 Kubevirt Introduce

Kubevirt What is it?

Kubevirt yes Red Hat Open source projects that run virtual machines as containers , be based on Kubernetes function , By using custom resources （CRD） And other Kubernetes Function to seamlessly expand existing clusters , To provide a set of virtualization tools that can be used to manage virtual machines API.

The overall architecture

The picture above depicts Kubevirt The overall structure of , It contains four main key components ：

• virt-api:

  • by Kubevirt Provide API Service capability, , For example, many custom API request , Such as power on 、 To turn it off 、 Restart and other operations , adopt APIService As Kubernetes Apiserver Plug in for , Business can be done through Kubernetes Apiserver Ask directly to virt-api;

• virt-controller:

  • Kubevirt The controller , Function like Kubernetes Of controller-manager, Management and monitoring VMI Object and its associated Pod, Update its status ;

• virt-handler:

  • With Daemonset Formal deployment , Function like Kubelet, adopt Watch This machine VMI And instance resources , Manage all virtual machine instances on the host ;

  • The main actions are as follows :

    • send VMI As defined in Spec And the corresponding libvirt （ Local socket signal communication ） Keep in sync ;

    • Report and control the update of virtual machine status ;

    • Call the relevant plug-ins to initialize the network and storage resources on the node ;

    • Heat transfer related operations ;

• virt-launcher:

  • Kubevirt Will be for each one VMI Object to create a Pod, The Pod The main process of is virt-launcher,virt-launcher Of Pod Provides cgroups and namespaces The isolation ,virt-launcher Is the main process of the virtual machine instance .

  • virt-handler By way of VMI Of CRD Object passed to virt-launcher To inform virt-launcher start-up VMI. then ,virt-launcher Use local... In its container libvirtd Instance to start VMI.virt-launcher trusteeship VMI process , And in VMI Terminate after exit .

  • If Kubernetes Running at VMI Try closing... Before exiting virt-launcher Containers ,virt-launcher Will send the signal from Kubernetes Forwarding to VMI process , And try to delay the termination of the container , until VMI Successfully closed .

The following figure for virt-launcher And libvirt Communication overview ：

Resource objects

Kubevirt yes Kubernetes Virtual machine management plug-in , Realize the management function of virtual machine by customizing controller and resources , By customizing resources (CRD) Mechanism , meanwhile Kubevirt You can customize additional operations , To adjust the behavior that is not available in a regular container . Here are some key resources ：

• VirtualMachineInstance（VMI）： Is the smallest resource for managing virtual machines , One VirtualMachineInstance Object represents a running virtual machine instance , Contains the various configurations required for a virtual machine .

• VirtualMachine（VM）： For... In the cluster VirtualMachineInstance Provide management functions , For example, power on / To turn it off / Restart the virtual machine , Ensure the startup state of the virtual machine instance , And the virtual machine instance is 1:1 The relationship between .

• VirtualMachineInstanceMigrations： Resources required for virtual machine migration , A resource object is represented as a migration task , And reflect the status of virtual machine migration .

• VirtualMachineInstanceReplicaSet： similar ReplicaSet, You can specify the quantity , Batch create virtual machines .

• DataVolume:   It's right PVC The abstraction above , By customizing the data source , from CDI The controller is automatically created PVC And import the data to PVC For virtual machines .

The following is a VM Resource examples ：

apiVersion: kubevirt.io/v1alpha3kind: VirtualMachinemetadata:  labels:    kubevirt.io/vm: vm-cirros  name: vm-cirrosspec:  running: false  template:    metadata:      labels:        kubevirt.io/vm: vm-cirros    spec:      domain:        devices:          disks:          - disk:              bus: virtio            name: containerdisk          - disk:              bus: virtio            name: cloudinitdisk        machine:          type: ""        resources:          requests:            memory: 64M      terminationGracePeriodSeconds: 0      volumes:      - name: containerdisk        containerDisk:          image: kubevirt/cirros-container-disk-demo:latest      - cloudInitNoCloud:          userDataBase64: IyEvYmluL3NoCgplY2hvICdwcmludGVkIGZyb20gY2xvdWQtaW5pdCB1c2VyZGF0YScK        name: cloudinitdisk

02  Virtual machine management

In understanding Kubevirt What is it? , Its main architecture and key resource objects , Let's see how to use it Kubevirt Virtual machine management . It is mainly divided into virtual machine creation 、 Storage and network .

Virtual machine creation

Virtual machine creation is divided into creation  DataVolume( Prepare storage for the virtual machine ) and VMI  Two parts . The brief process is as follows ：

1. User pass kubectl/api establish VM object ;

2. virt-api adopt webhook check VM object ;

3. virt-controller Listen to the VM The creation of , Generate VMI object ;

4. virt-controller Listen to the VMI The creation of , Judge the virtual machine DataVolume Whether it is initialized , If it's not initialized , Create DateVolume Initialize and prepare the data required by the virtual machine ;

5. virtual machine DataVolume Once the initialization is complete ,virt-controller establish virt-launcher Pod To start the virtual machine ;

6. kubernetes Scheduling virtual machines Pod To a host in the cluster ;

7. virt-controller Watch To VMI Your container has started , to update VMI Object nodeName Field . Follow up work by virt-handler Take over for further operation ;

8. virt-handler（DaemonSet）Watch To VMI Assigned to the host running it , By acquiring Domain And vmi State to decide to send a command to start the virtual machine ;

9. virt-launcher Get virt-handler command , And libvirtd Instance communication to operate the virtual machine .

above , We have completed the preliminary creation of the virtual machine . But during the virtual machine creation process , Different status prompts may appear , This paper lists several common states and their diagrams ：

• Pending： The virtual machine instance has been created , Wait for the subsequent control process ;

• Scheduling： virtual machine Pod Created , Scheduling ;

• Scheduled： virtual machine Pod Dispatch complete , And in running state , After this state virt-controller Control over , from virt-handler Take over the follow-up work ;

• Running： The virtual machine is running normally ;

• Succeeded： The virtual machine received sigterm Exit due to signal or internal shutdown operation ;

• Failed： Virtual machine due to abnormal conditions crash;

Virtual machine storage

Storage part Kubevirt Can be based on Kubetnetes function , So it can be reused Kubernetes Storage design in , Such as PVC、PV etc. . meanwhile ,Kubevirt stay PVC Customized on DataVolume resources .

DataVolume By Kubevirt The team developed and open source container data import plug-ins CDI（containerized-data-importer） Defined by the CRD.

Before the virtual machine starts, it will create DataVolume object , from CDI The controller is automatically created PVC, And download data according to the supported data sources . Virtual Pod Before starting Kubernetes Would call CSI Mount the cloud disk on the host , It is expressed as a piece of equipment and hung on the equipment to launcher Pod in , Finally, the virtual machine passes virtio Drive access disk device .

 

Virtual machine network

On the Internet ,Kubevirt Reuse the Kubernetes Container network for , And on top of this 4 A virtual machine network model . At present Kubevirt Four main network modes are supported ：

// +k8s:openapi-gen=truetype InterfaceBindingMethod struct {
      Bridge     *InterfaceBridge     `json:"bridge,omitempty"`   Slirp      *InterfaceSlirp      `json:"slirp,omitempty"`   Masquerade *InterfaceMasquerade `json:"masquerade,omitempty"`   SRIOV      *InterfaceSRIOV      `json:"sriov,omitempty"`}

Virtual machines and container networks are interconnected , It can realize the interconnection between different forms of services of virtual machines and containers . Here's an introduction Bridge Pattern , This method is relatively simple to implement , But the network loss is also large ：

• Bridge In mode pod Of veth pair Still by cni Manage creation , and virt-launcher Will Pod IP cast off ,pod veth equipment eth0 Only as a bridge between the virtual network card of the virtual machine and the external network .

• virt-launcher stay pod Created in tap equipment vnet0 and bridge Bridge equipment br1, At the same time, a simple single ip dhcp server, Boot in virtual machine dhclient, Virt-launcher take IP Assign to virtual machine .

 

03  summary

The above is based on Kubevirt The technical scheme of constructing edge computing example , Also on the Kubevirt The architecture and main modules are described in detail .Kubevirt As Kubernetes Of CRD plug-in unit , Not only for Kubernetes It complements the ability of scheduling virtual machines , It also provides a relatively mature platform for solving the fusion scheduling of container and virtual machine 、 Viable solutions .

Reference material ：

[1] https://github.com/kubevirt/kubevirt

[2] https://kubevirt.io/