Project background

The cloud platform project of a national Ministry has been launched since 2020 The construction of cloud platform began in mid-2008 , In the later stage, it is expected that all VMware The host is migrated to the domestic cloud platform , However, due to the large number of virtual machines , The systematic research work has not been completed yet , The project is seriously overdue . from 2021 year 6 After getting involved in the project last month , We go through 「 cicada Prophet」 Automated research tools , Quickly help users complete business system research , And through ” Progressive disaster recovery “ The concept of migration eliminates users' concerns before migration , In less than 1 In more than months , Successfully completed the migration of multiple sets of business systems , Involving more than 100 virtual machines . This article focuses on how to use 「 cicada Prophet」 Research tools , Complete automation research and system information sorting .

User environment

The business system migrated by the customer this time runs in VMware On the main virtualization platform ,VMware ESXi The version mainly focuses on 5.1、5.5 and 6.5 On the version , total 20 platform ESXi The server , Number of virtual machines 1000 Table above , It is divided into business area and DMZ District .

Research environment preparation

Software installation

1、 Prepare the operating environment

stay ESXi Create a virtual machine on , The configuration is as follows ：

• operating system ：CentOS 7.3

• specifications ：2 CPU 2G Memory 40G

• The Internet ： Be able to access vCenterESXi,443 and 902 port (TCP agreement )

2、 Install the container

Log in to the system , Use root erection of tank Docker

wget https://download.docker.com/linux/static/stable/x86_64/docker-19.03.15.tgz

  Configure container

setenforce 0

tar -zxvf docker-19.03.15.tgz

mv docker/* /usr/bin/ && rm -rf docker

 

cat > /etc/systemd/system/docker.service <<-EOF

[Unit]

Description=Docker Application Container Engine

Documentation=https://docs.docker.com

After=network-online.target firewalld.service

Wants=network-online.target

 

 

[Service]

Type=notify

# the default is not to use systemd for cgroups because the delegate issues still

# exists and systemd currently does not support the cgroup feature set required

# for containers run by docker

ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2375 -H unix://var/run/docker.sock

ExecReload=/bin/kill -s HUP

# Having non-zero Limit*s causes performance problems due to accounting overhead

# in the kernel. We recommend using cgroups to do container-local accounting.

LimitNOFILE=infinity

LimitNPROC=infinity

LimitCORE=infinity

# Uncomment TasksMax if your systemd version supports it.

# Only systemd 226 and above support this version.

#TasksMax=infinity

TimeoutStartSec=0

# set delegate yes so that systemd does not reset the cgroups of docker containers

Delegate=yes

# kill only the docker process, not all processes in the cgroup

KillMode=process

# restart the docker process if it exits prematurely

Restart=on-failure

StartLimitBurst=3

StartLimitInterval=60s

 



[Install]

WantedBy=multi-user.target

EOF

Run container

systemctl daemon-reload

systemctl start docker

3、 install 「 cicada Prophet」 Research tools

docker pull \

  registry.cn-beijing.aliyuncs.com/oneprocloud-opensource/cloud-discovery-prophet:latest

4、 Run container

We will /root The directory is mapped to the inside of the container

docker run \

  --net host \

  --privileged=true \

  --name prophet \

  -v $HOME:/root \

  -dit \

  registry.cn-beijing.aliyuncs.com/oneprocloud-opensource/cloud-discovery-prophet:latest

Systematic research

System information collection

1、 Get into 「 cicada Prophet」 Containers

docker exec -ti prophet bash

2、 Scan active hosts in the whole network

Scan hosts in the specified network address range , And make records , It can be used as input for subsequent more detailed information collection . After scanning , Will be automatically in the specified /root Generate... Under path csv file .

prophet-cli scan --host 10.1.0.43-62 --output-path /root

3、 Fill in the user name / password

Will generate CSV File back to local , use Excel edit , Fill in the user name / Password information

CSV Structure description （ Fill in the red field, which is required ）

4、 Perform collection

After completion , take scan_hosts.csv The file is transferred to the host /root Under the table of contents , You can perform detailed information collection , The method is as follows ：

prophet-cli collect --host-file /root/scan_hosts.csv --output-path /root/

After the command runs , There will be summary information , If the acquisition fails , You need to query the error reason in the log

===========Summary==========

Total 73 host(s) in list, Need to check 2 host(s), success 0 hosts, failed 2 hosts.

Failed hosts: ['[WINDOWS]xxx', '[LINUX]xxx']

============================

After execution , Will be in ouput-path In the middle of host_collection_xxx.zip file , This file is the packaged file after collection .

System information analysis

Analyze the results of the previous step

prophet-cli report \

  --package-file /root/host_collection_20210804091400.zip \

  --output-path /root/

Last generated analysis_report.csv Is the file after analysis , Can be transferred back to the local , For further analysis , So far, all information collection work has been completed .

summary  

In this research work , The total running time of the script is 30 Within minutes , The total number of virtual machines surveyed 1049, among Window 582 platform ,Linux 467 platform ,CPU Total quantity 7734 nucleus , Total memory size 19T, The total storage capacity is 60TB.

In large-scale systematic research , Automation is a necessary means , At present, there are few tools in the field of migration research , therefore 「 cicada Prophet」 Tools are undoubtedly the best guarantee for the accuracy of systematic research .

Contact us

Now? ,「 cicada Prophet」 Open source online github, And synchronize to domestic gitee On .

• Github Address ：

  https://github.com/Cloud-Discovery/prophet

• Gitee Address ：

  https://gitee.com/cloud-discovery/prophet

We hope , Problems and new requirements in the use process , Can bring it up with us . You can github Submit on file issue, Or join our development communication group , Work with us to build Prophet.

「 cicada Prophet」

Automatic acquisition 、 Tool set for analysis , At present, support for physical machines 、VMware Collection and analysis of environment , In the future, it will be extended to cloud platform resources 、 Storage 、 Network and other resources .