High quality resource sharing

Learning route guidance （ Click unlock ）	Knowledge orientation	Crowd positioning
🧡 Python Actual wechat ordering applet 🧡	Progressive class	This course is python flask+ Perfect combination of wechat applet , From the deployment of Tencent to the launch of the project , Create a full stack ordering system .
Python Quantitative trading practice	beginner	Take you hand in hand to create an easy to expand 、 More secure 、 More efficient quantitative trading system

Doris Ansible Use guide

Apache Doris Introduce

Apache Doris Is a modern MPP Analytical database products . Only sub second response time is needed to get the query results , Effectively support real-time data analysis .Apache Doris The distributed architecture is very simple , Easy operation and maintenance , And can support 10PB The very large data set above .

Apache Doris It can meet a variety of data analysis needs , For example, fixed history report , Real time data analysis , Interactive data analysis and exploratory data analysis . Make your data analysis easier and more efficient ！

Doris Ansible brief introduction

​ Doris_ansible Is based on ansible Built Doris High availability cluster lightweight automatic operation and maintenance tool , It can be easily and quickly pulled up with one button locally Doris High availability cluster , And the start and stop of the cluster 、 Promotion and degradation 、 Expansion and contraction can be realized by a simple command , Enables users to focus on Doris Strong analytical ability , Instead of spending effort on operation and maintenance . In the future, more automation functions will be added . Such as monitoring 、 A series of functions such as operation and maintenance patrol inspection .

Doris Ansible Use steps

Pre base

1. install ansible

yum install -y ansible

2. download Doris_Ansible

git clone https://github.com/mklzl/doris_ansible

3. Cluster planning （ No need to configure , For reference only ）

# fe
master : 192.168.1.241
follower1: 192.168.1.239
follower2: 192.168.2.243

# be
backend1: 192.168.1.239
backend2: 192.168.1.241
backend3: 192.168.1.243

# broker
broker1: 192.168.1.239
broker2: 192.168.1.241
broker3: 192.168.1.243

#  Installation version and software package location 
/home/doris\_ansible/PALO-0.15.1-rc09-binary.tar.gz

#  The installation directory 
/home/doris\_ansible

#  Upgrade and downgrade the path of the software package 
/home/doris\_ansible/PALO-0.15.3-
release-binary.tar.gz

Start using

1. Configure cluster initialization configuration

• Assign cluster roles hosts

vi /etc/ansible/hosts（cluster1 Corresponding Doris Cluster of clusters id, Multiple different cluster configurations can be deployed here , Startup time , Specify the corresponding cluster ID You can operate the corresponding cluster ）

  ##  colony cluster1 Machines involved in ip
  [cluster1.doris\_hosts]
  192.168.1.239
  192.168.1.241
  192.168.1.243

  ## colony cluster1 in fe Of the machine ip
  [cluster1.frontends]
  192.168.1.239
  192.168.1.241
  192.168.1.243

  ## colony cluster1 in master Where the node is located ip
  [cluster1.master]
  192.168.1.241

  ## colony cluster1 in follower nodal ip
  [cluster1.follower]
  192.168.1.239
  192.168.1.243

  ## colony cluster1 in be nodal ip
  [cluster1.backends]
  192.168.1.239
  192.168.1.241
  192.168.1.243
  
  ##  To expand and shrink the volume fe Where ip
  [cluster1.scale\_fe]
  192.168.1.239

  ##  To expand and shrink the volume be Where ip
  [cluster1.scale\_be]
  192.168.1.239
   
  ##  To expand and shrink the volume broker Where ip
  [cluster1.scale\_broker]
  192.168.1.239

  ## colony cluster1 in broker Where the node is located ip
  [cluster1.brokers]
  192.168.1.239
  192.168.1.241
  192.168.1.243

• Configuration initialization file

vi ./conf/cluster1.yml( Configure the cluster here cluster1 Corresponding ip Group )

---
follower: [192.168.1.239,192.168.1.243]
backends: [192.168.1.239,192.168.1.241,192.168.1.243]
brokers: [192.168.1.239,192.168.1.241,192.168.1.243]
master: 192.168.1.241

vi ./conf/setup\_vars.yml
---
#  Production environment fe.conf Location path .
#  If there is no special configuration , It is recommended to use... In the installation package fe.conf, Please configure according to the situation of the machine priority\_networks
fe\_conf\_path: /home/doris\_ansible/fe.conf

# Production environment be.conf Location path .
#  If there is no special configuration , It is recommended to use... In the installation package be.conf, Please configure according to the situation of the machine priority\_networks
be\_conf\_path: /home/doris\_ansible/be.conf

# heartbeat\_service\_port, Please and be.conf Medium heartbeat\_service\_port Configuration is consistent 
heartbeat\_service\_port: 9050


# edit\_log\_port, Please and fe.conf Medium edit\_log\_port Configuration is consistent 
edit\_log\_port: 9010

# query\_port, Please and fe.conf Medium query\_port Configuration is consistent 
query\_port: 9030

# broker\_ipc\_port, Please and apache\_hdfs\_broker.conf Medium broker\_ipc\_port bring into correspondence with 
broker\_ipc\_port: 8000

#  To be installed doris The path of the compressed package , Please write the absolute path 
doris\_filepath: /home/doris\_ansible/PALO-0.15.1-rc09-binary.tar.gz

# doris The location where the compressed package should be decompressed and installed 
dest\_filepath: /home/doris\_ansible

# After decompressing ,doris Installation directory 
doris\_home: /home/doris\_ansible/PALO-0.15.1-rc09-binary

#  machine java\_home Location path , Please make sure that all machines are consistent 
java\_home: /usr/java/jdk1.8.0\_131

2. Initialize cluster

#cluster=cluster1  Is the specified operation on the corresponding cluster 
ansible-playbook -e "cluster=cluster1" ./core/setup.yml 

3. Adding roles

ansible-playbook -e "cluster=cluster1" ./core/add_roles.yml

4. Start and stop the cluster

# One click to stop the corresponding cluster 
ansible-playbook -e "cluster=cluster1" ./core/stop_all.yml

# Start the corresponding cluster with one click 
ansible-playbook -e "cluster=cluster1" ./core/start_all.yml

5. Cluster lifting stage

• Configuration set group upgrade and downgrade configuration

vi ./conf/upgrade\_vars.yml
---
# The path of the compressed package that needs to be rolled back or upgraded 
newdoris\_filepath: /home/doris\_ansible/PALO-0.15.3-release-binary.tar.gz

# Compressed package decompression path 
newdoris\_destpath: /home/doris\_ansible

# After decompression doirs\_home
newdoris\_home: /home/doris\_ansible/PALO-0.15.3-release-binary

#java\_home Location path 
java\_home: /usr/java/jdk1.8.0\_131

• Carry out lifting operation

ansible-playbook -e "cluster=cluster1" ./core/upgrade.yml

6. Cluster expansion and contraction

1. Configure cluster expansion and contraction capacity

vi /etc/ansible/hosts  ( Configure the role to expand or shrink hosts, You can configure multiple hosts)
##  To expand and shrink the volume fe Where ip
  [cluster1.scale\_fe]
  192.168.1.239

  ##  To expand and shrink the volume be Where ip
  [cluster1.scale\_be]
  192.168.1.239
   
  ##  To expand and shrink the volume broker Where ip
  [cluster1.scale\_broker]
  192.168.1.239

2. FE Expansion and contraction capacity

  vi ./conf/scale_fe_vars.yml
  ---
 ##  To expand and shrink the volume fe
  frontends: 192.168.1.239
  
 ##( Multiple ip Using arrays )
  frontends: [192.168.1.239,192.168.1.241]

3. BE Expansion and contraction capacity

 vi ./conf/scale_be_vars.yml
 ---
 ##  To expand and shrink the volume be
  backends: 192.168.1.239
  
 ##( Multiple ip Using arrays )
  backends: [192.168.1.239,192.168.1.241]

4. Broker Expansion and contraction capacity

  vi ./conf/scale_broker_vars.yml
  ---
 ##  To expand and shrink the volume broker
  brokers: 192.168.1.239
  
 ##( Multiple ip Using arrays )
  brokers: [192.168.1.239,192.168.1.241]

5. Perform expansion and contraction

#  Capacity expansion 
  #  Capacity expansion broker
  	ansible-playbook -e "cluster=cluster1 action=out" ./core/scale_broker.yml
  #  Capacity expansion be
  	ansible-playbook -e "cluster=cluster1 action=out" ./core/scale_be.yml
  #  Capacity expansion fe
  	ansible-playbook -e "cluster=cluster1 action=out" ./core/scale_fe.yml
#  Shrinkage capacity 
  # Shrinkage capacity broker
    ansible-playbook -e "cluster=cluster1 action=out" ./core/scale_broker.yml
  # Shrinkage capacity be
    ansible-playbook -e "cluster=cluster1 action=out" ./core/scale_be.yml
  # Shrinkage capacity fe
    ansible-playbook -e "cluster=cluster1 action=out" ./core/scale_fe.yml