Catalog

One .reync

1、rsync The server

2、 Synchronization mode

3.rsync Command format and parameters

rsync Two ways of local replication

4. Two representations of the configuration source

5. Case study

Two 、inotify

1、 brief introduction

2、inotifywait

3.rsync+inotify Case study

combination shell Script deployment

summary

---

One .reync

1、rsync The server

• rsync It's an open source 、 fast 、 Multifunctional 、 It is an excellent tool to realize full and incremental local or remote data synchronous backup . And the attribute information of the original data can not be changed , Realize the backup and migration of data .
• rsync The software is suitable for unix/linux/windows And so on
• rsync Is a fast and very similar file copying tool . It can instinctively copy , Remote Replication , Or remote daemon replication , It provides a large number of parameters to control all aspects of its behavior , It also allows very flexible ways to realize file transfer and replication
• With its delta-transfer Algorithm famous .
• rsync Listening port ：873
• rsync Operation mode ：C/S

2、 Synchronization mode

• 【1】 Full volume backup ：
  • All the original data is transmitted
  • Send the original document and the new document together
  • Copy in full , Low efficiency
• 【2】 Incremental backup
  • Before transmitting data, compare the data you have with the data I have through some algorithms , Transmit different data through the network
  • Incremental replication , Efficient

3.rsync Command format and parameters

rsync  [ Options ]   Original location     Target location 


 Common options 	      explain 
-r	         Recursive mode , Contains all the files in the directory and subdirectories 
-l	         For symbolic link files, still copy as symbolic link files 
-v	         Show details of the synchronization process 
-z	         Compress when transferring files goD
-p	         Keep the permission tag of the file 
-a	         Archiving mode , Recurse and keep object properties , Equate to -rlpt
-t	         Keep the time stamp of the file 
-g	         Keep the group tag of the file （ For super users only ）
-o	         Keep the owner tag of the file （ For super users only ）
-H	         Keep hard link files 
-A	         Retain ACL Attribute information 
-D	         Keep equipment files and other special files 
–-delete    	 Delete files that exist in the target location but not in the original location 
–checksum	 Decide whether to skip files based on the checksums of objects 

rsync Two ways of local replication

1. Do not add... After the original position /
2. Add... After the original position /

 Case study ：
 No addition /：rsync -avz ssl  shiyan  #ssl Not added after /, It means that you will ssl The contents of the directory are copied to shiyan Directory 
 Add /：  rsync -avz ssl/  shiyan  #ssl Add after /, It means that you will ssl Copy the contents of the directory to shiyan Directory 

4. Two representations of the configuration source

 Format 1 ：
 user name @ The host address :: Share module name 
rsync -avz [email protected]::wwwroot /root

 Format two ：
rsync:// user name @ The host address / Share module name 
rsync -avz rsync://[email protected]/wwwroot /root

5. Case study

 host 		IP Address 	 
server		192.168.29.55  rsync The source server 
rsync		192.168.29.33  rsync Target server 

Case deployment

# Old formula , Turn off firewall ....
systemctl stop firewalld.service 
setenforce 0



#rsync Server configuration 
rpm -q rsync
yum -y install rsync

# modify /etc/rsyncd.conf The configuration file 
vim /etc/rsyncd.conf 


uid = nobody					#root
gid = nobody					#root
use chroot = yes				# Locked in the source directory                                                 
address = 192.168.29.33		    # Monitor address 
port 873                     	# Listening port  tcp/udp 873, It can be done by  cat /etc/services | grep rsync  see                                            
log file = /var/log/rsyncd.log	# Log file location                   
pid file = /var/run/rsyncd.pid	# Store process ID File location of                   
hosts allow = 192.168.3.0/24	# The client address allowed to access  ( Access policy in uplink synchronization )
[wwwroot]   					## The first shared module                                                            
path = /var/www/html     		# The actual path to the source directory                                
comment = Document Root of www.ljm.com	
read only = yes             	# Is it read-only                                      
dont comperss = *.gz *.bz2 *.tgz *.zip *.rar *.z  	# File types that are no longer compressed during synchronization       
auth users = backuper           # Authorized account , Multiple accounts are separated by spaces                                  
secrets file = /etc/rsyncd_users.db # Data file for storing account information  

# Such as anonymity , Just put one of them  “auth users”  and  “secrets file” The configuration item can be removed 



# A concise version 

uid = nobody
gid = nobody
use chroot = yes
address = 192.168.29.33
port 873
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
hosts allow = 192.168.29.0/24
[wwwroot]
path = /var/www/html
comment = Document Root of www.ljm.com
read only = yes
dont comperss = *.gz *.bz2 *.tgz *.zip *.rar *.z
auth users = backuper
secrets file = /etc/user.db                         

# Create data file for backup account 
vim /etc/user.db
backuper:abc123		# There is no need to establish a system user with the same name 
chmod 600 /etc/user.db


# Ensure that all users have access to the source directory  /var/www/html  Have read permission 
mkdir -p /var/www/html
chmod +r /var/www/html/
ls -ld /var/www/html/

# start-up  rsync  Service program 
rsync --daemon
netstat -natp | grep rsync

# To shut down 
kill $(cat /var/run/rsyncd.pid)
rm -rf /var/run/rsyncd.pid

# Write a simple html
cd /var/www/html
echo "hello" >> 1.txt

server To configure 

# Sync 
mkdir /abc	
rsync -avz [email protected]::wwwroot /abc/		// Sync , Then enter the password 


# View syncs 
cd /abc
ls

# Set interaction free 
echo "abc123" > /etc/server.pass
chmod 600 /etc/server.pass
rsync -avz --password-file=/etc/server.pass [email protected]::wwwroot /abc
# At this time, you don't have to enter the password to synchronize again 

Two 、inotify

1、 brief introduction

•   You can monitor changes in the file system , And make a notification response
• adjustment inotify Kernel parameters （ Optimize ）
• /etc/sysctl.conf( Kernel parameter configuration file )
• inotifywait： For continuous monitoring , Real time output results
• inotifywatch： For short-term monitoring , Output the results after the task is completed
   

max_queue_events    	# Monitor event queue size 
max_user_instances  	# The maximum number of monitoring instances 
max_user_watches    	# The maximum number of monitoring files per instance   

2、inotifywait

• Format   inotifywait [ Parameters ]

 Common parameters 	     explain 
-m	         Continuous monitoring 
-r	         Recursively monitor all child objects 
-q	         Simplify the output 
-e	         Specify which event types to monitor 

3.rsync+inotify Case study

rsync To configure

#1. Shut down first rsync process 
kill $(cat /var/run/rsyncd.pid)
netstat -natp | grep rsync

#2. modify rsync The configuration file 
vim /etc/rsyncd.conf

uid = root
gid = root
use chroot = yes
address = 192.168.29.33
port 873
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
hosts allow = 192.168.29.33/24
[wwwroot]
path = /var/www/html
comment = Document Root of www.ljm.com
read only = no
dont comperss = *.gz *.bz2 *.tgz *.zip *.rar *.z
auth users = backuper
secrets file = /etc/user.db                         

#3. start-up rsync
rsync --daemon
netstat -natp | grep rsync

#4. To configure inotify Kernel parameters 
vim /etc/sysctl.conf 

fs.inotify.max_queued_events = 32768      # Monitor the time queue , The default is 16384
fs.inotify.max_user_instances = 1024	  # The maximum number of monitoring instances , The default is 128
fs.inotify.max_user_watches = 1048576	  # The maximum number of monitoring files per instance , The default is 8192
# When you want to monitor the directory 、 When the amount of file data is large or changes frequently , It is recommended to increase the parameter value 

sysctl -p   // Refresh 

• server To configure

#1. install inotify
yum -y install gcc gcc-c++ 	// Installation dependency 

# Drag the bag to /opt Under the table of contents 
tar zxvf inotify-tools-3.14.tar.gz -C /opt	

#2. Compilation and installation 
cd /opt/inotify-tools-3.14/
./configure
make -j 4 && make install	

#3. monitor  /abc  Next , Add files 、 Moving files, etc , And track the screen output 
inotifywait -mrq -e modify,create,move,delete /abc
# Then open another terminal , Conduct   establish 、 Delete 、 rename   Wait for the operation , See the changes 
touch 4.txt
touch 5.txt
rm -rf 4.txt
mv 5.txt 6.txt

combination shell Script deployment

#1. Script 
vim /opt/inotify.sh

#!/bin/bash
INOTIFY_CMD="inotifywait -mrq -e create,delete,move,modify,attrib /abc/"
RSYNC_CMD="rsync -apzH --delete --password-file=/etc/server.pass /abc/ [email protected]::wwwroot/"
$INOTIFY_CMD | while read DIRECTORY EVENT FILE 
do
	if [ $(pgrep rsync | wc -l) -le 0 ]; then
		$RSYNC_CMD
	fi
done


#2. weighting 
chmod +x /opt/inotify.sh
chmod +x /etc/rc.d/rc.local
echo "/opt/inotify.sh" >> /etc/rc.d/rc.local


#3. test 
cd /abc
touch 1.txt
# Back to rsync see , It is found that it has been synchronized 

summary

• rsync Is an open source fast backup tool , It can mirror and synchronize the whole directory tree between different hosts , Support incremental backup , And keep links and permissions , And the optimized synchronization algorithm is used , Perform compression before transmission , So it is very suitable for remote backup 、 Image server and other applications .