One 、awk Introduce

Memory usage

[root@localhost ~]# free
s              total        used        free      shared  buff/cache   available
Mem:        1743568      338732      981252        9476      423584     1366680
Swap:      12451824           0    12451824
[root@localhost ~]# head -3 /proc/meminfo
MemTotal:        1743568 kB
MemFree:          981528 kB
MemAvailable:    1366972 kB
[root@localhost ~]#

awk Is a line editor , It can intercept every row and column , Data processing and calculation can be realized .awk Think of each line in the file as a record , The separator recorded in the record is the newline character , Each column is a field The default separator between fields is one or more spaces or tab tabs .

Two 、awk The basic usage of —awk Data extraction function

Test content ：
[root@localhost awk]# cat test.sh
1 the quick beown fox junps over the lazy cat . dog
2 the quick beown fox junps over the lazy cat . dog
3 the quick beown fox junps over the lazy cat . dog
4 the quick beown fox junps over the lazy cat . dog
5 the quick beown fox junps over the lazy cat . dog

1、awk To field （ Column ） Extraction of

Field extraction ： Extract a column of data in a text and print it out
Field related built-in variables
$0 Represents the entire line of text
$1 Represents the first data field in a text line
$2 Represents the second data field in the text line
$N Represents the... In a text line N Data fields
$NF Represents the last data field in a text line

# Print all 
[root@localhost awk]# awk '{print $0}' test.sh
1 the quick beown fox junps over the lazy cat . dog
2 the quick beown fox junps over the lazy cat . dog
3 the quick beown fox          junps over the lazy cat . dog
4 the quick beown fox junps over the lazy cat . dog
5 the quick beown fox junps over the lazy cat . dog

# Print the last column 
[root@localhost awk]# awk '{print $NF}' test.sh
dog
dog
dog
dog
dog

# Print the third column 
[root@localhost awk]# awk '{print $3}' test.sh
quick
quick
quick
quick
quick

2、awk To record （ That's ok ） Extraction of

Record extraction ： Extract a line from a text and print it out
There are two methods to extract records ：a、 Pass line number b、 Through regular matching
Record related built-in variables
NR： Specify line number

# Extract all the contents of the third line 
[root@localhost awk]# awk 'NR==3{print $0}' test.sh
3 the quick beown fox          junps over the lazy cat . dog

2.1 Specify the separator

“-F”

# as follows ： Print the characters in the first column of the first row , With “：” Colon as separator 
[root@localhost awk]# awk -F ":" 'NR==1{print $1}' passwd
root
[root@localhost awk]# awk -F: 'NR==1{print $1}' passwd
root

# as follows ： Print the first... Of the first line 、 3、 ... and 、 Five columns of characters , With “：” Colon as separator 
[root@localhost awk]# awk -F ":" 'NR==1{print $1,$3,$5}' passwd
root 0 root

# Custom output format 
[root@localhost awk]# awk -F ":" 'NR==1{print $1 "-" $3 "-" $5}' passwd
root-0-root

# Output into ：“account:root UID:0 DESC:root”
[root@localhost awk]# awk -F ":" 'NR==1{print "account:"$1, "UID:"$3, "DESC:" $5}' passwd
account:root UID:0 DESC:root

3、awk Priority of program

About awk Program execution priority ,BEGIN Is the highest priority code block , It's execution program Previously executed , There is no need to provide a data source , Because it does not involve the processing of PBC data , Nor does it depend on PROGRAM Code block ;PROGRAM What do you do with data streams , Is a required code block , It is also the default code block , Therefore, the data source must be provided during execution ,end It is the operation after processing the complete data stream , If it is necessary to carry out end Code block , You have to need program Support for , A single cannot execute .

# Priority presentation 
[root@localhost awk]# awk 'BEGIN{print "hello ayitual"}{print $0}END{print "bye ayitula"}' test.sh
hello ayitual
1 the quick beown fox junps over the lazy cat . dog
2 the quick beown fox junps over the lazy cat . dog
3 the quick beown fox          junps over the lazy cat . dog
4 the quick beown fox junps over the lazy cat . dog
5 the quick beown fox junps over the lazy cat . dog

bye ayitula

#begin You can execute without a data source 
[root@localhost awk]# awk 'BEGIN{print "hello ayitual"}'
hello ayitual

# No data source provided , Unable to execute successfully 
[root@localhost awk]# awk '{print "hello ayitual"}'
^C
[root@localhost awk]# awk 'END{print "hello ayitual"}'
^C

3、 ... and 、awk Advanced usage of

awk It's a language , Then it will conform to the characteristics of the language , In addition to defining variables , It can also be transported , Process control .

1、awk Define an array

Array definition ： Array name 【 Indexes 】= value
Define an array array, There are two elements , Namely 100,200, Print array elements .

[root@localhost awk]# awk 'BEGIN{array[0]=100;array[1]=200;print array[0],array[1]}'
100 200

2、awk operation

1. The assignment operation =
2. Comparison operations > >= == < <= !=
3. Mathematical operations + - * / % ** ++ –
4. Logical operations && ||
5. Matching operations ~ !~

example ： Calculate memory usage

# Calculate memory usage 
[root@localhost awk]# head -2 /proc/meminfo |awk 'NR==1{t=$2}NR==2{f=$2;print(t-f)*100/t "%"}'
30.3599%

[root@localhost awk]# seq 1 10 > mun
[root@localhost awk]# awk '$1>5{print $0}' mun
6
7
8
9
10
[root@localhost awk]# awk '$1>=5{print $0}' mun
5
6
7
8
9
10
[root@localhost awk]# awk '$1 !=5{print $0}' mun
1
2
3
4
6
7
8
9
10
# Fuzzy matching 
[root@localhost awk]# awk -F: '$1 ~ "ro"{print $0}' passwd
root:x:0:0:root:/root:/bin/bash
[root@localhost awk]# awk -F: '$1 !~ "ro"{print $0}' passwd
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
systemd-network:x:192:192:systemd Network Management:/:/sbin/nologin
dbus:x:81:81:System message bus:/:/sbin/nologin
polkitd:x:999:998:User for polkitd:/:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
postfix:x:89:89::/var/spool/postfix:/sbin/nologin
mqm:x:998:1000::/var/mqm:/bin/bash
[root@localhost awk]#

3、awk environment variable

Variable	describe
FIELDWIDTHS	A list of numbers separated by spaces , Define the exact width of each data field with spaces
FS	Enter the column separator
OFS	Output column separator
RS	Enter the line separator
ORS	Output line separator

# FIELSWISTHS: Redefine the column width and print , Be careful not to use $0 Promise all , because $0 Is to print all the contents of this line , The fields you define will not be printed 
[root@localhost awk]# awk 'BEGIN{FIELDWIDTHS="5 2 8"}NR==1{print $1,$2,$3}' passwd
root: x: 0:0:root
[root@localhost awk]# awk -F: 'NEGIN{FS=":"}$1 !~ "ro"{print $0}' passwd
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
systemd-network:x:192:192:systemd Network Management:/:/sbin/nologin
dbus:x:81:81:System message bus:/:/sbin/nologin
polkitd:x:999:998:User for polkitd:/:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
postfix:x:89:89::/var/spool/postfix:/sbin/nologin
mqm:x:998:1000::/var/mqm:/bin/bash

[root@localhost awk]# awk 'BEGIN{FS=":";OFS="-"}NR==1{print $1,$3,$5}' passwd
root-0-root

[root@localhost awk]# awk 'BEGIN{RS=""}{print $1,$2,$3,$4,$5}' mun
1 2 3 4 5

4、 Process control

1、if Judgment statement
2、for Loop statement
3、while Loop statement
4、do…while sentence
5、 Cycle control

if Judgment statement

[root@localhost awk]# awk '{if($1>5)print $0}' mun
6
7
8
9
10
[root@localhost awk]# awk '{
    
> if ($1<5)
>   print $1*2
> else
>   print $1/2
> }' mun 2 4 6 8 2.5 3 3.5 4 4.5 5 [root@localhost awk]# awk '{
    if ($1<5) print $1*2;else print $1/2}' mun
2
4
6
8
2.5
3
3.5
4
4.5
5

# Each row and column add up 
[root@localhost awk]# awk -v 'sum=0' '{sum+=$1} END{print sum}' mun
55
[root@localhost awk]# echo 60 50 100 > num2
[root@localhost awk]# echo 150 30 10 >> num2
[root@localhost awk]# echo 70 100 40 >> num2
[root@localhost awk]# cat num2
60 50 100
150 30 10
70 100 40

for Loop statement

# Add each column 
[root@localhost awk]# awk '{
    
> for (i=1;i<4;i++)
>   sum+=$i
> print sum}' num2 210 400 610 [root@localhost awk]# awk '{
    sum=0;for (i=1;i<4;i++){
    sum+=$i}print sum}' num2
210
190
210

while Loop statement

[root@localhost awk]# awk '{
    
> sum=0
> i=1
> while (i<4) {
    
> sum+=$i
> i++
> }
> print sum}' num2 210 190 210 [root@localhost awk]# awk '{
    sum=0;i=1;while (i<4) {
    sum+=$i;i++}print sum}' num2
210
190
210

do…while Loop statement

# When the first value of each line is less than 150 Add the second value 
[root@localhost awk]# awk '{
    
> sum=0
> i=1
> do{
    
> sum+=$i
> i++
> }while (sum<150)
> print sum
> }' num2 210 150 170 [root@localhost awk]# awk '{
    sum=0;i=1;do{
    sum+=$i;i++}while (sum<150)print sum}' num2
210
150
170

Loop control statement

break Out of the loop , Continue to execute subsequent loop statements

# The sum of the values in each row is greater than 150 Stop accumulating when 
[root@localhost awk]# awk '{sum=0;i=1;while (i<4) {sum+=$i;if (sum>150){break}i++}print sum}' num2
210
180
170

5、 Tips

Print lines
[root@localhost awk]# awk ‘END{print NR}’ test.sh
6
Print the last column
[root@localhost awk]# awk ‘END{print $0}’ test.sh

Print the number of columns
[root@localhost awk]# awk ‘END{print NF}’ test.sh
0