文本三剑客：awk（截取）、grep(过滤)、sed(替换)

目录

1、grep的基本使用命令

2、grep使用正则表达式

2.1 正则的定义

2.2 通配符

1、grep的基本使用命令

grep：过滤 通用的正则表达式分析程序

grep、egrep、fgrep：做匹配来过滤的

pattern 模式---》模板   可以接受一个正则表达式

用途：在文件中查找并显示包含指定字符串的行。

格式：grep [选项].... 模式 目标文件

grep -i :查找时忽略大小写。

[[email protected] opt]#  ps -ef |grep -i network
root       6144      1  0 10:57 ?        00:00:00 /usr/sbin/NetworkManager --no-daemon
root       9826   8828  0 22:26 pts/2    00:00:00 grep --color=auto -i network

grep -v 反转查找 过滤出没有的。

[[email protected] ~]# grep -v sbin /etc/passwd   #过滤出不含sbin的行
root:x:0:0:root:/root:/bin/bash
angel:x:1000:1000::/home/angel:/bin/bash
aj:x:1001:1001::/home/aj:/bin/bash
qiantao:x:1002:1002::/home/qiantao:/bin/bash
yalin:x:1003:1003::/home/yalin:/bin/bash
jj:x:1004:1004::/home/jj:/bin/bash
.....部分截取

[[email protected] ~]# grep sbin /etc/passwd   #过滤出含有sbin的行。
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt

grep -n :显示符合模式要求的行号。

[[email protected] opt]# grep -n root /etc/passwd
1:root:x:0:0:root:/root:/bin/bash
10:operator:x:11:0:operator:/root:/sbin/nologin
27:apache:x:48:48:Apache:/opt/rh/httpd24/root/usr/share/httpd:/sbin/nologin
[[email protected] opt]# grep -o sbin /etc/passwd

grep -o：只显示匹配的行的内容。

[[email protected] opt]# grep -o sbin /etc/passwd
sbin
sbin
sbin
sbin
sbin
sbin
sbin
sbin
sbin
sbin
sbin
sbin

grep -r：递归搜索所有文件以及目录下的所有子文件。（当前目录下）

[[email protected] opt]# cd /etc/sysconfig/
[[email protected] sysconfig]# pwd
/etc/sysconfig
[[email protected] sysconfig]# grep en *
anaconda:# This file has been generated by the Anaconda Installer 21.48.22.147-1
authconfig:FAILLOCKARGS="deny=4 unlock_time=1200"
grep: cbq: 是一个目录
grep: console: 是一个目录
cpupower:CPUPOWER_START_OPTS="frequency-set -g performance"
cpupower:CPUPOWER_STOP_OPTS="frequency-set -g ondemand"
crond:# CRONDARGS= :  any extra command-line startup arguments for crond
ebtables-config:# If EBTABLES_BINARY_FORMAT="no" then restoring the firewall rules
ebtables-config:# Save current firewall rules on stop.
ebtables-config:# Save current firewall rules on restart.
ebtables-config:# Save rule counters when saving a kernel table to a file. If the
ebtables-config:# rule counters were saved, they will be restored when restoring the table.
grub:GRUB_CMDLINE_LINUX="crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet"
init:# terminal sequence to move to that column. You could change this
init:MOVE_TO_COL="echo -en \\033[${RES_COL}G"
init:# terminal sequence to set color to a 'success' color (currently: green)
init:SETCOLOR_SUCCESS="echo -en \\033[0;32m"
init:# terminal sequence to set color to a 'failure' color (currently: red)
init:SETCOLOR_FAILURE="echo -en \\033[0;31m"
init:# terminal sequence to set color to a 'warning' color (currently: yellow)
init:SETCOLOR_WARNING="echo -en \\033[0;33m"
init:# terminal sequence to reset to the default color.
init:SETCOLOR_NORMAL="echo -en \\033[0;39m"

加-r也能查找，递归查找当前目录下的所有文件里面包含里面目录下的子文件，匹配含有ens的行。

[[email protected] sysconfig]# grep ens * -r  
kdump:#What is the images extension.  Relocatable kernels don't have one
network-scripts/ifdown-eth:# it under the terms of the GNU General Public License, version 2,
network-scripts/ifdown-eth:# GNU General Public License for more details.
network-scripts/ifdown-eth:# You should have received a copy of the GNU General Public License
network-scripts/ifdown-tunnel:# it under the terms of the GNU General Public License, version 2,
network-scripts/ifdown-tunnel:# GNU General Public License for more details.
network-scripts/ifdown-tunnel:# You should have received a copy of the GNU General Public License
network-scripts/ifup-eth:# it under the terms of the GNU General Public License, version 2,
network-scripts/ifup-eth:# GNU General Public License for more details.
network-scripts/ifup-eth:# You should have received a copy of the GNU General Public License
network-scripts/ifup-eth:# Bonding initialization. For DHCP, we need to enslave the devices early,
network-scripts/ifup-eth:    # add the bits to setup the needed post enslavement parameters
network-scripts/ifup-ipv6:# Enable IPv6 RFC3041 privacy extensions if desired
network-scripts/ifup-tunnel:# it under the terms of the GNU General Public License, version 2,
network-scripts/ifup-tunnel:# GNU General Public License for more details.
network-scripts/ifup-tunnel:# You should have received a copy of the GNU General Public License
network-scripts/ifup-wireless:# it under the terms of the GNU General Public License, version 2,
network-scripts/ifup-wireless:# GNU General Public License for more details.
network-scripts/ifup-wireless:# You should have received a copy of the GNU General Public License
network-scripts/network-functions:            awk '{ print toupper(gensub(/.*link\/[^ ]* ([[:alnum:]:]*).*/,

grep sbin -n /etc/passwd #找到/etc/passwd里面的sbin 所在行，并显示行号

[[email protected] sysconfig]# grep sbin -n /etc/passwd
2:bin:x:1:1:bin:/bin:/sbin/nologin
3:daemon:x:2:2:daemon:/sbin:/sbin/nologin
4:adm:x:3:4:adm:/var/adm:/sbin/nologin
5:lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
6:sync:x:5:0:sync:/sbin:/bin/sync
7:shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
8:halt:x:7:0:halt:/sbin:/sbin/halt

grep -A：找到匹配的行以及后几行

grep -B：输出匹配的行以及前几行

# -A 2：找到ftp 所在的第二行以及后几行，-n：显示行号
[[email protected] sysconfig]# grep ftp -n -A 2 /etc/passwd  
12:ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
13-nobody:x:99:99:Nobody:/:/sbin/nologin
14-systemd-network:x:192:192:systemd Network Management:/:/sbin/nologin



#-B 2；找到第二行以及前几行
[[email protected] sysconfig]# grep ftp -n -B 2 /etc/passwd  
10-operator:x:11:0:operator:/root:/sbin/nologin
11-games:x:12:100:games:/usr/games:/sbin/nologin
12:ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin

2、grep使用正则表达式

2.1 正则的定义

使用一些特殊符号+字母和数字按照某个正确的规则组合成一个公式来表示某个意思这就是正则表达式。

有什么用：用来查找，根据正则表达式表达出来的意思，去海量的文本里面查找符合这个意思的内容。

很多命令都有查找功能：vim、grep、sed、awk等。

grep过滤，通用的正则表达式分析程序。

正则表达式：regular expression  ---re。过滤文本的规则。

2.2 通配符

 [abc]:表示abc中的任意一个字符。

[^abc]:不含abc中的任意一个。

通配符：在shell里的叫法。

* ? .的含义：

元字符: 正则表达式。

元字符：用来描述其他字符，有特殊意义的字符。

meta character 

分组捕获：分组向后引用，（）括起来的表示分组 \1 \2b表示第一个分组和第二个分组获取到的内容。

正则：

1、基本正则：元字符比较少 ^ $ ? *

2、扩展正则 ： | + {}

grep -E = egrep

grep [OPTIONS] PATTERN [FILE...]

grep [OPTIONS] [-e PATTERN | -f FILE] [FILE...]

grep, egrep fgerp - print lines matching a pattern

匹配模式在行里查找，然后输出--》输出匹配模式的行。

模式（pattern）是什么？

模式是grep里包含了正则表达式的一个模型（字符串、样例）

安装某个模型根据正则表达式去构成的一个公式。--表达一个意思

例如：匹配以wangshuai开头的三次，{0-9}表示数字0-9。：egrep “^wangshuai{3}{0-9}+ " ws.txt

写正则表达式。其实就是找共性的特点，也就是找规律。

字符串[email protected]字符串2.字符串3：[0-Z_][email protected][0-Z]+\.[a-Z]+

查找邮箱：

[[email protected] lianxi]# cat mail.txt

[email protected]  fdffaf1212

[email protected]  8989

[email protected] fjdkfj

[[email protected] lianxi]# egrep "[0-Z_][email protected][0-Z]+\.[a-Z]+" mail.txt
[email protected]  fdffaf1212
[email protected]  8989
[email protected] fjdkfj

 egrep -o ：只显示匹配的字符串。

[[email protected] lianxi]# egrep -o  "[0-Z_][email protected][0-Z]+\.[a-Z]+" mail.txt
[email protected]
[email protected]
[email protected]

xargs：①将列转为一行输出；②把前一条传入的命令作为后一条的参数使用。

[[email protected] lianxi]# egrep -o  "[0-Z_][email protected][0-Z]+\.[a-Z]+" mail.txt|xargs
[email protected] [email protected] [email protected]

grep ^[^a-zA-Z0-9_] grep_test.txt 显示不以字母、数字、下划线开头的行。

grep -v ^[a-zA-Z0-9_] grep _test.txt 不显示出显示不以字母、数字、下划线开头的行。

[[email protected] lianxi]# echo "a1a2"|egrep "([a-z][0-9])"
a1a2
[[email protected] lianxi]# echo "a1a2"|egrep "([a-z][0-9])\1"
[[email protected] lianxi]# echo "a1a1"|egrep "([a-z][0-9])\1"
a1a1
[[email protected] lianxi]# echo "a1a1bbccdd"|egrep "([a-z][0-9])\1bb([a-z])\2"
a1a1bbccdd
①egrep "b.*" grep-test.txt
②egrep "#.*" grep-test.txt 
③egrep "#.{2}*" grep-test.txt

前一项：

前一项
[[email protected] lianxi]# egrep "([a-z][0-9]) +" grep-test.txt 匹配一次及以上，括号是把他们作为一个整体
a1b2
[[email protected] lianxi]# grep ^[0-9] grep-test.txt  过滤出以数字开头的行^[0-9]
123
22
[[email protected] lianxi]# grep "[^0-9]" grep-test.txt       [^0-9]过滤出不以数字开头的行
[[email protected] lianxi]# grep ^# grep-test.txt   过滤出grep-test.txt 文本中以“^’开头的内容
#abc
[[email protected] lianxi]# grep b$ grep-test.txt   过滤出grep-test.txt 文本中以”b“结尾的内容
 #bbb
 #bbb
[[email protected] lianxi]# grep [a-z] grep-test.txt  过滤出grep-test.txt 文本中含有字母的内容
[[email protected] lianxi]# grep [a-Z] grep-test.txt  过滤出grep-test.txt 文本中大小写字母的内容
[[email protected] lianxi]# grep "[^a]" grep-test.txt  过滤出不以a开头的行
[[email protected] lianxi]# grep "[^0-9]" grep-test.txt
[[email protected] lianxi]# grep "[rs]" /etc/passwd   过滤出/etc/passwd中含有r或者s的行
[[email protected] lianxi]# grep "[^rs]" /etc/passwd    不含r或s的行

[[email protected] lianxi]# grep -E "1[0-9]{8,10}" grep-test.txt  -E 支持正则表达式， {8，10}8表示匹配前一项
18569628804

[[email protected] lianxi]# egrep "#.{2}"grep-test.txt   匹配#开头的两次
[[email protected] lianxi]# egrep "#.*"grep-test.txt
[[email protected] lianxi]# egrep "b.*" grep-test.txt
[[email protected] lianxi]# egrep "b*[a-z]" grep-test.txt
[[email protected] lianxi]# egrep "^#|^[0-9]" grep-test.txt    正则或： 匹配出以#开头或者数字开头的行
#abc
123
22
18569628804

\b 表示匹配词边界，单词的边界

\B 表示匹配非词边界

[[email protected] lianxi]# echo "#westaa" |egrep "\bwest"
#westaa
[[email protected] lianxi]# echo "#west" |egrep "west\b"
#west
[[email protected] lianxi]# echo "#westaa" |egrep "west\b"
[[email protected] lianxi]# echo "#westaa" |egrep "\bwest\B"
#westaa
[[email protected] lianxi]# echo "#westaa" |egrep "\Bwest\B"