Shell编程牛客网和Leetcode

shell部分的练习在牛客网有34题，在Leetcode有4题，总体来说难度不大，熟练就好。

牛客部分：

写一个 bash脚本以输出一个文本文件 Nowcoder.txt中的行数
示例:
假设 Nowcoder.txt 内容如下：

#include <iostream>
using namespace std;
int main()
{
    int a = 10;
    int b = 100;
    cout << "a + b:" << a + b << endl;
    return 0;
}

你的脚本应当输出：
9

#!/bin/bash
# basic
方法1 cat Nowcoder.txt | wc -l
# grep
方法2 grep -c '.*' Nowcoder.txt
方法3 grep -c '' Nowcoder.txt
# sed
方法4 sed -n '$=' Nowcoder.txt
# awk
方法5 awk '/.*/{i++}END{print i}' Nowcoder.txt
方法6 awk '//{i++}END{print i}' Nowcoder.txt
方法7 awk 'END{print NR}' Nowcoder.txt 
方法8 awk '{print NR}' Nowcoder.txt | tail -n1
方法9 awk '//' Nowcoder.txt | wc -l
方法10 awk -F: '/.*/' Nowcoder.txt | wc -l

2.打印文件的最后5行

描述

经常查看日志的时候，会从文件的末尾往前查看，于是请你写一个 bash脚本以输出一个文本文件 Nowcoder.txt中的最后5行
示例:
假设 Nowcoder.txt 内容如下：

#include<iostream>
using namespace std;
int main()
{
int a = 10;
int b = 100;
cout << "a + b:" << a + b << endl;
return 0;
}

你的脚本应当输出：

int a = 10;
int b = 100;
cout << "a + b:" << a + b << endl;
return 0;
}

# basic
方法1 cat Nowcoder.txt | tail -n5
方法2 tail -n5 Nowcoder.txt
# awk
方法3 awk -F: 'BEGIN{i=1}{arg[i]=$0;i++}END{for(j=NR-4;j<=NR;j++)print arg[j]}'

3.输出7的倍数

描述

写一个 bash脚本以输出数字 0 到 500 中 7 的倍数(0 7 14 21...)的命令

#!/bin/bash
# basic
方法1 seq 0 7 500
# for((i=0;i<=500;i++))
# do
#     if(($i%7==0))
#         then 
#         echo $i
#     fi
# done
方法2 sum=0
# while [ $sum -le 500 ]
# do
#     echo $sum
#     sum=$[$sum+7]
# done
方法3 awk
awk -F: 'BEGIN{
for(i=0;i<=500;i++)
    if(i%7==0)
        print i
}'

4.输出第5行的内容

描述

写一个 bash脚本以输出一个文本文件 Nowcoder.txt 中第5行的内容。

示例:
假设 Nowcoder.txt 内容如下：
welcome
to
Nowcoder
this
is
shell
code

你的脚本应当输出：
is

#!/bin/bash
# basic
方法1 head -n5 Nowcoder.txt | tail -n1
方法2 i=1
# for j in `cat Nowcoder.txt`
# do
#     if [[ $i -eq 5 ]]
#         then
#             echo $j
#     fi
#     i=$[$i+1]
# done
方法3 i=1
while read p
do
    if [[ $i -eq 5 ]]
        then
            echo $p
    fi
    i=$[$i+1]
done < Nowcoder.txt
# sed
方法4 sed -n '5p' Nowcoder.txt
# awk
方法5 awk -F: 'NR==5{print $0}' Nowcoder.txt

5.打印空行的行号

描述

写一个 bash脚本以输出一个文本文件 Nowcoder.txt中空行的行号,可能连续,从1开始

示例:
假设 Nowcoder.txt 内容如下：

a
b

c

d

e


f

你的脚本应当输出：
3
5
7
9
10

#!/bin/bash
# basic
方法1 i=1
# while read p
# do
#     if [[ $p == '' ]]
#         then 
#         echo $i
#     fi
#     i=$[$i+1]
# done < Nowcoder.txt
# grep
方法2 grep -n '^$' Nowcoder.txt | awk -F: '{print $1}'
方法3 grep -n '^$' Nowcoder.txt | sed 's#:##g'
# sed
方法4 sed -n '/^$/=' Nowcoder.txt
# awk
方法5 awk -F: '$0=="" {print NR}' Nowcoder.txt
方法6 awk -F: '/^$/ {print NR}' Nowcoder.txt
方法7 awk -F: '{if($0=="") {print NR}}' Nowcoder.txt

6.去掉空行

描述

写一个 bash脚本以去掉一个文本文件 Nowcoder.txt中的空行
示例:
假设 Nowcoder.txt 内容如下：

abc

567


aaa
bbb



ccc

你的脚本应当输出：
abc
567
aaa
bbb
ccc

#!/bin/bash
# basic
方法1 while read p
# do
#     if [[ $p == '' ]]
#         then 
#         continue
#     fi
#     echo $p
# done < Nowcoder.txt
方法2 for i in `cat Nowcoder.txt`
# do
#     echo $i
# done
方法3 cat Nowcoder.txt | awk NF
方法4 cat Nowcoder.txt | awk -F: '{if($0!="") {print $0}}'
方法5 cat Nowcoder.txt | awk -F: '!/^$/ {print $0}' 
# grep
方法6 grep -v '^$' Nowcoder.txt
方法7 grep -E '[a-zA-Z0-9]+' Nowcoder.txt
# sed
方法8 sed -r '/^$/d' Nowcoder.txt
方法9 sed -nr '/^$/!p' Nowcoder.txt
# awk
方法10 awk NF Nowcoder.txt
方法11 awk -F: '{if($0!="") {print $0}}' Nowcoder.txt
方法12 awk -F: '!/^$/ {print $0}' Nowcoder.txt

7.打印字母数小于8的单词

描述

写一个 bash脚本以统计一个文本文件 Nowcoder.txt中字母数小于8的单词。

示例:
假设 Nowcoder.txt 内容如下：
how they are implemented and applied in computer

你的脚本应当输出：
how
they
are

and applied

in

说明:
不要担心你输出的空格以及换行的问题

#!/bin/bash
# basic
方法1 for p in `cat Nowcoder.txt`
# do
#     if [[ $(echo $p | wc -L) -lt 8 ]]
#         then
#             echo $p
#     fi
# done 
方法2 for p in `cat Nowcoder.txt`
# do
#     if [[ $(echo $p | wc -m) -lt 9 ]]
#         then
#             echo $p
#     fi
# done 
方法3 for p in `cat Nowcoder.txt`
# do
#     if [[ ${#p} -lt 8 ]]
#         then
#             echo $p
#     fi
# done 
方法4 cat Nowcoder.txt | xargs -n1 | while read p
do
    if [[ ${#p} -lt 8 ]]
        then echo $p
    fi
done
#awk
方法5 awk '{for(i=1;i<=NF;i++) if(length($i)<8) print $i}' Nowcoder.txt

8.统计所有进程占用内存大小的和

描述

假设 Nowcoder.txt 内容如下：
root 2 0.0 0.0 0 0 ? S 9月25 0:00 [kthreadd]
root 4 0.0 0.0 0 0 ? I< 9月25 0:00 [kworker/0:0H]
web 1638 1.8 1.8 6311352 612400 ? Sl 10月16 21:52 test
web 1639 2.0 1.8 6311352 612401 ? Sl 10月16 21:52 test
tangmiao-pc 5336 0.0 1.4 9100240 238544 ?? S 3:09下午 0:31.70 /Applications

以上内容是通过ps aux | grep -v 'RSS TTY' 命令输出到Nowcoder.txt文件下面的
请你写一个脚本计算一下所有进程占用内存大小的和:

#!/bin/bash
# awk
方法1 awk -F" " '{sum=sum+$6}END{print sum}' Nowcoder.txt
方法2 awk -F" " 'BEGIN{sum=0}{sum=sum+$6}END{print sum}' Nowcoder.txt
方法3 awk 'BEGIN{sum=0}{sum=sum+$6}END{print sum}' Nowcoder.txt

9.统计每个单词出现的个数

描述

写一个 bash脚本以统计一个文本文件 Nowcoder.txt 中每个单词出现的个数。

为了简单起见，你可以假设：
Nowcoder.txt只包括小写字母和空格。
每个单词只由小写字母组成。
单词间由一个或多个空格字符分隔。

示例:
假设 Nowcoder.txt 内容如下：
welcome Nowcoder
welcome to Nowcoder
Nowcoder
你的脚本应当输出（以词频升序排列）：
to 1
welcome 2
Nowcoder 3

说明:
不要担心个数相同的单词的排序问题，每个单词出现的个数都是唯一的。

#!/bin/bash
#awk
awk -F" " '{for(i=1;i<=NF;i++){arr[$i]++}}END{for(j in arr)print j, arr[j]}' Nowcoder.txt | sort -nk2

10.第二列是否有重复

描述

给定一个 Nowcoder.txt文件，其中有3列信息，如下实例，编写一个shell脚本来检查文件第二列是否有重复，且有几个重复，并提取出重复的行的第二列信息：
实例：
20201001 python 99
20201002 go 80
20201002 c++ 88
20201003 PHP 77
20201001 go 88
20201005 shell 89
20201006 java 70
20201008 c 100
20201007 java 88
20201006 go 97

结果：
2 java
3 go

#!/bin/bash
# awk
awk -F" " '{array[$2]++}END{
    for(i in array)
        if(array[i] > 1)
            print array[i], i}' Nowcoder.txt

11.转置文件的内容

描述

写一个 bash脚本来转置文本文件 Nowcoder.txt中的文件内容。

为了简单起见，你可以假设：
你可以假设每行列数相同，并且每个字段由空格分隔

示例:
假设 Nowcoder.txt 内容如下：
job salary
c++ 13
java 14
PHP 12

你的脚本应当输出（以词频升序排列）：
job c++ java PHP
salary 13 14 12

#!/bin/bash
#awk
# awk '{print $1}' Nowcoder.txt
# awk '{print $2}' Nowcoder.txt
awk -F " " '{for(i=1;i<=NF;i++){
    if(NR==1){
        array[i]=$i
    }else{
        array[i]=array[i] $i
    }
}}END{
    for(j=1;j<=NF;j++){
        print array[j]
    }
}' Nowcoder.txt

12.打印每一行出现的数字个数

描述

写一个 bash脚本以统计一个文本文件 Nowcoder.txt中每一行出现的1,2,3,4,5数字个数并且要计算一下整个文档中一共出现了几个1,2,3,4,5数字数字总数。

示例:
假设 Nowcoder.txt 内容如下：
a12b8
10ccc
2521abc
9asf
你的脚本应当输出：
line1 number: 2
line2 number: 1
line3 number: 4
line4 number: 0
sum is 7

说明:
不要担心你输出的空格以及换行的问题

#!/bin/bash
#basic
方法1 cnt=1
# sum=0
# while read p
# do
#     a=`echo $p | grep -oE "[1-5]" | wc -l`
#     echo line$cnt number:$a
#     cnt=$[$cnt+1]
#     sum=$[$sum+$a]
# done 
# echo sum is $sum
# < Nowcoder.txt
方法2 cnt=1
# sum=0
# while read p
# do
#     a=`echo $p | sed 's#[a-Z|6-9|0]##g' | awk '{print length($0)}'`
#     echo line$cnt number:$a
#     cnt=$[$cnt+1]
#     sum=$[$sum+$a]
# done 
# echo sum is $sum
# < Nowcoder.txt

# awk
方法3 awk '{
gsub(/[^1-5]/,"",$0)
print "line"NR" number: "length($0)
sum+=length($0)
}END{print "sum is "sum}' Nowcoder.txt

13.去掉所有包含this的句子

描述

写一个 bash脚本以实现一个需求，去掉输入中含有this的语句，把不含this的语句输出
示例:
假设输入如下：
that is your bag
is this your bag?
to the degree or extent indicated.
there was a court case resulting from this incident
welcome to Nowcoder

你的脚本获取以上输入应当输出： that is your bag

to the degree or extent indicated.
welcome to Nowcoder

说明:
你可以不用在意输出的格式，包括空格和换行

#!/bin/bash
# grep
方法1 grep -v 'this' Nowcoder.txt
# sed
方法2 sed '/this/d' Nowcoder.txt
方法3 sed -rn '/this/!p' Nowcoder.txt
# awk
方法4 awk '!/this/' Nowcoder.txt
方法5 awk '{
# a=split($0, array, " ")
# }{for(i=1;i<=a;i++){
#     if(array[i]=="this"){
#         next
#     }
# }{print $0}}' Nowcoder.txt

方法6 awk '{
j=0
for(i=1;i<=NF;i++){
    if($i=="this"){
        j+=1
    }
}
if(j==0){
    print $0
}
}' Nowcoder.txt

14.求平均值

描述

写一个bash脚本以实现一个需求，求输入的一个的数组的平均值

第1行为输入的数组长度N
第2~N行为数组的元素，如以下为:
数组长度为4，数组元素为1 2 9 8
示例:
4
1
2
9
8

那么平均值为:5.000(保留小数点后面3位)
你的脚本获取以上输入应当输出：
5.000

#!/bin/bash
# basic
方法1 sum=0
# num=0
# while read p
# do
#     if [[ $num -eq 0 ]]
#         then 
#         num=$[$num+1]
#         continue
#     fi
#     sum=$[$sum+$p]
#     num=$[$num+1]

# done < Nowcoder.txt
# echo "scale=3;$sum/$[$num-1]" | bc
# awk
方法2 awk '{if(NR!=1)
#         sum+=$0
#     else num+=$0}END{printf "%.3f",sum/num}' Nowcoder.txt

15.去掉不需要的单词

描述

写一个 bash脚本以实现一个需求，去掉输入中的含有B和b的单词
示例:
假设输入如下：
big
Nowcoder
Betty
basic
test

你的脚本获取以上输入应当输出：
Nowcoder test

说明:
你可以不用在意输出的格式，空格和换行都行

#!/bin/bash
方法1 cnt=1
# sum=0
# while read p
# do
#     a=`echo $p | sed 's#[a-Z|6-9|0]##g' | awk '{print length($0)}'`
#     echo line$cnt number:$a
#     cnt=$[$cnt+1]
#     sum=$[$sum+$a]
# done 
# echo sum is $sum
# < Nowcoder.txt
# grep
方法2 grep -v 'this' Nowcoder.txt
# sed
方法3 sed '/this/d' Nowcoder.txt
方法4 sed -rn '/this/!p' Nowcoder.txt
# awk
方法5 awk '!/this/' Nowcoder.txt
方法6 awk '{
# a=split($0, array, " ")
# }{for(i=1;i<=a;i++){
#     if(array[i]=="this"){
#         next
#     }
# }{print $0}}' Nowcoder.txt


方法7 awk '{
# j=0
# for(i=1;i<=NF;i++){
#     if($i=="this"){
#         j+=1
#     }
# }
# if(j==0){
#     print $0
# }
# }' Nowcoder.txt



# grep
方法8 grep -v '[bB]' Nowcoder.txt
方法9 grep -v -E 'b|B' Nowcoder.txt

# sed
方法10 sed '/[b,B]/d' Nowcoder.txt
方法11 sed -nr '/[b,B]/!p' Nowcoder.txt
# awk
方法12 awk '!/[b,B]/' Nowcoder.txt
方法13 awk '$0!~/[b,B]/{print $0}' Nowcoder.txt

16.判断输入的是否为IP地址

描述

写一个脚本统计文件 Nowcoder.txt中的每一行是否是正确的IP地址。如果是正确的IP地址输出：yes 如果是错误的IP地址，四段号码的话输出：no，否则的话输出：error 假设Nowcoder.txt内容如下

192.168.1.1
192.168.1.0
300.0.0.0
123

你的脚本应该输出

yes yes no error

#!/bin/bash
# awk
awk -F"." '{
if(NF!=4) print("error")
else{
    info="yes"
    for(i=1;i<=NF;i++){
        if($i<0||$i>255){
            info="no"
            break
        }
    }
    print(info)
    }
}' Nowcoder.txt
17.将字段逆序输出文件的每行

描述
将字段逆序输出文件Nowcoder.txt的每一行，其中每一字段都是用英文冒号: 相分隔。假设Nowcoder.txt内容如下：
nobody:*:-2:-2:Unprivileged User:/var/empty:/usr/bin/false root:*:0:0:System Administrator:/var/root:/bin/sh
你的脚本应当输出
/usr/bin/false:/var/empty:Unprivileged User:-2:-2:*:nobody /bin/sh:/var/root:System Administrator:0:0:*:root
#!/bin/bash
# awk
方法1 awk -F: -vOFS=: '{print $NF,$6,$5,$4,$3,$2,$1}' Nowcoder.txt
# awk -F: '{
# for(i=NF;i>=1;i--){
#     if(i==1){
#         print $i
#         break
#     }
#     printf("%s",$i ":") 
# }
# }' Nowcoder.txt
方法2 awk -F: '{
for(i=1;i<=NF;i++){
    array[NF+1-i]=$i
}
for(j in array){
    if(j==NF){
        printf("%s",array[j] "\n")
    }else{
        printf("%s", array[j] ":")
    }
}

}' Nowcoder.txt
18.域名进行计数排序处理

描述
假设我们有一些域名，存储在Nowcoder.txt里，现在需要你写一个脚本，将域名取出并根据域名进行计数排序处理。假设Nowcoder.txt内容如下：
http://www.nowcoder.com/index.html http://www.nowcoder.com/1.html http://m.nowcoder.com/index.html
你的脚本应该输出：
2 www.nowcoder.com 1 m.Nowcoder.com
#!/bin/bash
# awk
awk -F"[//]+" '{array[$2]+=1}END{for(i in array){print array[i],i}}' Nowcoder.txt | sort -rnk 1
19.打印等腰三角形

描述
打印边长为5的等腰三角形。你的脚本应该输出
*
* *
* * *
* * * *
* * * * *
#!/bin/bash
# basic
方法1 echo "    *"
# echo "   * *"
# echo "  * * *"
# echo " * * * *"
# echo "* * * * *"
方法2 for (( i=1;i<=5;i++ )) 
# do
#     for (( j=5-i;j>0;j-- ))
#         do
#             printf " "
#         done
#         for(( k=i;k>0;k-- ))
#             do
#                 printf "* "
#             done
#     printf "\n"
# done
# awk
方法3 awk 'BEGIN{for(i=1;i<=5;i++){
for(j=5-i;j>0;j--){
    printf(" ")
}
for(k=i;k>0;k--){
    printf("* ")
}
printf("\n")
}}'
20.打印只有一个数字的行

描述
假设我们有一个Nowcoder.txt，现在需要你写脚本，打印只有一个数字的行。假设Nowcoder.txt内容如下
haha 1 2ab cd 77
那么你的脚本应该输出
1 2ab
#!/bin/bash
# basic
方法1 while read p
# do
#     num=`echo $p | grep -oE "[0-9]" | wc -l`
#     if [ $num -eq 1 ]
#         then
#         echo $p
#     fi
# done < Nowcoder.txt

方法2 while read p
do
    num=`echo $p | sed 's#[^0-9]##g' | awk '{print length($0)}'`
    if [ $num -eq 1 ]
        then
        echo $p
    fi
done < Nowcoder.txt

方法3 cnt=1
# sum=0
# while read p
# do
#     a=`echo $p | grep -oE "[1-5]" | wc -l`
#     echo line$cnt number:$a
#     cnt=$[$cnt+1]
#     sum=$[$sum+$a]
# done 
# echo sum is $sum
# < Nowcoder.txt
方法4 cnt=1
# sum=0
# while read p
# do
#     a=`echo $p | sed 's#[a-Z|6-9|0]##g' | awk '{print length($0)}'`
#     echo line$cnt number:$a
#     cnt=$[$cnt+1]
#     sum=$[$sum+$a]
# done 
# echo sum is $sum
# < Nowcoder.txt

# awk
方法5 awk '{
# gsub(/[^1-5]/,"",$0)
# print "line"NR" number: "length($0)
# sum+=length($0)
# }END{print "sum is "sum}' Nowcoder.txt
21.格式化输出

描述
我们有一个文件Nowcoder.txt，里面的每一行都是一个数字串，假设数字串为“123456789”，那么我们要输出为123,456,789。假设Nowcoder.txt内容如下
1 12 123 1234 123456
那么你的脚本输出如下
1 12 123 1,234 123,456
#!/bin/bash
# basic
方法1 while read p
# do
#     # 连续数字个数
#     c=0
#     # 数字的总个数
#     num=`echo $p | grep -oE '[0-9]' | wc -l`
#     # 输出结果
#     str=""
#     for(( i=$num-1;i>=0;i-- ))
#         do
#             # 计算需要截断的位置
#             c=$[ $c+1 ]
#             # 一个一个取数字，从后往前
#             str="${p:$i:1}$str"
#             # 数据长度大于3,指针没到底,计数器为3
#             if [ $num -gt 3 ] && [ $i -gt 0 ] && [ $[ $c % 3 ] -eq 0 ] 
#                 then 
#                 str=",$str"
#             fi
#         done
#     echo ${str}
# done < Nowcoder.txt
方法2 while read p
do
    printf "%'d\n" $p
done

方法3 while read p
# do
#     printf "%'.f\n" $p
# done
22.处理文本

描述
假设我们有一个Nowcoder.txt，假设里面的内容如下
111:13443 222:13211 111:13643 333:12341 222:12123
现在需要你写一个脚本按照以下的格式输出
[111] 13443 13643 [222] 13211 12123 [333] 12341
#!/bin/bash
# awk
awk -F":" '{
array[$1]=array[$1] "\n" $2
}END{for(i in array){
print("["i"]", array[i])
}}' Nowcoder.txt
23.Nginx日志分析1-IP统计

描述
假设Nginx的日志我们存储在Nowcoder.txt里，格式如下：
192.168.1.20 - - [21/Apr/2020:14:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0" 192.168.1.21 - - [21/Apr/2020:15:27:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0" 192.168.1.22 - - [21/Apr/2020:21:27:49 +0800] "GET /3/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0" 192.168.1.23 - - [21/Apr/2020:22:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0" 192.168.1.24 - - [22/Apr/2020:15:27:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0" 192.168.1.25 - - [22/Apr/2020:15:26:49 +0800] "GET /3/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0" 192.168.1.20 - - [23/Apr/2020:08:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0" 192.168.1.21 - - [23/Apr/2020:09:20:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0" 192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0" 192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0" 192.168.1.20 - - [23/Apr/2020:14:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0" 192.168.1.21 - - [23/Apr/2020:15:27:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0" 192.168.1.22 - - [23/Apr/2020:15:27:49 +0800] "GET /3/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0" 192.168.1.25 - - [23/Apr/2020:16:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0" 192.168.1.24 - - [23/Apr/2020:20:27:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0" 192.168.1.25 - - [23/Apr/2020:20:27:49 +0800] "GET /3/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0" 192.168.1.20 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0" 192.168.1.21 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0" 192.168.1.22 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0" 192.168.1.22 - - [23/Apr/2020:15:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0" 192.168.1.21 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
现在需要你统计出2020年4月23号的访问ip次数，并且按照次数降序排序。你的脚本应该输出：
5 192.168.1.22 4 192.168.1.21 3 192.168.1.20 2 192.168.1.25 1 192.168.1.24
#!/bin/bash
# grep
方法1 grep "23/Apr/2020" Nowcoder.txt | awk -F " " '{array[$1]++}END{for(i in array){print array[i],i}}' | sort -rnk 1
方法2 grep "23/Apr/2020" Nowcoder.txt | cut -c '-12' | sort | uniq -c | sort -rnk 1 | sed 's#^ *##'
方法3 grep "23/Apr/2020" Nowcoder.txt | awk -F " " '{print $1}' | sort | uniq -c | sort -rnk 1 | sed 's#^ *##'
方法4 grep "23/Apr/2020" Nowcoder.txt | awk -F " " '{print $1}' | sort | uniq -c | sort -rnk 1 
方法5 grep "23/Apr/2020" Nowcoder.txt | awk -F " " '{print $1}' | sort | uniq -c | sort -rnk 1 | awk '{print $1,$2}'
# sed
方法6 sed -n '/\[23\/Apr\/2020/p' | awk -F " " '{print $1}' | sort | uniq -c | sort -rnk 1 | awk '{print $1,$2}'
方法7 sed -n '/\[23\/Apr\/2020/p' | awk -F " " '{print $1}' | sort | uniq -c | sort -rnk 1 | sed 's#^ *##'
# awk
方法8 awk -F " " '$4~/\[23\/Apr\/2020/ {
# array[$1]++
# }END{
# for(i in array){
# print array[i], i
# }
# }' Nowcoder.txt | sort -rnk 1
24.Nginx日志分析2-统计某个时间段的IP

描述
假设Nginx的日志我们存储在Nowcoder.txt里，格式如下：
192.168.1.20 - - [21/Apr/2020:14:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [21/Apr/2020:15:27:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [21/Apr/2020:21:27:49 +0800] "GET /3/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.23 - - [21/Apr/2020:22:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.24 - - [22/Apr/2020:15:27:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [22/Apr/2020:15:26:49 +0800] "GET /3/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:08:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:09:20:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:14:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:15:27:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:15:27:49 +0800] "GET /3/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [23/Apr/2020:16:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.24 - - [23/Apr/2020:20:27:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [23/Apr/2020:20:27:49 +0800] "GET /3/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:22:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:23:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
现在你需要统计2020年04月23日20-23点的去重IP访问量，你的脚本应该输出 5
#!/bin/bash
# grep
方法1 grep -E '23/Apr/2020:2[0-2]' Nowcoder.txt | awk -F " " '{print $1}' | sort | uniq -c | wc -l
# sed 
方法2 sed -n '/\[23\/Apr\/2020\:20/,/\[23\/Apr\/2020\:23/p' Nowcoder.txt | awk -F " " '{print $1}' | sort | uniq -c | wc -l 
# awk
方法3 awk -F " " '$4~/\[23\/Apr\/2020\:2[0-3]/' Nowcoder.txt | awk '{print $1}' | sort | uniq -c | wc -l
25.Nginx日志分析3-统计访问3次以上的IP

描述
假设Nginx的日志我们存储在Nowcoder.txt里，格式如下：
192.168.1.20 - - [21/Apr/2020:14:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [21/Apr/2020:15:27:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [21/Apr/2020:21:27:49 +0800] "GET /3/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.23 - - [21/Apr/2020:22:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.24 - - [22/Apr/2020:15:27:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [22/Apr/2020:15:26:49 +0800] "GET /3/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:08:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:09:20:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:14:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:15:27:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:15:27:49 +0800] "GET /3/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [23/Apr/2020:16:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.24 - - [23/Apr/2020:20:27:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [23/Apr/2020:20:27:49 +0800] "GET /3/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:22:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:23:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"

现在需要你写脚本统计访问3次以上的IP，你的脚本应该输出
6 192.168.1.22
5 192.168.1.21
4 192.168.1.20
#!/bin/bash
# awk
awk -F " " '{array[$1]++}END{for(i in array){
if(array[i] > 3){
print(array[i], i)
}
}}' Nowcoder.txt | sort -rnk 1
26.Nginx日志分析4-查询某个IP的详细访问情况

描述
假设Nginx的日志我们存储在Nowcoder.txt里，格式如下：
192.168.1.20 - - [21/Apr/2020:14:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [21/Apr/2020:15:27:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [21/Apr/2020:21:27:49 +0800] "GET /3/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.23 - - [21/Apr/2020:22:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.24 - - [22/Apr/2020:15:27:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [22/Apr/2020:15:26:49 +0800] "GET /3/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:08:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:09:20:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:14:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:15:27:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:15:27:49 +0800] "GET /3/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [23/Apr/2020:16:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.24 - - [23/Apr/2020:20:27:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [23/Apr/2020:20:27:49 +0800] "GET /3/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:22:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:23:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
现在需要你查询192.168.1.22的详细访问情况，按访问频率降序排序。你的脚本应该输出
4 /1/index.PHP 2 /3/index.PHP
#!/bin/bash
# grep
方法1 grep '192.168.1.22' Nowcoder.txt | awk -F " " '{array[$7]++}END{for(i in array){
# print(array[i],i)
# }}' | sort -rnk 1
# sed
方法2 sed -n '/192.168.1.22/p' Nowcoder.txt | awk -F " " '{array[$7]++}END{for(i in array){
print(array[i],i)
}}' | sort -rnk 1
# awk
方法3 awk -F " " '$1~/192.168.1.22/{array[$7]++}END{for(i in array){
# print(array[i], i)
# }}' Nowcoder.txt | sort -rnk 1
27.Nginx日志分析5-统计爬虫抓取404的次数

描述
假设Nginx的日志我们存储在Nowcoder.txt里，格式如下：
192.168.1.20 - - [21/Apr/2020:14:12:49 +0800] "GET /1/index.PHP HTTP/1.1" 301 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [21/Apr/2020:15:00:49 +0800] "GET /2/index.PHP HTTP/1.1" 500 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [21/Apr/2020:21:21:49 +0800] "GET /3/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.23 - - [21/Apr/2020:22:10:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.24 - - [22/Apr/2020:15:00:49 +0800] "GET /2/index.PHP HTTP/1.1" 200 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [22/Apr/2020:15:26:49 +0800] "GET /3/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:08:05:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
192.168.1.21 - - [23/Apr/2020:09:20:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 200 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:14:12:49 +0800] "GET /1/index.PHP HTTP/1.1" 200 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:15:00:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:15:00:49 +0800] "GET /3/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
192.168.1.22 - - [23/Apr/2020:15:00:49 +0800] "GET /3/index.PHP HTTP/1.1" 200 490 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
192.168.1.24 - - [23/Apr/2020:20:27:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [23/Apr/2020:20:27:49 +0800] "GET /3/index.PHP HTTP/1.1" 200 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 300 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 500 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:22:10:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:23:59:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
现在需要你统计百度爬虫抓取404的次数，你的脚本应该输出 2
#!/bin/bash
# grep
方法1 grep 'http://www.baidu.com/search/spider.html' Nowcoder.txt | grep '404' | wc -l
# awk
方法2 count=0
# sed
方法3 sed -rn '/.*404.*http\:\/\/www\.baidu\.com\/search\/spider\.html.*/p' Nowcoder.txt | wc -l
方法4 awk -F " " '{
# if($15 ~ "http://www.baidu.com/search/spider.html" && $9 ~ "404"){
# count++
# }
# }
# END{
# print(count)
# }' Nowcoder.txt
28.Nginx日志分析6-统计每分钟的请求数

描述
假设Nginx的日志我们存储在Nowcoder.txt里，格式如下：
192.168.1.20 - - [21/Apr/2020:14:12:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [21/Apr/2020:15:00:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [21/Apr/2020:21:21:49 +0800] "GET /3/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.23 - - [21/Apr/2020:22:10:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.24 - - [22/Apr/2020:15:00:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [22/Apr/2020:15:26:49 +0800] "GET /3/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:08:05:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Baiduspider"
192.168.1.21 - - [23/Apr/2020:09:20:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:14:12:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:15:00:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:15:00:49 +0800] "GET /3/index.PHP HTTP/1.1" 404 490 "-" "Baiduspider"
192.168.1.25 - - [23/Apr/2020:16:15:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.24 - - [23/Apr/2020:20:27:49 +0800] "GET /2/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [23/Apr/2020:20:27:49 +0800] "GET /3/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:22:10:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:23:59:49 +0800] "GET /1/index.PHP HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
现在需要你统计每分钟的请求数，并且按照请求数降序排序。你的脚本应该输出
5 20:27 4 15:00 2 22:10 2 14:12 2 10:27 1 23:59 1 21:21 1 16:15 1 15:26 1 09:20 1 08:05
#!/bin/bash
# awk
方法1 awk -F " " '{array[substr($4,14,5)]++}END{
# for(i in array){
# print array[i] " " i
# }
# }' Nowcoder.txt | sort -rnk 1

方法2 awk -F ":" '{array[$2":"$3]++}END{
for(i in array){
print array[i] " " i
}
}' Nowcoder.txt | sort -rnk 1
29.netstat练习1-查看各个状态的连接数

描述
假设netstat命令运行的结果我们存储在Nowcoder.txt里，格式如下：

Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 0.0.0.0:6160 0.0.0.0:* LISTEN tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN tcp 0 0 172.16.56.200:41856 172.16.34.144:3306 ESTABLISHED tcp 0 0 172.16.56.200:49822 172.16.0.24:3306 ESTABLISHED tcp 0 0 172.16.56.200:49674 172.16.0.24:3306 ESTABLISHED tcp 0 0 172.16.56.200:42316 172.16.34.144:3306 ESTABLISHED tcp 0 0 172.16.56.200:44076 172.16.240.74:6379 ESTABLISHED tcp 0 0 172.16.56.200:49656 172.16.0.24:3306 ESTABLISHED tcp 0 0 172.16.56.200:58248 100.100.142.4:80 TIME_WAIT tcp 0 0 172.16.56.200:50108 172.16.0.24:3306 ESTABLISHED tcp 0 0 172.16.56.200:41944 172.16.34.144:3306 ESTABLISHED tcp 0 0 172.16.56.200:35548 100.100.32.118:80 TIME_WAIT tcp 0 0 172.16.56.200:39024 100.100.45.106:443 TIME_WAIT tcp 0 0 172.16.56.200:41788 172.16.34.144:3306 ESTABLISHED tcp 0 0 172.16.56.200:58260 100.100.142.4:80 TIME_WAIT tcp 0 0 172.16.56.200:41812 172.16.34.144:3306 ESTABLISHED tcp 0 0 172.16.56.200:41854 172.16.34.144:3306 ESTABLISHED tcp 0 0 172.16.56.200:58252 100.100.142.4:80 TIME_WAIT tcp 0 0 172.16.56.200:49586 172.16.0.24:3306 ESTABLISHED tcp 0 0 172.16.56.200:41754 172.16.34.144:3306 ESTABLISHED tcp 0 0 172.16.56.200:50466 120.55.222.235:80 TIME_WAIT tcp 0 0 172.16.56.200:38514 100.100.142.5:80 TIME_WAIT tcp 0 0 172.16.56.200:49832 172.16.0.24:3306 ESTABLISHED tcp 0 0 172.16.56.200:52162 100.100.30.25:80 ESTABLISHED tcp 0 0 172.16.56.200:50372 172.16.0.24:3306 ESTABLISHED tcp 0 0 172.16.56.200:50306 172.16.0.24:3306 ESTABLISHED tcp 0 0 172.16.56.200:49600 172.16.0.24:3306 ESTABLISHED tcp 0 0 172.16.56.200:41908 172.16.34.144:3306 ESTABLISHED tcp 0 0 172.16.56.200:60292 100.100.142.1:80 TIME_WAIT tcp 0 0 172.16.56.200:37650 100.100.54.133:80 TIME_WAIT tcp 0 0 172.16.56.200:41938 172.16.34.144:3306 ESTABLISHED tcp 0 0 172.16.56.200:49736 172.16.0.24:3306 ESTABLISHED tcp 0 0 172.16.56.200:41890 172.16.34.144:3306 ESTABLISHED udp 0 0 127.0.0.1:323 0.0.0.0:* udp 0 0 0.0.0.0:45881 0.0.0.0:* udp 0 0 127.0.0.53:53 0.0.0.0:* udp 0 0 172.16.56.200:68 0.0.0.0:* udp6 0 0 ::1:323 :::* raw6 0 0 :::58 :::* 7
现在需要你查看系统tcp连接中各个状态的连接数，并且按照连接数降序输出。你的脚本应该输出如下：
ESTABLISHED 22 TIME_WAIT 9 LISTEN 3
#!/bin/bash
# grep
方法1 grep 'tcp' Nowcoder.txt | awk -F " " '{array[$6]++}END{
# for(i in array){
# print i " " array[i]
# }
# }' | sort -rnk 2
# sed
方法2 sed -n '/tcp/p' Nowcoder.txt | awk -F " " '{array[$6]++}END{
# for(i in array){
# print i " " array[i]
# }
# }' | sort -rnk 2
# awk
方法3 awk -F " " '{
if($1 == "tcp"){
array[$6]++
}
}END{
for(i in array){
print i " " array[i]
}
}' | sort -rnk 2
30.netstat练习2-查看和3306端口建立的连接

描述
假设netstat命令运行的结果我们存储在Nowcoder.txt里，格式如下：
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:6160 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp 0 0 172.16.56.200:41856 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49822 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49674 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:42316 172.16.34.143:3306 ESTABLISHED
tcp 0 0 172.16.56.200:44076 172.16.240.74:6379 ESTABLISHED
tcp 0 0 172.16.56.200:49656 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:58248 100.100.142.4:80 TIME_WAIT
tcp 0 0 172.16.56.200:50108 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41944 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:35548 100.100.32.118:80 TIME_WAIT
tcp 0 0 172.16.56.200:39024 100.100.45.106:443 TIME_WAIT
tcp 0 0 172.16.56.200:41788 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:58260 100.100.142.4:80 TIME_WAIT
tcp 0 0 172.16.56.200:41812 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41854 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:58252 100.100.142.4:80 TIME_WAIT
tcp 0 0 172.16.56.200:49586 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41754 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:50466 120.55.222.235:80 TIME_WAIT
tcp 0 0 172.16.56.200:38514 100.100.142.5:80 TIME_WAIT
tcp 0 0 172.16.56.200:49832 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:52162 100.100.30.25:80 ESTABLISHED
tcp 0 0 172.16.56.200:50372 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:50306 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49600 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41908 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:60292 100.100.142.1:80 TIME_WAIT
tcp 0 0 172.16.56.200:37650 100.100.54.133:80 TIME_WAIT
tcp 0 0 172.16.56.200:41938 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49736 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41890 172.16.34.144:3306 ESTABLISHED
udp 0 0 127.0.0.1:323 0.0.0.0:*
udp 0 0 0.0.0.0:45881 0.0.0.0:*
udp 0 0 127.0.0.53:53 0.0.0.0:*
udp 0 0 172.16.56.200:68 0.0.0.0:*
udp6 0 0 ::1:323 :::*
raw6 0 0 :::58 :::* 7
现在需要你查看和本机3306端口建立连接并且状态是established的所有IP，按照连接数降序排序。你的脚本应该输出
10 172.16.0.24 9 172.16.34.144 1 172.16.34.143
#!/bin/bash
awk -F " " '{if($6 == "ESTABLISHED"){print $5}}' Nowcoder.txt | grep '3306' | awk -F ":" '{array[$1]++}END{
for(i in array){
print array[i] " " i
}}' | sort -rnk 1
31.netstat练习3-输出每个IP的连接数

描述
假设netstat命令运行的结果我们存储在Nowcoder.txt里，格式如下：

Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:6160 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp 0 0 172.16.56.200:41856 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49822 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49674 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:42316 172.16.34.143:3306 ESTABLISHED
tcp 0 0 172.16.56.200:44076 172.16.240.74:6379 ESTABLISHED
tcp 0 0 172.16.56.200:49656 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:58248 100.100.142.4:80 TIME_WAIT
tcp 0 0 172.16.56.200:50108 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41944 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:35548 100.100.32.118:80 TIME_WAIT
tcp 0 0 172.16.56.200:39024 100.100.45.106:443 TIME_WAIT
tcp 0 0 172.16.56.200:41788 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:58260 100.100.142.4:80 TIME_WAIT
tcp 0 0 172.16.56.200:41812 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41854 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:58252 100.100.142.4:80 TIME_WAIT
tcp 0 0 172.16.56.200:49586 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41754 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:50466 120.55.222.235:80 TIME_WAIT
tcp 0 0 172.16.56.200:38514 100.100.142.5:80 TIME_WAIT
tcp 0 0 172.16.56.200:49832 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:52162 100.100.30.25:80 ESTABLISHED
tcp 0 0 172.16.56.200:50372 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:50306 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49600 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41908 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:60292 100.100.142.1:80 TIME_WAIT
tcp 0 0 172.16.56.200:37650 100.100.54.133:80 TIME_WAIT
tcp 0 0 172.16.56.200:41938 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49736 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41890 172.16.34.144:3306 ESTABLISHED
udp 0 0 127.0.0.1:323 0.0.0.0:*
udp 0 0 0.0.0.0:45881 0.0.0.0:*
udp 0 0 127.0.0.53:53 0.0.0.0:*
udp 0 0 172.16.56.200:68 0.0.0.0:*
udp6 0 0 ::1:323 :::*
raw6 0 0 :::58 :::* 7
现在需要你输出每个IP的连接数，按照连接数降序排序。你的脚本应该输出
172.16.0.24 10 172.16.34.144 9 100.100.142.4 3 0.0.0.0 3 172.16.34.143 1 172.16.240.74 1 120.55.222.235 1 100.100.54.133 1 100.100.45.106 1 100.100.32.118 1 100.100.30.25 1 100.100.142.5 1 100.100.142.1 1
#!/bin/bash
awk -F " " '{
if($1 == "tcp"){
split($5, a, ":")
array[a[1]]++
}}END{
for(i in array){
print i " " array[i]
}
}' Nowcoder.txt | sort -rnk 2
32.netstat练习4-输出和3306端口建立连接总的各个状态数目

描述
假设netstat命令运行的结果我们存储在Nowcoder.txt里，格式如下：
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:6160 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp 0 0 172.16.56.200:41856 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49822 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49674 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:42316 172.16.34.143:3306 ESTABLISHED
tcp 0 0 172.16.56.200:44076 172.16.240.74:6379 ESTABLISHED
tcp 0 0 172.16.56.200:49656 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:58248 100.100.142.4:80 TIME_WAIT
tcp 0 0 172.16.56.200:50108 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41944 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:35548 100.100.32.118:80 TIME_WAIT
tcp 0 0 172.16.56.200:39024 100.100.45.106:443 TIME_WAIT
tcp 0 0 172.16.56.200:41788 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:58260 100.100.142.4:80 TIME_WAIT
tcp 0 0 172.16.56.200:41812 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41854 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:58252 100.100.142.4:80 TIME_WAIT
tcp 0 0 172.16.56.200:49586 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41754 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:50466 120.55.222.235:80 TIME_WAIT
tcp 0 0 172.16.56.200:38514 100.100.142.5:80 TIME_WAIT
tcp 0 0 172.16.56.200:49832 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:52162 100.100.30.25:80 ESTABLISHED
tcp 0 0 172.16.56.200:50372 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:50306 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49600 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41908 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:60292 100.100.142.1:80 TIME_WAIT
tcp 0 0 172.16.56.200:37650 100.100.54.133:80 TIME_WAIT
tcp 0 0 172.16.56.200:41938 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49736 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41890 172.16.34.144:3306 ESTABLISHED
udp 0 0 127.0.0.1:323 0.0.0.0:*
udp 0 0 0.0.0.0:45881 0.0.0.0:*
udp 0 0 127.0.0.53:53 0.0.0.0:*
udp 0 0 172.16.56.200:68 0.0.0.0:*
udp6 0 0 ::1:323 :::*
raw6 0 0 :::58 :::* 7
现在需要你输出和本机3306端口建立连接的各个状态的数目，按照以下格式输出
TOTAL_IP表示建立连接的ip数目 TOTAL_LINK表示建立连接的总数目
TOTAL_IP 3 ESTABLISHED 20 TOTAL_LINK 20
#!/bin/bash
b=0
awk -F " " '$1~"tcp"' Nowcoder.txt | awk '{if($5~/.*:3306/ && $6 == "ESTABLISHED"){
split($5,a,":")
array[a[1]]++
b++
}}END{
printf("TOTAL_IP %d\nesTABLISHED %d\nTOTAL_LINK %d",length(array),b,b)
}' 
33.业务分析-提取值

描述
假设我们的日志Nowcoder.txt里，内容如下
12-May-2017 10:02:22.789 信息 [main] org.apache.catalina.startup.VersionLoggerListener.log Server version:Apache Tomcat/8.5.15
12-May-2017 10:02:22.813 信息 [main] org.apache.catalina.startup.VersionLoggerListener.log Server built:May 5 2017 11:03:04 UTC
12-May-2017 10:02:22.813 信息 [main] org.apache.catalina.startup.VersionLoggerListener.log Server number:8.5.15.0
12-May-2017 10:02:22.814 信息 [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name:Windows, OS Version:10
12-May-2017 10:02:22.814 信息 [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture:x86_64
现在需要你提取出对应的值，输出内容如下
serverVersion:Apache Tomcat/8.5.15 serverName:8.5.15.0 osName:Windows osversion:10
#!/bin/bash
awk -F "[:,]" '{
if($0~"Server version"){
printf("serverVersion:%s",$4)
printf("\n")
}
if($0~"Server number"){
printf("serverName:%s",$4)
printf("\n")
}
if($0~"OS Name"){
printf("osName:%s",$4)
printf("\n")
printf("osversion:%s",$6)
}
}' Nowcoder.txt
34.ps分析-统计VSZ,RSS各自总和

描述
假设命令运行的结果我们存储在Nowcoder.txt里，格式如下：
USER PID %cpu %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.1 37344 4604 ? Ss 2020 2:13 /sbin/init
root 231 0.0 1.5 166576 62740 ? Ss 2020 15:15 /lib/systemd/systemd-journald
root 237 0.0 0.0 0 0 ? S< 2020 2:06 [kworker/0:1H]
root 259 0.0 0.0 45004 3416 ? Ss 2020 0:25 /lib/systemd/systemd-udevd
root 476 0.0 0.0 0 0 ? S< 2020 0:00 [edac-poller]
root 588 0.0 0.0 276244 2072 ? Ssl 2020 9:49 /usr/lib/accountsservice/accounts-daemon
message+ 592 0.0 0.0 42904 3032 ? Ss 2020 0:01 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation
root 636 0.0 0.0 65532 3200 ? Ss 2020 1:51 /usr/sbin/sshd -D
daemon 637 0.0 0.0 26044 2076 ? Ss 2020 0:00 /usr/sbin/atd -f
root 639 0.0 0.0 29476 2696 ? Ss 2020 3:29 /usr/sbin/cron -f
root 643 0.0 0.0 20748 1992 ? Ss 2020 0:26 /lib/systemd/systemd-logind
syslog 645 0.0 0.0 260636 3024 ? Ssl 2020 3:17 /usr/sbin/rsyslogd -n
root 686 0.0 0.0 773124 2836 ? Ssl 2020 26:45 /usr/sbin/nscd
root 690 0.0 0.0 19472 252 ? Ss 2020 14:39 /usr/sbin/irqbalance --pid=/var/run/irqbalance.pid
ntp 692 0.0 0.0 98204 776 ? Ss 2020 25:18 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 108:114
uuidd 767 0.0 0.0 28624 192 ? Ss 2020 0:00 /usr/sbin/uuidd --socket-activation
root 793 0.0 0.0 128812 3148 ? Ss 2020 0:00 Nginx: master process /usr/sbin/Nginx -g daemon on; master_process on;
www-data 794 0.0 0.2 133376 9120 ? S 2020 630:57 Nginx: worker process
www-data 795 0.0 0.2 133208 8968 ? S 2020 633:02 Nginx: worker process
www-data 796 0.0 0.2 133216 9120 ? S 2020 634:24 Nginx: worker process
www-data 797 0.0 0.2 133228 9148 ? S 2020 632:56 Nginx: worker process
web 955 0.0 0.0 36856 2112 ? Ss 2020 0:00 /lib/systemd/systemd --user
web 956 0.0 0.0 67456 1684 ? S 2020 0:00 (sd-pam)
root 1354 0.0 0.0 8172 440 tty1 Ss+ 2020 0:00 /sbin/agetty --noclear tty1 linux
root 1355 0.0 0.0 7988 344 ttyS0 Ss+ 2020 0:00 /sbin/agetty --keep-baud 115200 38400 9600 ttyS0 vt220
root 2513 0.0 0.0 0 0 ? S 13:07 0:00 [kworker/u4:1]
root 2587 0.0 0.0 0 0 ? S 13:13 0:00 [kworker/u4:2]
root 2642 0.0 0.0 0 0 ? S 13:17 0:00 [kworker/1:0]
root 2679 0.0 0.0 0 0 ? S 13:19 0:00 [kworker/u4:0]
root 2735 0.0 0.1 102256 7252 ? Ss 13:24 0:00 sshd: web [priv]
web 2752 0.0 0.0 102256 3452 ? R 13:24 0:00 sshd: web@pts/0
web 2753 0.5 0.1 14716 4708 pts/0 Ss 13:24 0:00 -bash
web 2767 0.0 0.0 29596 1456 pts/0 R+ 13:24 0:00 ps aux
root 10634 0.0 0.0 0 0 ? S Nov16 0:00 [kworker/0:0]
root 16585 0.0 0.0 0 0 ? S< 2020 0:00 [bioset]
root 19526 0.0 0.0 0 0 ? S Nov16 0:00 [kworker/1:1]
root 28460 0.0 0.0 0 0 ? S Nov15 0:03 [kworker/0:2]
root 30685 0.0 0.0 36644 2760 ? Ss 2020 0:00 /lib/systemd/systemd --user
root 30692 0.0 0.0 67224 1664 ? S 2020 0:00 (sd-pam)
root 32689 0.0 0.0 47740 2100 ? Ss 2020 0:00 /usr/local/ilogtail/ilogtail
root 32691 0.2 0.5 256144 23708 ? Sl 2020 1151:31 /usr/local/ilogtail/ilogtail
现在需要你统计VSZ，RSS各自的总和（以M兆为统计），输出格式如下
MEM TOTAL VSZ_SUM:3250.8M,RSS_SUM:179.777M
#!/bin/bash
awk -F " " 'BEGIN{v=0;r=0}{
v+=$5
r+=$6
}END{
print("MEM TOTAL")
printf("VSZ_SUM:%sM,RSS_SUM:%sM",v/1024,r/1024)
}' Nowcoder.txt

Leetcode部分：

1.统计词频

写一个 bash 脚本以统计一个文本文件 words.txt 中每个单词出现的频率。

为了简单起见，你可以假设：

    words.txt只包括小写字母和 ' ' 。
    每个单词只由小写字母组成。
    单词间由一个或多个空格字符分隔。

示例:

假设 words.txt 内容如下：

the day is sunny the the
the sunny is is

你的脚本应当输出（以词频降序排列）：

the 4
is 3
sunny 2
day 1

说明:

    不要担心词频相同的单词的排序问题，每个单词出现的频率都是唯一的。
    你可以使用一行 Unix pipes 实现吗？

来源：力扣（LeetCode）
链接：https://leetcode.cn/problems/word-frequency
著作权归领扣网络所有。商业转载请联系官方授权，非商业转载请注明出处。

# Read from the file words.txt and output the word frequency list to stdout.
#!/bin/bash
awk -F " " '{for(i=1;i<=NF;i++){
array[$i]++
}}END{for(j in array){
print j " " array[j]
}}' words.txt | sort -rnk 2

2.有效电话号码

给定一个包含电话号码列表（一行一个电话号码）的文本文件 file.txt，写一个单行 bash 脚本输出所有有效的电话号码。

你可以假设一个有效的电话号码必须满足以下两种格式： (xxx) xxx-xxxx 或 xxx-xxx-xxxx。（x 表示一个数字）

你也可以假设每行前后没有多余的空格字符。

示例：

假设 file.txt 内容如下：

987-123-4567
123 456 7890
(123) 456-7890

你的脚本应当输出下列有效的电话号码：

987-123-4567
(123) 456-7890

来源：力扣（LeetCode）
链接：https://leetcode.cn/problems/valid-phone-numbers
著作权归领扣网络所有。商业转载请联系官方授权，非商业转载请注明出处。

# Read from the file file.txt and output all valid phone numbers to stdout.
grep -E '^([0-9]{3}-|\([0-9]{3}\) )[0-9]{3}-[0-9]{4}$' file.txt

3.转置文件

给定一个文件 file.txt，转置它的内容。

你可以假设每行列数相同，并且每个字段由 ' ' 分隔。

示例：

假设 file.txt 文件内容如下：

name age
alice 21
ryan 30

应当输出：

name alice ryan
age 21 30

来源：力扣（LeetCode）
链接：https://leetcode.cn/problems/transpose-file
著作权归领扣网络所有。商业转载请联系官方授权，非商业转载请注明出处。

# Read from the file file.txt and print its transposed content to stdout.
awk -F " " '{for(i=1;i<=NF;i++){
    if(NR==1){
        array[i]=$i
    }else{
        array[i]=array[i] " " $i
    }
}}END{
    for(j=1;j<=NF;j++){
        print array[j]
    }
}'  file.txt

4.第十行

给定一个文本文件 file.txt，请只打印这个文件中的第十行。

示例:

假设 file.txt 有如下内容：

Line 1
Line 2
Line 3
Line 4
Line 5
Line 6
Line 7
Line 8
Line 9
Line 10

你的脚本应当显示第十行：

Line 10

说明:
1. 如果文件少于十行，你应当输出什么？
2. 至少有三种不同的解法，请尝试尽可能多的方法来解题。

来源：力扣（LeetCode）
链接：https://leetcode.cn/problems/tenth-line
著作权归领扣网络所有。商业转载请联系官方授权，非商业转载请注明出处。

# Read from the file file.txt and output the tenth line to stdout.
awk '{if(NR==10){
print $0
}}' file.txt

力扣的题都在前面牛客部分做过就随便跑一下，希望有帮助，大家多讨论。

Shell编程牛客网和Leetcode

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

描述

相关推荐