我有一些catchall日志文件,格式如下：
timestamp event summary foo details account name: userA bar more details timestamp event summary baz details account name: userB qux more details timestamp etc.
我想在日志文件中搜索userB,如果找到,则从前面的时间戳回显到(但不包括)以下时间戳.可能会有几个与我的搜索匹配的事件.能够在每场比赛中回应某种—开始—和—结束 – 这将是一件好事.
这对pcregrep -M来说是完美的,对吧？问题是,GnuWin32的pcregrep在多行regexps搜索大文件时崩溃,这些全能日志可能是100兆或更多.
我试过的
到目前为止,我的hackish解决方法是使用grep -B15 -A30找到匹配的行并打印周围的内容,然后将现在更易管理的块管道输入pcregrep进行抛光.问题是某些事件少于十行,而其他事件少于30行;我遇到了一些意外的结果,遇到了较短的事件.
:parselog <username> <logfile>
set silent=1
set count=0
set deez=20\d\d-\d\d-\d\d \d\d:\d\d:\d\d
echo Searching %~2 for records containing %~1...
for /f "delims=" %%I in (
'grep -P -i -B15 -A30 ":\s+\b%~1\b(@mydomain\.ext)?$" "%~2" ^| pcregrep -M -i "^%deez%(.|\n)+?\b%~1\b(@mydomain\.ext|\r?\n)(.|\n)+?\n%deez%" 2^>NUL'
) do (
echo(%%I| findstr "^20[0-9][0-9]-[0-9][0-9]-[0-9][0-9].[0-9][0-9]:[0-9][0-9]:[0-9][0-9]" >NUL && (
if defined silent (
set silent=
set found=1
set /a "count+=1"
echo;
echo ---------------start of record !count!-------------
) else (
set silent=1
echo ----------------end of record !count!--------------
echo;
)
)
if not defined silent echo(%%I
)
goto :EOF
有一个更好的方法吗？我遇到了一个看起来很有趣的awk命令,例如：
awk "/start pattern/,/end pattern/" logfile
……但它也需要匹配中间模式.不幸的是,我对awk语法并不熟悉.有什么建议么？
Ed Morton建议我提供一些示例记录和预期输出.
示例全能
2013-03-25 08:02:32 Auth.Critical 169.254.8.110 Mar 25 08:02:32 dc3 MSWinEventLog 2 Security 11730158 Mon Mar 25 08:02:28 2013 529 Security NT AUTHORITY\SYSTEM N/A Audit Failure dc3 2 Logon Failure:
Reason: Unknown user name or bad password
User Name: user5f
Domain: MYDOMAIN
Logon Type: 3
Logon Process: Advapi
Authentication Package: Negotiate
Workstation Name: dc3
Caller User Name: dc3$
Caller Domain: MYDOMAIN
Caller Logon ID: (0x0,0x3E7)
Caller Process ID: 400
Transited Services: -
Source Network Address: 169.254.7.86
Source Port: 40838
2013-03-25 08:02:32 Auth.Critical 169.254.8.110 Mar 25 08:02:32 dc3 MSWinEventLog 2 Security 11730159 Mon Mar 25 08:02:29 2013 680 Security NT AUTHORITY\SYSTEM N/A Audit Failure dc3 9 Logon attempt by: MICROSOFT_AUTHENTICATION_PACKAGE_V1_0
Logon account: USER6Q
Source Workstation: dc3
Error Code: 0xC0000234
2013-03-25 08:02:32 Auth.Critical 169.254.8.110 Mar 25 08:02:32 dc3 MSWinEventLog 2 Security 11730160 Mon Mar 25 08:02:29 2013 539 Security NT AUTHORITY\SYSTEM N/A Audit Failure dc3 2 Logon Failure:
Reason: Account locked out
User Name: USER6Q@MYDOMAIN.TLD
Domain: MYDOMAIN
Logon Type: 3
Logon Process: Advapi
Authentication Package: Negotiate
Workstation Name: dc3
Caller User Name: dc3$
Caller Domain: MYDOMAIN
Caller Logon ID: (0x0,0x3E7)
Caller Process ID: 400
Transited Services: -
Source Network Address: 169.254.7.89
Source Port: 55314
2013-03-25 08:02:32 Auth.Notice 169.254.5.62 Mar 25 08:36:38 DC4.mydomain.tld MSWinEventLog 5 Security 201326798 Mon Mar 25 08:36:37 2013 4624 Microsoft-Windows-Security-Auditing N/A Audit Success DC4.mydomain.tld 12544 An account was successfully logged on.
Subject:
Security ID: S-1-0-0
Account Name: -
Account Domain: -
Logon ID: 0x0
Logon Type: 3
New Logon:
Security ID: S-1-5-21-606747145-1409082233-725345543-160838
Account Name: DEPTACCT16$
Account Domain: MYDOMAIN
Logon ID: 0x1158e6012c
Logon GUID: {BCC72986-82A0-4EE9-3729-847BA6FA3A98}
Process Information:
Process ID: 0x0
Process Name: -
Network Information:
Workstation Name:
Source Network Address: 169.254.114.62
Source Port: 42183
Detailed Authentication Information:
Logon Process: Kerberos
Authentication Package: Kerberos
Transited Services: -
Package Name (NTLM only): -
Key Length: 0
This event is generated when a logon session is created. It is generated on the computer that was accessed.
The subject fields indicate...
2013-03-25 08:02:32 Auth.Critical 169.254.8.110 Mar 25 08:02:32 dc3 MSWinEventLog 2 Security 11730162 Mon Mar 25 08:02:30 2013 675 Security NT AUTHORITY\SYSTEM N/A Audit Failure dc3 9 Pre-authentication failed:
User Name: USER8Y
User ID: %{S-1-5-21-606747145-1409082233-725345543-3904}
Service Name: krbtgt/MYDOMAIN
Pre-Authentication Type: 0x0
Failure Code: 0x19
Client Address: 169.254.87.158
2013-03-25 08:02:32 Auth.Critical etc.
示例命令
call :parselog user6q \\path\to\catch-all.log
预期结果
---------------start of record 1-------------
2013-03-25 08:02:32 Auth.Critical 169.254.8.110 Mar 25 08:02:32 dc3 MSWinEventLog 2 Security 11730159 Mon Mar 25 08:02:29 2013 680 Security NT AUTHORITY\SYSTEM N/A Audit Failure dc3 9 Logon attempt by: MICROSOFT_AUTHENTICATION_PACKAGE_V1_0
Logon account: USER6Q
Source Workstation: dc3
Error Code: 0xC0000234
---------------end of record 1-------------
---------------start of record 2-------------
2013-03-25 08:02:32 Auth.Critical 169.254.8.110 Mar 25 08:02:32 dc3 MSWinEventLog 2 Security 11730160 Mon Mar 25 08:02:29 2013 539 Security NT AUTHORITY\SYSTEM N/A Audit Failure dc3 2 Logon Failure:
Reason: Account locked out
User Name: USER6Q@MYDOMAIN.TLD
Domain: MYDOMAIN
Logon Type: 3
Logon Process: Advapi
Authentication Package: Negotiate
Workstation Name: dc3
Caller User Name: dc3$
Caller Domain: MYDOMAIN
Caller Logon ID: (0x0,0x3E7)
Caller Process ID: 400
Transited Services: -
Source Network Address: 169.254.7.89
Source Port: 55314
---------------end of record 2-------------
这就是GNU awk所需要的(对于IGNORECASE)：
$cat tst.awk
function prtRecord() {
if (record ~ regexp) {
printf "-------- start of record %d --------%s",++numRecords,ORS
printf "%s",record
printf "--------- end of record %d ---------%s%s",numRecords,ORS,ORS
}
record = ""
}
BEGIN{ IGNORECASE=1 }
/^[[:digit:]]+-[[:digit:]]+-[[:digit:]]+/ { prtRecord() }
{ record = record $0 ORS }
END { prtRecord() }
或任何awk：
$cat tst.awk
function prtRecord() {
if (tolower(record) ~ tolower(regexp)) {
printf "-------- start of record %d --------%s",ORS
}
record = ""
}
/^[[:digit:]]+-[[:digit:]]+-[[:digit:]]+/ { prtRecord() }
{ record = record $0 ORS }
END { prtRecord() }
无论哪种方式,您都可以在UNIX上运行它：
$awk -v regexp=user6q -f tst.awk file
我不知道Windows语法,但我希望它非常相似,如果不相同的话.
请注意在脚本中使用tolower()使比较小写的两边都匹配,因此匹配不区分大小写.如果你可以传入一个正确的搜索正则表达式,那么你不需要在比较的任何一侧调用tolower(). nbd,它可能会略微加快脚本速度.
$awk -v regexp=user6q -f tst.awk file
-------- start of record 1 --------
2013-03-25 08:02:32 Auth.Critical 169.254.8.110 Mar 25 08:02:32 dc3 MSWinEventLog 2 Security
11730159 Mon Mar 25 08:02:29 2013 680 Security NT AUTHORITY\SYSTEM N/A Audit Failure
dc3 9 Logon attempt by: MICROSOFT_AUTHENTICATION_PACKAGE_V1_0
Logon account: USER6Q
Source Workstation: dc3
Error Code: 0xC0000234
--------- end of record 1 ---------
-------- start of record 2 --------
2013-03-25 08:02:32 Auth.Critical 169.254.8.110 Mar 25 08:02:32 dc3 MSWinEventLog 2 Security
11730160 Mon Mar 25 08:02:29 2013 539 Security NT AUTHORITY\SYSTEM N/A Audit Failure
dc3 2 Logon Failure:
Reason: Account locked out
User Name: USER6Q@MYDOMAIN.TLD
Domain: MYDOMAIN
Logon Type: 3
Logon Process: Advapi
Authentication Package: Negotiate
Workstation Name: dc3
Caller User Name: dc3$
Caller Domain: MYDOMAIN
Caller Logon ID: (0x0,0x3E7)
Caller Process ID: 400
Transited Services: -
Source Network Address: 169.254.7.89
Source Port: 55314
--------- end of record 2 ---------
版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 dio@foxmail.com 举报，一经查实，本站将立刻删除。
