Sponsored Content
Top Forums Shell Programming and Scripting Getting info from a huge log file Post 302554399 by cwiggler on Monday 12th of September 2011 02:11:56 AM
Old 09-12-2011
Hello everyone.

I was trying to get another data but i am having problem with field seperator based on the log sample below.

main
Code:
|08:52:01.304|0x00001450|2 |        |RAM_24   |      RBL: RBL Code 10   
|08:52:01.304|0x00001450|2 |        |RAM_24   |      DRPD: DRPD 789123
|08:52:01.304|0x00001450|2 |        |RAM_24   |      RTR: RTR Incomplete

trigger
Code:
|08:52:01.335|0x00001450|-1|        |BIOPDB    |.\ESRvalueafter(618) : Caught ERROR(MFC) exception - BIOS-INFO20530: Alert on the bios get info on top

If the log sample has a seperator of something like this below i works

fs=':\\| +'; main='RBL: '; trig='BIOS-INFO'
awk -F"$fs" -v main="$main" -v trig="$trig" '$2 ~ main { first = $0; getline; second = $0; getline; third = $0; } first && $2 ~ trig { printf "%s\n%s\n%s\n", first, second, third }' INPUTFILE

main
Code:
|08:52:01.304|0x00001450|2 |        |RAM_24   :|      RBL: RBL Code 10   
|08:52:01.304|0x00001450|2 |        |RAM_24   :|      DRPD: DRPD 789123
|08:52:01.304|0x00001450|2 |        |RAM_24   :|      RTR: RTR Incomplete

trigger
Code:
|08:52:01.335|0x00001450|-1|        |BIOPDB    :|.\ESRvalueafter(618) : Caught ERROR(MFC) exception - BIOS-INFO20530: Alert on the bios get info on top

but using the sample log above with the using this it doesn't get any output.

fs='|'; main='RBL: '; trig='BIOS-INFO'
awk -F"$fs" -v main="$main" -v trig="$trig" '$6 ~ main { first = $0; getline; second = $0; getline; third = $0; } first && $6 ~ trig { printf "%s\n%s\n%s\n", first, second, third }' INPUTFILE

anyone can explain why? i can't understand why with FS :\\| it works and assign field var to $2. could anyone disect it please. thanks
 

10 More Discussions You Might Find Interesting

1. Linux

Searching for gaps in huge (2.2G) log file?

I've got a 2.2 Gig syslog file from our Cisco firewall appliance. The problem is that we've been seeing gaps in the syslog for anywhere from 10 minutes to 2 hours. Currently I've just been using 'less' and paging through the file to see if I can find any noticeable gaps. Obviously this isn't the... (3 Replies)
Discussion started by: deckard
3 Replies

2. Shell Programming and Scripting

insert a header in a huge data file without using an intermediate file

I have a file with data extracted, and need to insert a header with a constant string, say: H|PayerDataExtract if i use sed, i have to redirect the output to a seperate file like sed ' sed commands' ExtractDataFile.dat > ExtractDataFileWithHeader.dat the same is true for awk and... (10 Replies)
Discussion started by: deepaktanna
10 Replies

3. Shell Programming and Scripting

Extract info from log file and compute using time date stamp

Looking for a shell script or a simple perl script . I am new to scripting and not very good at it . I have 2 directories . One of them holds a text file with list of files in it and the second one is a daily log which shows the file completion time. I need to co-relate both and make a report. ... (0 Replies)
Discussion started by: breez_drew
0 Replies

4. Shell Programming and Scripting

Log File - Getting Info about preceding Date of Pattern Found

Ok Suppose I have a log file like the below: 2010-07-15 00:00:01,410 DEBUG 2010-07-15 00:01:01,410 DEBUG 2010-07-15 00:01:02,410 DEBUG com.af ajfajfaf affafadfadfd dfa fdfadfdfadfadf fafafdfadfdafadfdaffdaffadf afdfdafdfdafafd error error failure afdfadfdfdfdf EBUDGG eafaferror failure... (6 Replies)
Discussion started by: SkySmart
6 Replies

5. Shell Programming and Scripting

Help finding info from log file

Hi, I have a log file that contains information such as this: date id number command1 command2 command3 command4 data data data date id number command1 command2 command3 command4 (4 Replies)
Discussion started by: bbbngowc
4 Replies

6. Shell Programming and Scripting

Optimised way for search & replace a value on one line in a very huge file (File Size is 24 GB).

Hi Experts, I had to edit (a particular value) in header line of a very huge file so for that i wanted to search & replace a particular value on a file which was of 24 GB in Size. I managed to do it but it took long time to complete. Can anyone please tell me how can we do it in a optimised... (7 Replies)
Discussion started by: manishkomar007
7 Replies

7. Shell Programming and Scripting

Event logging to file and display to console | tee command is not able to log all info.

My intention is to log the output to a file as well as it should be displayed on the console > I have used tee ( tee -a ${filename} ) command for this purpose. This is working as expected for first few outputs, after some event loggin nothing is gettting logged in to the file but It is displaying... (3 Replies)
Discussion started by: sanoop
3 Replies

8. Shell Programming and Scripting

HELP: Shell Script to read a Log file line by line and extract Info based on KEYWORDS matching

I have a LOG file which looks like this Import started at: Mon Jul 23 02:13:01 EDT 2012 Initialization completed in 2.146 seconds. -------------------------------------------------------------------------------- -- Import summary for Import item: PolicyInformation... (8 Replies)
Discussion started by: biztank
8 Replies

9. Shell Programming and Scripting

How to select bulk of info from log file?

unix : sun shell : bash i need to select multiple rows with this format : <special format> 10 lines /<special format> from log file that have lots of info i thought of getting the number of the first line using grep -n "special format" file | cut -d: -f1 then pass it to shell... (2 Replies)
Discussion started by: scorpioneer
2 Replies

10. Shell Programming and Scripting

Help on script to capture info on log file for a particular time frame

Hi I have a system running uname -a Linux cmovel-db01 2.6.32-38-server #83-Ubuntu SMP Wed Jan 4 11:26:59 UTC 2012 x86_64 GNU/Linux I would like to capture the contents of /var/log/syslog from 11:00AM to 11:30AM and sent to this info via email. I was thinking in set a cron entry at that... (2 Replies)
Discussion started by: fretagi
2 Replies
rblsmtpd(1)						      General Commands Manual						       rblsmtpd(1)

NAME
rblsmtpd - blocks mail from RBL-listed sites. It works with any SMTP server that can run under tcpserver(1) SYNOPSIS
rblsmtpd opts prog DESCRIPTION
opts is a series of getopt-style options. prog consists of one or more arguments. Normally rblsmtpd runs prog. prog is expected to carry out an SMTP conversation to receive incoming mail messages. However, rblsmtpd does not invoke prog if it is told to block mail from this client. Instead it carries out its own limited SMTP conversa- tion, temporarily rejecting all attempts to send a message. Meanwhile it prints one line on descriptor 2 to log its activity. rblsmtpd drops the limited SMTP conversation after 60 seconds, even if the client has not quit by then. OPTIONS
-t n Change the timeout to n seconds. Blocked clients If the $RBLSMTPD environment variable is set and is nonempty, rblsmtpd blocks mail. It uses $RBLSMTPD as an error message for the client. Normally rblsmtpd runs under tcpserver(1); you can use tcprules(1) to set $RBLSMTPD for selected clients. If $RBLSMTPD is set and is empty, rblsmtpd does not block mail. If $RBLSMTPD is not set, rblsmtpd looks up $TCPREMOTEIP in the RBL, and blocks mail if $TCPREMOTEIP is listed. tcpserver sets up $TCPRE- MOTEIP as the IP address of the remote host. -r base Use base as an RBL source. An IP address a.b.c.d is listed by that source if d.c.b.a.base has a TXT record. rblsmtpd uses the con- tents of the TXT record as an error message for the client. -a base Use base as an anti-RBL source. An IP address a.b.c.d is anti-listed by that source if d.c.b.a.base has an A record. In this case rblsmtpd does not block mail. You may supply any number of -r and -a options. rblsmtpd tries each source in turn until it finds one that lists or anti-lists $TCPRE- MOTEIP. It also tries an RBL source of rbl.maps.vix.com if you do not supply any -r options. See http://maps.vix.com/rbl/ for more informa- tion about rbl.maps.vix.com. If you want to run your own RBL source or anti-RBL source for rblsmtpd, you can use rbldns from the DNScache (djbdns) package. Temporary errors Normally, if $RBLSMTPD is set, rblsmtpd uses a 451 error code in its limited SMTP conversation. This tells legitimate clients to try again later. It gives innocent relay operators a chance to see the problem, prohibit relaying, get off the RBL, and get the mail delivered. However, if $RBLSMTPD begins with a hyphen, rblsmtpd removes the hyphen and uses a 553 error code. This tells legitimate clients to bounce the message immediately. There are several error-handling options for RBL lookups: -B (Default.) Use a 451 error code for IP addresses listed in the RBL. -b Use a 553 error code for IP addresses listed in the RBL. -C (Default.) Handle RBL lookups in a ``fail-open'' mode. If an RBL lookup fails temporarily, assume that the address is not listed; if an anti-RBL lookup fails temporarily, assume that the address is anti-listed. Unfortunately, a knowledgeable attacker can force an RBL lookup or an anti-RBL lookup to fail temporarily, so that his mail is not blocked. -c Handle RBL lookups in a ``fail-closed'' mode. If an RBL lookup fails temporarily, assume that the address is listed (but use a 451 error code even with -b). If an anti-RBL lookup fails temporarily, assume that the address is not anti-listed (but use a 451 error code even if a subsequent RBL lookup succeeds with -b). Unfortunately, this sometimes delays legitimate mail. SEE ALSO
tcpserver(1), tcprules(1), tcprulescheck(1), fixcrio(1), recordio(1), rblsmtpd(1), tcpclient(1), who@(1), date@(1), finger@(1), http@(1), tcpcat(1), mconnect(1), tcp-environ(5) http://cr.yp.to/ucspi-tcp.html rblsmtpd(1)
All times are GMT -4. The time now is 01:16 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy