Parse log files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Parse log files
# 1  
Old 03-17-2015
Parse log files

Hi all,

We are having a sample log like
Code:
....
test.log:2015.03.17 06:16:24 >> ABC.generateMethod() MethodAException while processing Request! DataForm: Header ---  dtd: template.dtd, titleName: berger, requestId: 1503170032131, documentName: invoice123, hostName: acme.net, userName: userABC -- Media ---  mediaType: XML, serverName: ABCServer1 -- data section --- ...
.....

We would like to create a bash script that selects lines having documentName string and for each line print out values for: titleName, documentName and serverName
# 2  
Old 03-17-2015
Please show us explicitly the output you want to generate from this line in your log file.

What operating system are you using?

What shell are you using?

What have you tried to solve this problem on your own?
# 3  
Old 03-18-2015
Hi Don,

The output desired would be
Code:
titleName: berger
documentName: invoice123
serverName: ABCServer1

The code we used to start selecting lines from the log:
Code:
#!/bin/bash
fname = test.log
pattern = documentName
if [ -f "$fname" ]
then
    result=$(grep -i "$pattern" "$fname")
    echo "$result"
fi

# 4  
Old 03-18-2015
You could try something like:
Code:
#!/bin/bash
awk -F', | -{2,3} {1,2}' '
/documentName:/ {
	for(i = 1; i <= NF; i++)
		if($i ~ /^titleName:/)		tn = $i
		else if($i ~ /^documentName:/)	dc = $i
		else if($i ~ /^serverName:/)	sn = $i
	printf("%s\n%s\n%s\n", tn, dc, sn)
}' logfile

which, with your sample log file produces the output:
Code:
titleName: berger
documentName: invoice123
serverName: ABCServer1

You didn't answer my question about what OS you're using. If you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk.
# 5  
Old 03-18-2015
Try also
Code:
awk     'BEGIN                  {n=split("titleName documentName serverName", SEAR)}
         /documentName/         {for (i=1; i<=n; i++)   {match ($0, SEAR[i]": [^, ]*")
                                                         print substr($0, RSTART, RLENGTH)}
                                }
        ' file
titleName: berger
documentName: invoice123
serverName: ABCServer1

# 6  
Old 03-18-2015
Thanks, guys! Both worked fine.

---------- Post updated at 07:42 AM ---------- Previous update was at 07:29 AM ----------

One more thing I just realized, for retrieving also the information about the file and timestamp, like:
Code:
test.log:2015.03.17 06:16:24
titleName: berger
documentName: invoice123
serverName: ABCServer1

how can I modify the script? - it is a Linux Box
# 7  
Old 03-18-2015
I'd propose we leave this up to you, as an exercise... you may want to print out $1 in awk.
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parse audit log

I am trying to parse the audit log to find a particular date that associated with a user record. The Date and the context of the record that I need to extract from the audit.log are 11-07-2015, the username and the activity he or she performed that day. Here is my code: grep -c date -d... (3 Replies)
Discussion started by: dellanicholson
3 Replies

2. Shell Programming and Scripting

Parse A Log File

Hello All, Below is the excerpt from my Informatica log file which has 4 blocks of lines (starting with WRITER_1_*_1). Like these my log file will have multiple blocks of same pattern. WRITER_1_*_1> WRT_8161 TARGET BASED COMMIT POINT Thu May 08 09:33:21 2014... (13 Replies)
Discussion started by: Ariean
13 Replies

3. Shell Programming and Scripting

Parse 2 or more files into one.

Hi, I have a really simple question...I think. I want to be able to parse two or more files into one by reading the first record from each file into new file then go back to the first file and start reading the second record in from each file into new file and so on. I am new to using awk and am... (5 Replies)
Discussion started by: qray2011
5 Replies

4. UNIX for Dummies Questions & Answers

How can i parse my Unix log files??

Hello, i would like to parse Unix log files and i would like to use a Unix syslog analyzer. I'm going to use Eucalyptus and i would like to parse its log files. Is there any open source/free syslog parser?? Thanks, in advance! (2 Replies)
Discussion started by: g_p
2 Replies

5. Shell Programming and Scripting

parse log with sed

I've been searching for an hour on how to parse a file like this: 10.200.5.83 - - "GET /portal/edits.js HTTP/1.1" 200 24324 10.200.5.83 - - "GET /portal/objects/PortalConfig.js HTTP/1.1" 200 12187 10.200.5.84 - - "GET /portal/objects/CommonDialog.js HTTP/1.1" 200 8283 10.200.5.84 - - "GET... (4 Replies)
Discussion started by: dba_frog
4 Replies

6. Shell Programming and Scripting

Parse the log file

./abc.sh started at Sun Oct 24 06:42:04 PDT 2010 Message: ======= Summary Report of NAME count ----------------------------------------------------------------- Below is the output of the SQL query :- NAME COUNT... (2 Replies)
Discussion started by: sandy1028
2 Replies

7. Shell Programming and Scripting

Help to create a script to parse log files

Hello everybody, I need some help here to create a script to parse a log file. Here is a sample of the log file : 0x42258940 (Debug) Cache SUMMARY attrs now/668 min/668 max/668. 0x42258940 (Debug) RSVD SUMMARY reserved space max requested/128 MB accounted now/0 MB 0x42258940 (Debug)... (12 Replies)
Discussion started by: Samb95
12 Replies

8. Shell Programming and Scripting

perl parse log

Hi anyone can help.how can i get all second column data in this log below?? x 799002577959.pdf, 25728 bytes, 51 tape blocks x 800002357216.pdf, 25728 bytes, 51 tape blocks x aadb090910.txt, 80424 bytes, 158 tape blocks x tsese090909.txt, 13974 bytes, 28 tape blocks (4 Replies)
Discussion started by: netxus
4 Replies

9. Shell Programming and Scripting

shell scripts that parse log files

hi all ,i would like a shell script that parses log files and checks the contents for any anonalities,please help,thanks (4 Replies)
Discussion started by: trueman82
4 Replies
Login or Register to Ask a Question