Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

logfile parsing

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting logfile parsing
# 1  
Old 02-09-2010
logfile parsing
I thought I was pretty handy with awk until I got this one. Smilie

I'm trying to parse a log file where the events could have different delimiters (2 scripts is ok), the errors are spread over multiple lines, and I"m trying to figure out how to not read the same lines that have already been read.

The purpose being that when the script sees something the output will be redirected to mail/snmptrap/etc.

Code:
11/13/2009 1:05:28 PM ---------------------------------------------------------
11/13/2009 1:05:52 PM FTP Error: The password was not allowed
11/13/2009 1:05:52 PM Agile ChangeCAST failed for the following reason:^M
FTP Error: The password was not allowed
11/13/2009 1:05:52 PM There were errors found on profile <1>. The profile will be set as disabled.
11/13/2009 1:05:52 PM ---------------------------------------------------------
11/13/2009 1:28:09 PM *************** JVM Error ***************
11/13/2009 1:28:09 PM Nov 13, 2009 1:28:08 PM oracle.j2ee.rmi.RMIMessages EXCEPTION_ORIGINATES_FROM_THE_REMOTE_SERVER^M
WARNING: Exception returned by remote server: {0}^M
java.lang.NoClassDefFoundError: Lcom/agile/common/client/value/VOAudit;^M
        at java.lang.Class.getDeclaredFields0(Native Method)^M
        at java.lang.Class.privateGetDeclaredFields(Class.java:2232)^M
        at java.lang.Class.getDeclaredField(Class.java:1852)^M
        at java.io.ObjectStreamClass.getDeclaredSUID(ObjectStreamClass.java:1555)^M
        at java.io.ObjectStreamClass.access$600(ObjectStreamClass.java:47)^M
        at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:381)^M
        at java.security.AccessController.doPrivileged(Native Method)^M
        at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:373)^M
        at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:268)^M
        at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:504)^M
        at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1546)^M
        at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1460)^M
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1693)^M
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1299)^M
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1912)^M
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1836)^M
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1713)^M
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1299)^M
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:339)^M
        at com.evermind.server.rmi.RMIClientConnection.handleMethodInvocationResponse(RMIClientConnection.java:856)^M
        at com.evermind.server.rmi.RMIClientConnection.handleOrmiCommandResponse(RMIClientConnection.java:287)^M
        at com.evermind.server.rmi.RMIClientConnection.dispatchResponse(RMIClientConnection.java:242)^M
        at com.evermind.server.rmi.RMIClientConnection.processReceivedCommand(RMIClientConnection.java:224)^M
        at com.evermind.server.rmi.RMIConnection.handleCommand(RMIConnection.java:152)^M
        at com.evermind.server.rmi.RMIConnection.listenForOrmiCommands(RMIConnection.java:127)^M
        at com.evermind.server.rmi.RMIConnection.run(RMIConnection.java:107)^M
        at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:814)^M
        at java.lang.Thread.run(Thread.java:595)^M
java.lang.NoClassDefFoundError: Lcom/agile/common/client/value/VOAudit;^M
        at com.evermind.server.rmi.RMICall.EXCEPTION_ORIGINATES_FROM_THE_REMOTE_SERVER(RMICall.java:109)^M
        at com.evermind.server.rmi.RMICall.throwRecordedException(RMICall.java:125)^M
        at com.evermind.server.rmi.RMIClientConnection.obtainRemoteMethodResponse(RMIClientConnection.java:517)^M
        at com.evermind.server.rmi.RMIClientConnection.invokeMethod(RMIClientConnection.java:461)^M
        at com.evermind.server.rmi.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:63)^M
        at com.evermind.server.rmi.RecoverableRemoteInvocationHandler.invoke(RecoverableRemoteInvocationHandler.java:28)^M
        at com.evermind.server.ejb.StatefulSessionRemoteInvocationHandler.invoke(StatefulSessionRemoteInvocationHandler.java:31)^M
        at __Proxy1.sendObject(Unknown Source)^M
        at com.agile.changecast.agileobjects.bridge.AgileObjectsBridge.sendObject(AgileObjectsBridge.java:542)
11/13/2009 1:28:09 PM *****************************************
11/13/2009 2:09:39 PM FTP: PutFile failed to upload CPIMAST015788: 200.
11/13/2009 2:09:39 PM An unknown error has occurred, please notify your system administrator. ChangeCAST will now exit.

# 2  
Old 02-17-2010
Too complicated?
OK how about the best way to find the profile errors over a certain time-frame? For example if I ran the script every 15 minutes and count the times the profile alert happened.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Logfile monitoring with logfile replacement

Bonjour, I've wrote a script to monitor a logfile in realtime. It is working almost perfeclty except for two things. The script use the following technique : tail -fn0 $logfile | \ while read line ; do ... some stuff done First one, I'd like a way to end the monitoring script if a... (3 Replies)
Discussion started by: Warluck
3 Replies

2. Shell Programming and Scripting

Parsing a mixed format (flatfile+xml) logfile

I am trying to parse a file that looks like the below: There are thousands of lines like the above and the file is expected to run into hundreds of thousands. The issue i have is the mixed format of the file. If it was just an xmlfile, i would use an xmllint to parse the file. Now, i am... (11 Replies)
Discussion started by: goddevil
11 Replies

3. Shell Programming and Scripting

Parsing Logfile

Hi, I need to continuously monitor a logfile to get the log information between a process start and end. the logfile look like this abcdddddddddd cjjckkkkkkkkkkkk abc : Process started aaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbbb abc... (6 Replies)
Discussion started by: Byorg
6 Replies

4. Shell Programming and Scripting

looking for string in logfile

I have a shell script that used to look for a particular sting in the last line of a log file. Howeve this string now has moved to the 3rd or 4th line from bottom. Can anyone point me to how i easiest chage this one to look within the last frew lines (5 or so) for the particular string rather... (6 Replies)
Discussion started by: jjlinux
6 Replies

5. Shell Programming and Scripting

Logfile parsing with variable, multiple criterias among multiple lines

Hi all I've been working on a bash script parsing through debug/trace files and extracting all lines that relate to some search string. So far, it works pretty well. However, I am challenged by one requirement that is still open. What I want to do: 1) parse through a file and identify all... (3 Replies)
Discussion started by: reminder
3 Replies

6. Shell Programming and Scripting

Parsing of file for Report Generation (String parsing and splitting)

Hey guys, I have this file generated by me... i want to create some HTML output from it. The problem is that i am really confused about how do I go about reading the file. The file is in the following format: TID1 Name1 ATime=xx AResult=yyy AExpected=yyy BTime=xx BResult=yyy... (8 Replies)
Discussion started by: umar.shaikh
8 Replies

7. Shell Programming and Scripting

Perl parsing compared to Ksh parsing

#! /usr/local/bin/perl -w $ip = "$ARGV"; $rw = "$ARGV"; $snmpg = "/usr/local/bin/snmpbulkget -v2c -Cn1 -Cn2 -Os -c $rw"; $snmpw = "/usr/local/bin/snmpwalk -Os -c $rw"; $syst=`$snmpg $ip system sysName sysObjectID`; sysDescr.0 = STRING: Cisco Internetwork Operating System Software... (1 Reply)
Discussion started by: popeye
1 Replies

8. Shell Programming and Scripting

logfile

hi iam new of the ksh script.iwant in formation of how to call in logfile in ksh scripts. if the meaning in ksh. please help me thanks naveen.g (1 Reply)
Discussion started by: naveeng.81
1 Replies

9. Shell Programming and Scripting

last month's logfile

hi friends I need a shell script which will do the following Task Enter the month : if you enter 1 then it ll show you last 1 month's (starting from today).log file in the current directry. if you enter 4 then it ll show you last 4 month's (starting from today).log file in the current... (2 Replies)
Discussion started by: deep_kol
2 Replies

10. Shell Programming and Scripting

sort a logfile

Hi experts, I need help to sort a big logfile. logfile: ------- 67712 dkjd jd jj jjjj ------ kjkj jhjh kkk yggg lll hhh gffgf jj -------- i kkk kllkkl ------- Now I want every think between the "------" in one line. Normaly with paste no problem but you can see that the... (7 Replies)
Discussion started by: joerg
7 Replies
Login or Register to Ask a Question