Dealing with log files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Dealing with log files
# 1  
Old 01-19-2009
Dealing with log files

Hi ,
My requirement is that i need to search for a number of strings in a log file and print them with line numbers.The search should be date wise.
The sample log file is :

Code:
Jan 17 02:45:34 srim6165 MQSIv500[22581]: [ID 702911 user.error] (UKBRKR1P_B.LZ_
BENCHMARKS)[30]BIP2648E: Message backed out to a queue; node 'com.morleyfm.eai.b
enchmark.RetrieveBenchmarksFromODR.EAI.BENCHMKS.WBI.WBI.FlowInput'. : UKBRKR1P_B
.40b370fc-1901-0000-0080-d51319f22e3a: /build/S500_P/src/DataFlowEngine/ImbMqInp
utNode.cpp: 3477: ImbMqInputNode::eligibleForBackout: ComIbmMQInputNode: com/mor
leyfm/eai/benchmark/RetrieveBenchmarksFromODR#FCMComposite_1_1.com/morleyfm/eai/
common/GlobalInputSub#FCMComposite_1_1

Jan 17 02:47:43 srim6165 MQSIv500[22581]: [ID 702911 user.error] (UKBRKR1P_B.LZ_
BENCHMARKS)[30]BIP2232E: Error detected whilst handling a previous error in node
 'com.morleyfm.eai.benchmark.RetrieveBenchmarksFromODR.EAI.BENCHMKS.WBI.WBI.Thro
w'. : UKBRKR1P_B.40b370fc-1901-0000-0080-d51319f22e3a: /build/S500_P/src/DataFlo
wEngine/ImbDataFlowNode.cpp: 387: ImbDataFlowNode::logExceptionList: ComIbmThrow
Node: com/morleyfm/eai/benchmark/RetrieveBenchmarksFromODR#FCMComposite_1_1.com/
morleyfm/eai/common/GlobalInputSub#FCMComposite_1_5
Jan 17 02:47:43 srim6165 MQSIv500[22581]: [ID 702911 user.error] (UKBRKR1P_B.LZ_
BENCHMARKS)[30]BIP2230E: Error detected whilst processing a message in node 'com
.morleyfm.eai.benchmark.RetrieveBenchmarksFromODR.EAI.BENCHMKS.WBI.WBI.FlowEnvir
onment'. : UKBRKR1P_B.40b370fc-1901-0000-0080-d51319f22e3a: /build/S500_P/src/Da
taFlowEngine/PluginInterface/ImbJniNode.cpp: 892: ImbJniNode::evaluate: ComIbmJn
iNode: com/morleyfm/eai/benchmark/RetrieveBenchmarksFromODR#FCMComposite_1_1.com
/morleyfm/eai/common/GlobalInputSub#FCMComposite_1_2

Jan 17 02:47:43 srim6165 MQSIv500[22581]: [ID 702911 user.error] (UKBRKR1P_B.LZ_
BENCHMARKS)[30]BIP2230E: Error detected whilst processing a message in node 'com
.morleyfm.eai.benchmark.RetrieveBenchmarksFromODR.EAI.BENCHMKS.WBI.WBI.Begin'. :
 UKBRKR1P_B.40b370fc-1901-0000-0080-d51319f22e3a: /build/S500_P/src/DataFlowEngi
ne/PluginInterface/ImbJniNode.cpp: 902: ImbJniNode::evaluate: ComIbmJniNode: com
/morleyfm/eai/benchmark/RetrieveBenchmarksFromODR#FCMComposite_1_1.com/morleyfm/
eai/common/GlobalInputSub#FCMComposite_1_8

Jan 17 02:47:43 srim6165 MQSIv500[22581]: [ID 702911 user.error] (UKBRKR1P_B.LZ_
BENCHMARKS)[30]BIP2230E: Error detected whilst processing a message in node 'com
.morleyfm.eai.benchmark.RetrieveBenchmarksFromODR.Global_Exception_Format_sub.EA
I.HUB.EXCEPTION.IN.ANY.WBI'. : UKBRKR1P_B.40b370fc-1901-0000-0080-d51319f22e3a:
/build/S500_P/src/DataFlowEngine/ImbMqOutputNode.cpp: 849: ImbMqOutputNode::eval
uate: ComIbmMQOutputNode: com/morleyfm/eai/benchmark/RetrieveBenchmarksFromODR#F
CMComposite_1_5.Global_Exception_Format_sub#FCMComposite_1_1

Jan 18 02:47:43 srim6165 MQSIv500[22581]: [ID 702911 user.error] (UKBRKR1P_B.LZ_
BENCHMARKS)[30]BIP2628E: Exception condition detected on input node 'com.morleyf
m.eai.benchmark.RetrieveBenchmarksFromODR.EAI.BENCHMKS.WBI.WBI.FlowInput'. : UKB
RKR1P_B.40b370fc-1901-0000-0080-d51319f22e3a: /build/S500_P/src/DataFlowEngine/I
mbMqInputNode.cpp: 2127: ImbMqInputNode::readQueue: ComIbmMQInputNode: com/morle
yfm/eai/benchmark/RetrieveBenchmarksFromODR#FCMComposite_1_1.com/morleyfm/eai/co
mmon/GlobalInputSub#FCMComposite_1_1

Jan 18 02:47:44 srim6165 MQSIv500[22581]: [ID 702911 user.error] (UKBRKR1P_B.LZ_
BENCHMARKS)[30]BIP2648E: Message backed out to a queue; node 'com.morleyfm.eai.b
enchmark.RetrieveBenchmarksFromODR.EAI.BENCHMKS.WBI.WBI.FlowInput'. : UKBRKR1P_B
.40b370fc-1901-0000-0080-d51319f22e3a: /build/S500_P/src/DataFlowEngine/ImbMqInp
utNode.cpp: 3477: ImbMqInputNode::eligibleForBackout: ComIbmMQInputNode: com/mor
leyfm/eai/benchmark/RetrieveBenchmarksFromODR#FCMComposite_1_1.com/morleyfm/eai/
common/GlobalInputSub#FCMComposite_1_1

Jan 18 08:42:17 srim6165 MQSIv500[22576]: [ID 702911 user.error] (UKBRKR1P_B.ROU
TING2)[38]BIP2232E: Error detected whilst handling a previous error in node 'com
.morleyfm.eai.routers.JPMCentralGatewayHandler.EAI.COMMON_NON_SEQ.JPM.WBI.Throw'
. : UKBRKR1P_B.13e26cfc-1901-0000-0080-d51319f22e3a: /build/S500_P/src/DataFlowE
ngine/ImbDataFlowNode.cpp: 387: ImbDataFlowNode::logExceptionList: ComIbmThrowNo
de: com/morleyfm/eai/routers/JPMCentralGatewayHandler#FCMComposite_1_1.com/morle
yfm/eai/common/subflows/CommonInputSub#FCMComposite_1_5

Now i want to search that string for date Jan 17..How do I extract the contents of Jan 17 using shell script so that i can go ahead with the search ?

Thanks in advance !!
# 2  
Old 01-20-2009
You can grep for "Jan 17".
Quote:
cat log.file | grep "Jan 17"
# 3  
Old 01-20-2009
Hi ,
Thanks for the reply.But this will only give the first line containing Jan 17 from the log file.I need all the lines.
# 4  
Old 01-20-2009
Try awk.
Code:
awk '$1==month && $2!=day{f=0}$1==month && $2==day{f=1}f' month=Jan day=17 file

# 5  
Old 01-20-2009
Thanks danmero !!...
Can I pass the month and date as command line argument in this code as i will be using this awk in a shell script.
# 6  
Old 01-20-2009
Hi,
Thanks alot.The Problem is solved now.
# 7  
Old 01-20-2009
Code:
cat log.file | grep -B 5 "Jan 17"

This will print below 5 lines of the search.

-nilesh
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Programming

Dealing with XML comments

I'm writing my own simple XML parser as an experiment. It's a lot more complicated than it's supposed to be. Things supposedly forbidden in XML comments happen all the time in the wild. You're never, ever supposed to find -- inside <!-- xml comments --> but in practice, you don't just find... (2 Replies)
Discussion started by: Corona688
2 Replies

2. Shell Programming and Scripting

Dealing with edge-list

I have an edge-list with nodes, edge.txt A B B J J H C A G H G A A C K G I have another file which tells me which of these nodes are important, input.txt G C A (3 Replies)
Discussion started by: Sanchari
3 Replies

3. UNIX for Dummies Questions & Answers

Dealing with sum

I have file input 1/1/2013 1AS030A 0 1083 CHINA 1/1/2013 1AS030B 0 675 KOREA 1/1/2013 1AS035A 162 662 CHINA 1/1/2013 1AS035B 51 799 INDIA 1/1/2013 1AS035C 0 731 CHINA 1/2/2013 1AS073A 10 1375 KOREA... (5 Replies)
Discussion started by: radius
5 Replies

4. Shell Programming and Scripting

Dealing with multiple files

Korn Shell I have hundreds of small files like below created every day. A midnight cron job moves them to the location /u04/temp/logs But sometimes I have to manually move these files based a certain dates or time. I have two basic requirements 1.Using mv command I want to move all .dat... (2 Replies)
Discussion started by: kraljic
2 Replies

5. UNIX for Dummies Questions & Answers

Dealing with Empty files, AWK and Loops

I write this bit of code to calculate the mean and variance for all the files in one directory and print the mean and variance in a separate folder but with the same file name. FILES="data/*" for X in $FILES do name=$(basename $X) awk '{x=$0; s+=$0; n++} END{mean=s/n; for (i in x){ss... (20 Replies)
Discussion started by: A-V
20 Replies

6. Shell Programming and Scripting

Iterating over subdirectories and dealing with files within them

Hello, I am working on a coding project for a class and to test the program I have created, I have come up with 100 different test cases. The program takes four text files as input, so each of the test cases is contained in a folder with four files. I have a folder called 'tests', within which... (1 Reply)
Discussion started by: dpryor
1 Replies

7. Programming

Need help with Card Dealing Program

I'm currently making a card dealing program, it is suppose to display a list of cards like this: "Ace of Heart, is red" "Two of Heart, is red" . . "Ace of Spade, is black" and so on for all suits and numbers. here is my current code: #include <stdio.h> #include <stdlib.h> #include... (3 Replies)
Discussion started by: Izzy123
3 Replies

8. Shell Programming and Scripting

Dealing with files with spaces in the name

Hello, I'm a computer science major and I'm having problems dealing with file names with spaces in them. Particularly I'm saving a file name in a variable and then using the variable in a compare function i.e. a='te xt.txt' b='file2.txt' cmp $a $b If anyone could help me with this particular... (10 Replies)
Discussion started by: jakethegreycat
10 Replies

9. Shell Programming and Scripting

perl: When dealing with files that do not exist

I have a process run weekly where I must convert data formats for about thirty files. I read a text file that provides all of the filenames and switch settings. My perl code is: for ($j = 1; $j <= $k; $j++) { open(FIN2,$fin2) || die "open: $!"; do other stuff } Every once in... (2 Replies)
Discussion started by: joeyg
2 Replies

10. Shell Programming and Scripting

Help in dealing with arra

I am readinga file lin by line and based craeting a arry of unique elemenst from the second column of the line. However when i coem out of the while loop my array becomes empty , can eny one tell me what I would be doing wrong #!/bin/bash logfile="./mylog.dat" begin=100 end="$(( $begin +... (5 Replies)
Discussion started by: jojan
5 Replies
Login or Register to Ask a Question