Unix script help to read log file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Unix script help to read log file
# 1  
Old 04-20-2010
Unix script help to read log file

Hi

I have a big log file

:08,936 DEBUG [SoapUIMultiThreadedHttpConnectionManager] HttpConnectionManager.getConnection: config =
11:39:08,936 DEBUG [SoapUIMultiThreadedHttpConnectionManager] Getting free connection,
11:39:08,989 DEBUG [SoapUIMultiThreadedHttpConnectionManager] Freeing connection, hostConfig=HostConfiguration
11:39:08,989 DEBUG [SoapUIMultiThreadedHttpConnectionManager] Notifying no-one, there are no waiting threads
11:39:09,046 INFO [SoapUILoadTestRunner] LoadTest [LoadTest 1] progress: 3.3893334E-4, 1
11:39:09,104 DEBUG [SoapUIMultiThreadedHttpConnectionManager] HttpConnectionManager.getConnection: config = HostConfiguration[, timeout = 0
11:39:09,105 DEBUG [SoapUIMultiThreadedHttpConnectionManager] Getting free connection, hostConfig=HostConfiguration[host=
11:39:09,158 DEBUG [SoapUIMultiThreadedHttpConnectionManager] Freeing connection, hostConfig=HostConfiguration[
11:39:09,159 DEBUG [SoapUIMultiThreadedHttpConnectionManager] Notifying no-one, there are no waiting threads
11:39:09,282 DEBUG [SoapUIMultiThreadedHttpConnectionManager] HttpConnectionManager.getConnection: config =
11:39:09,282 DEBUG [SoapUIMultiThreadedHttpConnectionManager] Getting free connection, hostConfig=HostConfiguration
11:39:09,335 DEBUG [SoapUIMultiThreadedHttpConnectionManager] Freeing connection, hostConfig=HostConfiguration
11:39:09,335 DEBUG [SoapUIMultiThreadedHttpConnectionManager] Notifying no-one, there are no waiting threads


I want to grep the first occurrence of string "Notifying no-one, there are no waiting threads" and last occurrence and calculate the count of which this string has appeared .And calculate time diffrence.

For eg
First occurrence
11:39:09,335 DEBUG [SoapUIMultiThreadedHttpConnectionManager] Notifying no-one, there are no waiting threads

last occurrence
12:33:09,335 DEBUG [SoapUIMultiThreadedHttpConnectionManager] Notifying no-one, there are no waiting threads

Time difference is 12:33:09-11:39:09 between first occurrence and last occurrence and also count number of occurrences.


Input should be a file name
Out put must be count divided by (time diff is seconds) so that i can check number of occurrences per sec
# 2  
Old 04-20-2010
to grep the spezific line use

Code:
Code:

grep -i 'Notifying no-one, there are no waiting threads' file.txt

i don't howto calculate the time between the firt and last post
sorry
# 3  
Old 04-20-2010
Thank you .

I want to grep first occurrence of string and note down the time for it.Grep last occurrence of string and note down time .!!!!!!
# 4  
Old 04-20-2010
ok here the way:

FIRSTONE=`grep -i 'Notifying no-one, there are no waiting threads' log_file.txt | head -n1 | awk '{ print $1}'`

LASTONE=grep -i 'Notifying no-one, there are no waiting threads' log_file.txt | tail -n1 | awk '{ print $1}'

Now we have two dates

If you are on Linux convert these dates to UTC time..

FIRSTONE=`date --date="$FIRSTONE" +%s`
LASTONE=`date --date="$LASTONE" +%s`

Now subtract LASTONE from FIRST ONE ... and you get secs ...

Let me know if you need complete script..

---------- Post updated at 06:56 AM ---------- Previous update was at 06:41 AM ----------

Found similar thread
https://www.unix.com/shell-programmin...gaps-time.html
# 5  
Old 04-20-2010
Hi

Try this,
Code:
#!/bin/sh

filename=$1;

grep -i "Notifying no-one, there are no waiting threads" $filename > temp.txt

occur=`cat temp.txt | wc -l`
echo "Occurances in the file : - $occur"

time_first=`head -1 temp.txt | awk -F "," '{print $1}'`
time_last=`tail -1 temp.txt | awk -F "," '{print $1}'` 

first_sec=`date --date="$time_first" +%s`
last_sec=`date --date="$time_last" +%s`

diff=`expr $last_sec - $first_sec`
echo "Time diff $diff seconds"
rm temp.txt

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with UNIX script --Read from one file and delete entries in other

Hi Guru's The script has to read an entry from one file and delete the set of lines form other file. Below is the format of the file. In the below example, script should read the entries from input file 2 and delete the entries from input file 1. Input file 1 cn: test@test1.com abc:... (7 Replies)
Discussion started by: Samingla
7 Replies

2. Shell Programming and Scripting

Script to read a log file and run 2nd script if the dates match

# cat /tmp/checkdate.log SQL*Plus: Release 11.2.0.1.0 Production on Mon Sep 17 22:49:00 2012 Copyright (c) 1982, 2009, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production FIRST_TIME NEXT_TIME... (1 Reply)
Discussion started by: SarwalR
1 Replies

3. UNIX for Dummies Questions & Answers

Read a flat file, evaluate and create output. UNIX SCRIPT.

Hi all, I have a flat file as below; 470423495|1||TSA-A000073800||1|||1 471423495|1||TSA-A000073800||5|||5 472423495|1||TSA-A000073800||2|||7 473423495|1||TSA-A000073800||3|||3 I like to create a Unix script. The script have to valuate the last two columns, if the values are... (4 Replies)
Discussion started by: mrreds
4 Replies

4. UNIX for Advanced & Expert Users

Script to read log file

Hi, Im looking for a shell script which will search for a particular string in a log file as below scenario 1. I need to run URL http://localhost/client/update?feedid=200 in shell script at(eg)4:00 PM which will not take more than 15 mins to complete. 2. After 15 mins i need to... (6 Replies)
Discussion started by: Paulwintech
6 Replies

5. UNIX and Linux Applications

Perl Script to read an excel file into an array and search in the UNIX directories

Hi, I want the Perl script with versions 5.8.2 and 5.8.5 starting with #!/usr/bin/perl The Perl program should read the excel file or text file line by line and taking into an array and search in the UNIX directories for reference file of .jsp or .js or .xsl with path .The Object names... (2 Replies)
Discussion started by: pasam
2 Replies

6. Shell Programming and Scripting

script to constantly read the last 500 new logs in a log file

Hello, I would like to write a bash script that would monitor a log file for a certain number of logs, let's say 500 logs and when it reaches that number to write the last log to another file. For example, I want to watch the /var/adm/messages and everytime, there is 500 new logs that are... (1 Reply)
Discussion started by: Pouchie1
1 Replies

7. Shell Programming and Scripting

Unix Script to read the XML file from Website

Hi Experts, I need a unix shell script which can copy the xml file from the below pasted website and paste in in my unix directory. http://www.westpac.co.nz/olcontent/olcontent.nsf/fx.xml Thanks in Advance... (8 Replies)
Discussion started by: phani333
8 Replies

8. Shell Programming and Scripting

Help w/ script to read file and parse log message

Hi, I am working on the script to parsing the specific message like "aaaa" in multiple log files like N1-***,N2-***,N3-***... The script is to find the list of lof files which contains the message "aaaa" and export the list into excel filE. Can anyone give help? Thanks (2 Replies)
Discussion started by: shyork2001
2 Replies

9. Shell Programming and Scripting

shell script to read a line in gps receiver log file and append that line to new file

Hi, I have gps receiver log..its giving readings .like below Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. GPSD,R=1 $GPGSV,3,1,11,08,16,328,40,11,36,127,00,28,33,283,39,20,11,165,00*71... (3 Replies)
Discussion started by: gudivada213
3 Replies

10. UNIX for Dummies Questions & Answers

file read + unix script

hi, how can i read line by line from a file using unix shalle script? Thanks and Regards Vivek.S (2 Replies)
Discussion started by: vivekshankar
2 Replies
Login or Register to Ask a Question