Extract portion of log info based on specific word
Hi Gurus,
I'm using HP-UX B.11.23 operating system.
I've been trying to extract a specific wording for example: "A tool used by tp produced warnings" from my below log data, but could not find a way to solve it. My intention is, if the log contain the word: "A tool used by tp produced warnings", the script will extract all the relevant information for this message.
For example:
A tool used by tp produced warnings
Output:
Code:
Wed Sep 22 16:15:25 SST 2010
Importing DEVK909562 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 4
meaning:
A tool used by tp produced warnings
Wed Sep 22 16:16:24 SST 2010
Importing DEVK904946 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 4
meaning:
A tool used by tp produced warnings
DONE
Sample log data:
Code:
Wed Sep 22 16:15:25 SST 2010
Importing DEVK909562 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 4
meaning:
A tool used by tp produced warnings
DONE
Wed Sep 22 16:16:19 SST 2010
Importing DEVK909583 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 0
meaning:
Everything OK
DONE
Wed Sep 22 16:16:24 SST 2010
Importing DEVK904946 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 4
meaning:
A tool used by tp produced warnings
DONE
Wed Sep 22 16:16:37 SST 2010
Importing DEVK909579 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 0
meaning:
Everything OK
DONE
What is the best way to extract portion of log information based on the above message?
I m not clearly understand but i try something
did you mean if the log file contains process another script for details this fail issue ?
Code:
# sed '/A tool used by/N;s/.*\n\(.*\)/THIS IS A FAILURE MESSAGE -> PLEASE WAIT WRITING LOG DETAIL SCRIPT..\n\1/' logfile
Wed Sep 22 16:15:25 SST 2010
Importing DEVK909562 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 4
meaning:
THIS IS A FAILURE MESSAGE -> PLEASE WAIT WRITING LOG DETAIL SCRIPT..
DONE
Wed Sep 22 16:16:19 SST 2010
Importing DEVK909583 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 0
meaning:
Everything OK
DONE
Wed Sep 22 16:16:24 SST 2010
Importing DEVK904946 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 4
meaning:
THIS IS A FAILURE MESSAGE -> PLEASE WAIT WRITING LOG DETAIL SCRIPT..
DONE
Wed Sep 22 16:16:37 SST 2010
Importing DEVK909579 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 0
meaning:
Everything OK
DONE
I've tried using GREP as suggested, but it did not work. Please see below for errors.
Code:
# grep -B 13 "A tool used by tp produced warnings" Log.22Sep2010.161517
grep: illegal option -- B
usage: grep [-E|-F] [-c|-l|-q] [-bhinsvwx] -e pattern_list...
[-f pattern_file...] [file...]
usage: grep [-E|-F] [-c|-l|-q] [-bhinsvwx] [-e pattern_list...]
-f pattern_file... [file...]
usage: grep [-E|-F] [-c|-l|-q] [-bhinsvwx] pattern [file...]
I've also tried using SED, but this does not produce the correct output.
Code:
sed '/A tool used by/N;s/.*\n\(.*\)/THIS IS A FAILURE MESSAGE -> PLEASE WAIT WRITING LOG DETAIL SCRIPT..\n\1/' Log.22Sep2010.161517
Sorry if my explanation was not very clear. Let me try to put it in a better way.
I wanted to extract a portion of log file if it is contains this wording: "A tool used by tp produced warnings".
For example:
If the log file contains: "A tool used by tp produced warnings", then the script should extract the following information.
First sample output:
--------------------
Code:
Wed Sep 22 16:15:25 SST 2010
Importing DEVK909562 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 4
meaning:
A tool used by tp produced warnings
DONE
Wed Sep 22 16:16:24 SST 2010
Importing DEVK904946 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 4
meaning:
A tool used by tp produced warnings
DONE
Or alternatively, to make it simpler, if the script could produce the following output based on the given wording, it would be sufficient.
Second sample output:
-----------------------
Code:
Wed Sep 22 16:15:25 SST 2010
Importing DEVK909562 to client 800
A tool used by tp produced warnings
Wed Sep 22 16:16:24 SST 2010
Importing DEVK904946 to client 800
A tool used by tp produced warnings
Your script is working great and the output is what I'm expecting. Great.
However, there is a minor issue with the output when I run the script. The header is somehow cut off a little bit.
Please see below for its output.
Code:
ep 22 16:15:25 SST 2010
Importing DEVK909562 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 4
meaning:
A tool used by tp produced warnings
DONE
Wed Sep 22 16:16:24 SST 2010
Importing DEVK904946 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 4
meaning:
A tool used by tp produced warnings
Would it be possible to simplified the output and only showing the following output?
Code:
Wed Sep 22 16:15:25 SST 2010
Importing DEVK909562 to client 800
A tool used by tp produced warnings
Wed Sep 22 16:16:24 SST 2010
Importing DEVK904946 to client 800
A tool used by tp produced warnings
awk '
/ 2010$/||/^Importing/{s=s RS $0;next}
/^meaning:/ {
getline
if ($0~"A tool used by tp produced warnings")
{print s RS $0}
else {s=""}
}' infile
Wed Sep 22 16:15:25 SST 2010
Importing DEVK909562 to client 800
A tool used by tp produced warnings
Wed Sep 22 16:16:24 SST 2010
Importing DEVK904946 to client 800
A tool used by tp produced warnings
I have an input file with
A=xyz
B=pqr
I would want the value in Second Field (xyz or pqr) updated with a value present in Shell Variable based on the value passed in the first field. (A or B )
while read line
do
NEW_VALUE = `some functionality done on $line`
If $line=First Field-... (1 Reply)
I have a folder containing text files. I need to extract specific lines from the files of this folder based on another file input.txt. How can I do this with awk/sed?
file1
ARG 81.9 8 81.9 0
LEU 27.1 9 27.1 0
PHE .0 10 .0 0
ASP 59.8 11 59.8 0
ASN 27.6 12 27.6 0
ALA .0 13 .0 0... (5 Replies)
I have a LOG file which looks like this
Import started at: Mon Jul 23 02:13:01 EDT 2012
Initialization completed in 2.146 seconds.
--------------------------------------------------------------------------------
--
Import summary for Import item: PolicyInformation... (8 Replies)
I have a huge file (about 2 millions records) contains data separated by “,” (comma). As part of the requirement, I can't change the format. The objective is to remove some of the records with the following condition. If the 23rd field on each line start with 302 , I need to remove that from the... (4 Replies)
Hi Gurus
I'm using HPUX B.11.23 ia64, the below sample log data was extracted from HPUX commnad: last -R.
Sample data:
root pts/ta userpc Wed Aug 11 09:46 - 20:21 (10:35)
root pts/ta userpc Wed Aug 11 09:44 - 20:10 (10:34)
root pts/ta userpc Wed Aug 11... (4 Replies)
I regularly extract lines of text from files based on the presence of a particular keyword; I place the extracted lines into another text file. This takes about 2 hours to complete using the "sort" command then Kate's find & highlight facility.
I've been reading the forum & googling and can find... (4 Replies)
I am using Unix on Mac OS X 10.5.6.
I am trying to extract the last entry of a log (text) file. As seen below, each log entry looks like the following (date and time change with each log entry):
I want the script to extract everything quoted above, including the "===" dividers.
... (2 Replies)
hi guys i need to extract information from log files generated by an application.
log file has the following lines for each process..
----------------------------------------------
Fri Aug 03 12:06:43 WST 2007 INFO: Running project PROJECT1
Fri Aug 03 12:06:43 WST 2007 INFO: Source Files... (7 Replies)
hello,
I want to grep the log file according to time and get the portion of log from one particular time to other.
I can grep for individual lines by time but how should I print lines continuously from given start time till end till given end time.
Appreciate your ideas,
Thanks
chandra (8 Replies)