Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Extract portion of log info based on specific word

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extract portion of log info based on specific word
# 1  
Old 09-27-2010
Question Extract portion of log info based on specific word
Hi Gurus,

I'm using HP-UX B.11.23 operating system.

I've been trying to extract a specific wording for example: "A tool used by tp produced warnings" from my below log data, but could not find a way to solve it. My intention is, if the log contain the word: "A tool used by tp produced warnings", the script will extract all the relevant information for this message.

For example:
A tool used by tp produced warnings

Output:
Code:
Wed Sep 22 16:15:25 SST 2010
Importing DEVK909562 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 4
meaning:
  A tool used by tp produced warnings

Wed Sep 22 16:16:24 SST 2010
Importing DEVK904946 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 4
meaning:
  A tool used by tp produced warnings
DONE


Sample log data:
Code:
Wed Sep 22 16:15:25 SST 2010
Importing DEVK909562 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 4
meaning:
  A tool used by tp produced warnings
DONE
Wed Sep 22 16:16:19 SST 2010
Importing DEVK909583 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 0
meaning:
  Everything OK
DONE
Wed Sep 22 16:16:24 SST 2010
Importing DEVK904946 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 4
meaning:
  A tool used by tp produced warnings
DONE
Wed Sep 22 16:16:37 SST 2010
Importing DEVK909579 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 0
meaning:
  Everything OK
DONE


What is the best way to extract portion of log information based on the above message?

Appreciate for any of your help and advice.

Thank you.

Cheers.
# 2  
Old 09-27-2010
Code:
grep -B 13 "A tool used by tp produced warnings" infile

# 3  
Old 09-27-2010
I m not clearly understand but i try something Smilie
did you mean if the log file contains process another script for details this fail issue ?

Code:
# sed '/A tool used by/N;s/.*\n\(.*\)/THIS IS A FAILURE MESSAGE -> PLEASE WAIT WRITING LOG DETAIL SCRIPT..\n\1/' logfile
Wed Sep 22 16:15:25 SST 2010
Importing DEVK909562 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 4
meaning:
THIS IS A FAILURE MESSAGE -> PLEASE WAIT WRITING LOG DETAIL SCRIPT..
DONE
Wed Sep 22 16:16:19 SST 2010
Importing DEVK909583 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 0
meaning:
  Everything OK
DONE
Wed Sep 22 16:16:24 SST 2010
Importing DEVK904946 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 4
meaning:
THIS IS A FAILURE MESSAGE -> PLEASE WAIT WRITING LOG DETAIL SCRIPT..
DONE
Wed Sep 22 16:16:37 SST 2010
Importing DEVK909579 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 0
meaning:
  Everything OK
DONE

# 4  
Old 09-27-2010
Hi All,

Thank you for your response and advice.

I've tried using GREP as suggested, but it did not work. Please see below for errors.

Code:
# grep -B 13 "A tool used by tp produced warnings" Log.22Sep2010.161517
grep: illegal option -- B
usage: grep [-E|-F] [-c|-l|-q] [-bhinsvwx] -e pattern_list...
        [-f pattern_file...] [file...]
usage: grep [-E|-F] [-c|-l|-q] [-bhinsvwx] [-e pattern_list...]
        -f pattern_file... [file...]
usage: grep [-E|-F] [-c|-l|-q] [-bhinsvwx] pattern [file...]

I've also tried using SED, but this does not produce the correct output.
Code:
sed '/A tool used by/N;s/.*\n\(.*\)/THIS IS A FAILURE MESSAGE -> PLEASE WAIT WRITING LOG DETAIL SCRIPT..\n\1/' Log.22Sep2010.161517

Sorry if my explanation was not very clear. Let me try to put it in a better way.

I wanted to extract a portion of log file if it is contains this wording: "A tool used by tp produced warnings".

For example:
If the log file contains: "A tool used by tp produced warnings", then the script should extract the following information.

First sample output:
--------------------
Code:
Wed Sep 22 16:15:25 SST 2010
Importing DEVK909562 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 4
meaning:
  A tool used by tp produced warnings
DONE
Wed Sep 22 16:16:24 SST 2010
Importing DEVK904946 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 4
meaning:
  A tool used by tp produced warnings
DONE

Or alternatively, to make it simpler, if the script could produce the following output based on the given wording, it would be sufficient.

Second sample output:
-----------------------
Code:
Wed Sep 22 16:15:25 SST 2010
Importing DEVK909562 to client 800
  A tool used by tp produced warnings
Wed Sep 22 16:16:24 SST 2010
Importing DEVK904946 to client 800
  A tool used by tp produced warnings

Appreciate if anyone could help out.

Thanks a lot.

Cheers.
# 5  
Old 09-28-2010
Code:
awk '
{s=s RS $0}
/^meaning:/ {
                getline
                if ($0~"A tool used by tp produced warnings")
                       {print s RS $0}
                else {s=""}
            }' infile

This User Gave Thanks to rdcwayx For This Post:
superHonda123
# 6  
Old 09-28-2010
Hi rdcwayx,

Thank you for your response.

Your script is working great and the output is what I'm expecting. Great.

However, there is a minor issue with the output when I run the script. The header is somehow cut off a little bit.

Please see below for its output.

Code:
ep 22 16:15:25 SST 2010
Importing DEVK909562 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 4
meaning:
  A tool used by tp produced warnings

DONE
Wed Sep 22 16:16:24 SST 2010
Importing DEVK904946 to client 800
This is tp version 372.04.57 (release 700, unicode enabled)
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
This is R3trans version 6.14 (release 700 - 05.03.09 - 08:28:00).
unicode enabled version
R3trans finished (0000).
Warning: Parameter DBLIBPATH is no longer used.
tp finished with return code: 4
meaning:
  A tool used by tp produced warnings

Would it be possible to simplified the output and only showing the following output?

Code:
Wed Sep 22 16:15:25 SST 2010
Importing DEVK909562 to client 800
  A tool used by tp produced warnings
Wed Sep 22 16:16:24 SST 2010
Importing DEVK904946 to client 800
  A tool used by tp produced warnings

Hope that it's not too much to ask for.

Thank you for your help.

Cheers.
# 7  
Old 09-28-2010
Code:
awk '
/ 2010$/||/^Importing/{s=s RS $0;next}
/^meaning:/ {
                getline
                if ($0~"A tool used by tp produced warnings")
                       {print s RS $0}
                else {s=""}
            }' infile

Wed Sep 22 16:15:25 SST 2010
Importing DEVK909562 to client 800
  A tool used by tp produced warnings

Wed Sep 22 16:16:24 SST 2010
Importing DEVK904946 to client 800
  A tool used by tp produced warnings

This User Gave Thanks to rdcwayx For This Post:
superHonda123
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Update a specific field in file with Variable value based on other Key Word

I have an input file with A=xyz B=pqr I would want the value in Second Field (xyz or pqr) updated with a value present in Shell Variable based on the value passed in the first field. (A or B ) while read line do NEW_VALUE = `some functionality done on $line` If $line=First Field-... (1 Reply)
Discussion started by: infernalhell
1 Replies

2. Shell Programming and Scripting

Extract specific lines based on another file

I have a folder containing text files. I need to extract specific lines from the files of this folder based on another file input.txt. How can I do this with awk/sed? file1 ARG 81.9 8 81.9 0 LEU 27.1 9 27.1 0 PHE .0 10 .0 0 ASP 59.8 11 59.8 0 ASN 27.6 12 27.6 0 ALA .0 13 .0 0... (5 Replies)
Discussion started by: alanmathew84
5 Replies

3. Shell Programming and Scripting

HELP: Shell Script to read a Log file line by line and extract Info based on KEYWORDS matching

I have a LOG file which looks like this Import started at: Mon Jul 23 02:13:01 EDT 2012 Initialization completed in 2.146 seconds. -------------------------------------------------------------------------------- -- Import summary for Import item: PolicyInformation... (8 Replies)
Discussion started by: biztank
8 Replies

4. Shell Programming and Scripting

Extract data based on specific search criteria

I have a huge file (about 2 millions records) contains data separated by “,” (comma). As part of the requirement, I can't change the format. The objective is to remove some of the records with the following condition. If the 23rd field on each line start with 302 , I need to remove that from the... (4 Replies)
Discussion started by: jaygamini
4 Replies

5. Shell Programming and Scripting

How to extract log info based on last month?

Hi Gurus I'm using HPUX B.11.23 ia64, the below sample log data was extracted from HPUX commnad: last -R. Sample data: root pts/ta userpc Wed Aug 11 09:46 - 20:21 (10:35) root pts/ta userpc Wed Aug 11 09:44 - 20:10 (10:34) root pts/ta userpc Wed Aug 11... (4 Replies)
Discussion started by: superHonda123
4 Replies

6. Shell Programming and Scripting

Extract all content that match exactly only specific word

Input: 21 templeta parent 35718 36554 . - . ID=parent_cluster_50.21.11; Name=Partial%20parent%20for%20training%20set; 21 templeta kids 35718 36554 . - . ID=_52; Parent=parent_cluster_5085.21.11; 21 templeta ... (7 Replies)
Discussion started by: patrick87
7 Replies

7. Shell Programming and Scripting

Extract lines of text based on a specific keyword

I regularly extract lines of text from files based on the presence of a particular keyword; I place the extracted lines into another text file. This takes about 2 hours to complete using the "sort" command then Kate's find & highlight facility. I've been reading the forum & googling and can find... (4 Replies)
Discussion started by: DionDeVille
4 Replies

8. UNIX for Dummies Questions & Answers

How to extract a portion of text from a log file

I am using Unix on Mac OS X 10.5.6. I am trying to extract the last entry of a log (text) file. As seen below, each log entry looks like the following (date and time change with each log entry): I want the script to extract everything quoted above, including the "===" dividers. ... (2 Replies)
Discussion started by: atilano
2 Replies

9. AIX

need to extract info from log files

hi guys i need to extract information from log files generated by an application. log file has the following lines for each process.. ---------------------------------------------- Fri Aug 03 12:06:43 WST 2007 INFO: Running project PROJECT1 Fri Aug 03 12:06:43 WST 2007 INFO: Source Files... (7 Replies)
Discussion started by: kirantalla
7 Replies

10. Shell Programming and Scripting

extract a portion of log file

hello, I want to grep the log file according to time and get the portion of log from one particular time to other. I can grep for individual lines by time but how should I print lines continuously from given start time till end till given end time. Appreciate your ideas, Thanks chandra (8 Replies)
Discussion started by: chandra004
8 Replies
Login or Register to Ask a Question