The UNIX and Linux Forums  
Hello and Welcome from United States to the UNIX and Linux Forums! Thank You for Visiting and Joining Our Global Community.

Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
.
google unix.com



Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
extracting data from files.. anchal_khare Shell Programming and Scripting 1 04-03-2008 06:56 AM
extracting integer from data grotesque Shell Programming and Scripting 4 01-18-2008 08:18 AM
Value too large to be stored in data type??? limame AIX 5 06-18-2007 10:35 PM
Extracting certain data from a sentence dbrundrett Shell Programming and Scripting 7 12-17-2003 11:22 AM
gzip:Value too large for defined data type dangral UNIX for Dummies Questions & Answers 1 10-14-2003 04:11 PM

Closed Thread
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish Powered by Powered by Google
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 07-12-2007
bas bas is offline
Registered User
  
 

Join Date: Jul 2007
Posts: 5
Extracting data from large logs.

Hi all,
I have a large log file that gets created daily.

I need to be able to pull text out of the log when I hit a pattern including the 7 lines above it and the 3 lines below and output it to a new text file.

Line 1
Line 2
Line 3
Line 4
Line 5
Line 6
Line 7
Pattern
Line 9
Line 10
Line 11

I've tried using grep, awk and sed but am still a newbie and am having troubles.
Any help would be appreciated.
  #2 (permalink)  
Old 07-12-2007
Nick31 Nick31 is offline
Registered User
  
 

Join Date: May 2007
Posts: 8
Well, you could do it in perl by building an array of 11 strings, and push in your lines until your array is full. Then check the 8th string for your pattern. If it doesn't match, pop the first one out, push a new one in, check again. When it does match, print out your array. It's a brute force method, but it should work.

You could do this same concept in awk or sed using buffers.

You might also try grep -p if you can identify a way to divide your lines into paragraphs. For example, if every log entry starts with a line of "++++++" and the text you're searching for is the 8th line of the log entry, do:
grep -p"++++++" "searchstring" logfile

That would be the easiest method.
  #3 (permalink)  
Old 07-12-2007
bas bas is offline
Registered User
  
 

Join Date: Jul 2007
Posts: 5
Thanks for the reply Nick.

The extract I need does have a pattern of "========", but so do all the unwanted sections.

I didn't know I could put multiple lines into a buffer which may be of some help.

If I use the grep method, it will take almost the entire log as output unless I can specify it to only keep the section with the pattern I'm looking for.
  #4 (permalink)  
Old 07-12-2007
Nick31 Nick31 is offline
Registered User
  
 

Join Date: May 2007
Posts: 8
That's exactly what the -p option does. Think of it as instead of searching line by line, search section by section, with the section divider being "=======", for whatever. Instead of just printing the one line where the pattern exists, it prints the one section.

If you're having problems getting the sections to break up correctly, try using fgrep -p. You might need to include an entire line of the "=" as the argument to -p to get it to work. I have a log file where the divider is a line of * characters, and I remember having to play around with it for a while until I found the combination that worked.
  #5 (permalink)  
Old 07-12-2007
ghostdog74 ghostdog74 is offline Forum Advisor  
Registered User
  
 

Join Date: Sep 2006
Posts: 2,509
man your grep and see if there's a -A or -B option.
  #6 (permalink)  
Old 07-13-2007
Klashxx's Avatar
Klashxx Klashxx is offline Forum Advisor  
HP-UX/Linux/Oracle
  
 

Join Date: Feb 2006
Location: Almerķa, Spain
Posts: 393
An awk solution:
Code:
awk '
BEGIN {i=1}
{
vec[i]=$0
i++
if ( $0 ~ /Pattern/ ) {
          for (j=i-8;j<i;j++)
             print vec[j]
          for (j=1;j<=3;j++)
             {
             getline
             print
             }
          i=1
          }
}' file

Last edited by Klashxx; 07-13-2007 at 05:20 AM..
Closed Thread

Bookmarks

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On




All times are GMT -4. The time now is 03:09 PM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0