Search for multiple lines in large file Post: 302363062

Sponsored Content

Top Forums Shell Programming and Scripting Search for multiple lines in large file Post 302363062 by praveen123 on Monday 19th of October 2009 10:09:27 AM

10-19-2009

Registered User

Search for multiple lines in large file

Hi,
I have a requirement to search for a string in a large log file along with few lines before and after the the string. The following script was sufficient to search such an entry.

STRING_TO_GREP="$1"
FILE_TO_GREP="$2"
NUMBER_OF_LINES_BEFORE=$3
NUMBER_OF_LINES_AFTER=$4
for i in `grep -n "${STRING_TO_GREP}" $FILE_TO_GREP|cut -d ':' -f1`
do
echo "Line Number $i"
echo "--------------"
i1=$(($i-NUMBER_OF_LINES_BEFORE))
i2=$(($i+NUMBER_OF_LINES_AFTER))
if [ ${i1} -le 0 ]; then
i1=1
fi
sed -n "${i1},${i2}p" $FILE_TO_GREP
done

But my problem is that I do not know the value of NUMBER_OF_LINES_BEFORE and NUMBER_OF_LINES_AFTER beforehand, but needs to be identified based on the occurence of another string S1 just before the occurence of STRING_TO_GREP.

For example, if the file F1.txt contains:
a
b
c
d
e
f
g
h
i
j
c

I want to retrieve the following text based on the grep for 'f' with 'c' as the start of text and 'i' as the end. Please note that I do not want to include the c given in the end of file.
c
d
e
f
g
h
i

Thanks & Regards,
Praveen

praveen123

View Public Profile for praveen123

Find all posts by praveen123

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Large file search

How do I search through a file that is so large (over 1GB) and find out exactly on what the exact pattern is found. I've used grep and know the line exist but what it returns is not complete because there are blobs in the data and there are control characters, etc and part of the data may beyond...

2. Shell Programming and Scripting

Pattern search in multiple lines

Hi, I have to search those statements from the file which starts from "shanky"(only shanky, shanky09 or 09shanky is not allowed) and ends with ");". These two string can be in a same line or different line. And also i have to negate those lines which starts with #. Can any one please give me...

3. Shell Programming and Scripting

Script to search a large file with a list of terms in another file

Hi- I am trying to search a large file with a number of different search terms that are listed one per line in 3 different files. Most importantly I need to be able to do a case insensitive search. I have tried just using egrep -f but it doesn't seam to be able to handle the -i option when...

4. Shell Programming and Scripting

How to search for multiple lines and put them into one paragraph?

Dear all, I'm trying to manipulate a data file and putting a certain lines into one paragraph. What am I actually want to do is that search some lines in a data file. These lines begin with "1\1\GINC-" and end with "\\@" or the following two empty lines as shown in blue. A part of the text...

5. Shell Programming and Scripting

search and replace, when found, delete multiple lines, add new set of lines?

hey guys, I tried searching but most 'search and replace' questions are related to one liners. Say I have a file to be replaced that has the following: $ cat testing.txt TESTING AAA BBB CCC DDD EEE FFF GGG HHH ENDTESTING This is the input file:

6. Shell Programming and Scripting

Search and swap multiple lines in file using Perl

Hi all, I have a vcd file with a bunch of lines containing an array, like this $var wire 1 b a $end $var wire 1 c a $end $var wire 1 d a $end $var wire 1 e a $end $var wire 1 f b $end $var wire 1 g b $end $var wire 1 h b $end $var wire 1 i b $end I want it like this: $var wire 1 e a...

7. Shell Programming and Scripting

How to search multiple patterns and remove lines from a file?

Hi, I have a file content as below. Table : PAYR Displayed fields: 15 of 15 Fixed columns: 4 List width 0999...

8. Shell Programming and Scripting

Removing multiple lines from input file, if multiple lines match a pattern.

GM, I have an issue at work, which requires a simple solution. But, after multiple attempts, I have not been able to hit on the code needed. I am assuming that sed, awk or even perl could do what I need. I have an application that adds extra blank page feeds, for multiple reports, when...

9. UNIX for Beginners Questions & Answers

Search spaces in 5th column in large file

i have a file having 5 columns with more than million records. And i want to search using UNIX command to find if there are any spaces in 5th column. any please help.

LEARN ABOUT OPENDARWIN

look

LOOK(1) 						    BSD General Commands Manual 						   LOOK(1)

NAME

     look -- display lines beginning with a given string

SYNOPSIS

     look [-df] [-t termchar] string [file ...]

DESCRIPTION

     The look utility displays any lines in file which contain string as a prefix.  As look performs a binary search, the lines in file must be
     sorted.

     If file is not specified, the file /usr/share/dict/words is used, only alphanumeric characters are compared and the case of alphabetic char-
     acters is ignored.

     Options:

     -d      Dictionary character set and order, i.e. only alphanumeric characters are compared.

     -f      Ignore the case of alphabetic characters.

     -t      Specify a string termination character, i.e. only the characters in string up to and including the first occurrence of termchar are
	     compared.

     The look utility exits 0 if one or more lines were found and displayed, 1 if no lines were found, and >1 if an error occurred.

FILES

     /usr/share/dict/words  the dictionary

SEE ALSO

     grep(1), sort(1)

COMPATIBILITY

     The original manual page stated that tabs and blank characters participated in comparisons when the -d option was specified.  This was incor-
     rect and the current man page matches the historic implementation.

HISTORY

     A look utility appeared in Version 7 AT&T UNIX.

BSD
								   June 14, 1993							       BSD