Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Better and efficient way to reverse search a file for first matched line number.

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Better and efficient way to reverse search a file for first matched line number.
# 1  
Old 10-07-2011
Better and efficient way to reverse search a file for first matched line number.
How to reverse search for a matched string in a file. Get line# of the first matched line. I am getting '2' into 'lineNum' variable.

But it feels like I am using too many commands. Is there a better more efficiant way to do this on Unix?

abc.log
Code:
 
aaaaaaaaaaaaa
bbbbbbbbbbbbb
ccccccccccccc
aa Started aa
ddddddddddddd

Code:
 
vStart='Started'; myLog=abc.log
lineNum=`/usr/xpg4/bin/tail -r $myLog | cat -n | sed -n "/$vStart/{p;q;}" | awk '{print $1}'`

# 2  
Old 10-07-2011
what is the output you are expecting?

--ahamed

---------- Post updated at 08:37 PM ---------- Previous update was at 08:24 PM ----------

This will print the first occurence from last i.e. output = 4
Code:
awk '{a[++i]=$0} END{for(j=i;j>0;j--){if(a[j] ~ "Started"){print j;j=0}}}' input_file

Looks like you are in solaris. Use nawk.

--ahamed

---------- Post updated at 08:42 PM ---------- Previous update was at 08:37 PM ----------

or

Code:
grep -n  Started input_file | tail -1 | cut -f1 -d":"

or

Code:
grep -n  Started input_file | sed -n '{s/\([0-9]\).*/\1/g;$p}'

--ahamed
# 3  
Old 10-07-2011
Another awk solution
Code:
vStart='Started'
awk -vn=$vStart '$0~n{p=NF+1} END{print p}' infile

# 4  
Old 10-07-2011
Code:
vStart='Started'; myLog=abc.log
awk -vn=$vStart '$0~n{print NR;exit}' $myLog

# 5  
Old 10-07-2011
Quote:
Originally Posted by kchinnam
But it feels like I am using too many commands. Is there a better more efficiant way to do this on Unix?

...<snip>...

Code:
 
vStart='Started'; myLog=abc.log
lineNum=`/usr/xpg4/bin/tail -r $myLog | cat -n | sed -n "/$vStart/{p;q;}" | awk '{print $1}'`


You don't need to use cat or awk. Just use sed's = command (which prints the current line number).
Code:
tail -r file | sed -n '/pat/{=;q;}'

If the reason you're asking about efficiency is because it's a large file, then this tail|sed approach is much better than any awk or grep solution. Why? The other suggestions must all read the file in its entirety. tail -r will seek to the end of the file and start reading there. As soon as sed finds a matching line it will exit. This will terminate tail (via SIGPIPE).

Regards,
Alister
# 6  
Old 10-07-2011
Quote:
Originally Posted by alister
You don't need to use cat or awk. Just use sed's = command (which prints the current line number).
Code:
tail -r file | sed -n '/pat/{=;q;}'

If the reason you're asking about efficiency is because it's a large file, then this tail|sed approach is much better than any awk or grep solution. Why? The other suggestions must all read the file in its entirety. tail -r will seek to the end of the file and start reading there. As soon as sed finds a matching line it will exit. This will terminate tail (via SIGPIPE).

Regards,
Alister
if the search pattern is near the at the beginning of the file "tail -r" does needs more time.
Code:
sed operates by performing the following cycle on each line of input: 
first, sed reads one line from the input stream, removes any trailing newline, and places it in the pattern space.

so tail or "tail -r" is always may not be the best solution to just sed without pipe.

regards
ygemici
# 7  
Old 10-07-2011
Quote:
Originally Posted by ygemici
if the search pattern is near the at the beginning of the file "tail -r" does needs more time.
Code:
sed operates by performing the following cycle on each line of input: 
first, sed reads one line from the input stream, removes any trailing newline, and places it in the pattern space.

so tail or "tail -r" is always may not be the best solution to just sed without pipe.

regards
ygemici
You are mistaken. Take a close look at the original code provided by the OP. It counts from the end of the file until the first match from the end.

Any approach that begins at the beginning of the file would have to read every line in the file for two reasons:

1) To be certain that the last match (first from the end of the file) has been found
2) The total number of lines in the file would be necessary to calculate a line number indexed from the end of the file (NR - x + 1).

Regards,
Alister
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Search a string inside a pattern matched block of a file

How to grep for searching a string within a begin and end pattern of a file. Sent from my Redmi 3S using Tapatalk (8 Replies)
Discussion started by: Baishali
8 Replies

2. Shell Programming and Scripting

Efficient way to search array in text file by awk

I have one array SPLNO with approx 10k numbers.Now i want to search the subscriber number from MDN.TXT file (containing approx 1.5 lac record)from the array.if subscriber number found in array it will perform below operation.my issue is that it's taking more time because for one number it's search... (6 Replies)
Discussion started by: siramitsharma
6 Replies

3. Shell Programming and Scripting

Perl - use search keywords from array and search a file and print 3rd field when matched

Hi , I have been trying to write a perl script to do this job. But i am not able to achieve the desired result. Below is my code. my $current_value=12345; my @users=("bob","ben","tom","harry"); open DBLIST,"<","/var/tmp/DBinfo"; my @input = <DBLIST>; foreach (@users) { my... (11 Replies)
Discussion started by: chidori
11 Replies

4. Shell Programming and Scripting

Need an efficient way to search for a tag in an xml file having millions of rows

Hi, I have an XML file with around 1 billion rows in it and i am trying to find the number of times a particular tag occurs in it. The solution i am using works but takes a lot of time (~1 hr) .Please help me with an efficient way to do this. Lets say the input file is <Root> ... (13 Replies)
Discussion started by: Sheel
13 Replies

5. Shell Programming and Scripting

search a string in a particular column of file and return the line number of the line

Hi All, Can you please guide me to search a string in a particular column of file and return the line number of the line where it was found using awk. As an example : abc.txt 7000,john,2,1,0,1,6 7001,elen,2,2,0,1,7 7002,sami,2,3,0,1,6 7003,mike,1,4,0,2,1 8001,nike,1,5,0,1,8... (3 Replies)
Discussion started by: arunshankar.c
3 Replies

6. Shell Programming and Scripting

how do I break line in a file when a pattern is matched ?

Hi All, I am stuck for quite sometime now. Below is a line in my file - GS|ED|001075|001081|20110626|1806|100803|X|004010ST|130|100803001 This line occurs only once and it is the second line. I have to break this line into two lines from ST (bold) such that it looks like -... (5 Replies)
Discussion started by: ihussain
5 Replies

7. Shell Programming and Scripting

Match a line in File 1 with Column in File 2 and print whole line in file 2 when matched

Hi Experts, I am very new to scripting and have a prb since few days and it is urgent to solve so much appreciated if u help me. i have 2 files file1.txt 9647810043118 9647810043126 9647810043155 9647810043161 9647810043166 9647810043185 9647810043200 9647810043203 9647810043250... (22 Replies)
Discussion started by: mustafa.abdulsa
22 Replies

8. UNIX for Dummies Questions & Answers

How to read contents of a file from a given line number upto line number again specified by user

Hello Everyone. I am trying to display contains of a file from a specific line to a specific line(let say, from line number 3 to line number 5). For this I got the shell script as shown below: if ; then if ; then tail +$1 $3 | head -n $2 else ... (5 Replies)
Discussion started by: grc
5 Replies

9. Shell Programming and Scripting

reverse search a text file from a specified line

Hello All, I have a following task that I need to accomplish through a script or program and I am looking for some help as I have exhausted my ideas. 1. given: a text file with thousands of lines 2. find: pattern A in file and get line number ( grep -n works) 3. find: the first occurence of... (14 Replies)
Discussion started by: PacificWonder
14 Replies

10. Shell Programming and Scripting

search for the matched pattern by tracing back from the line

Hi, I want to grep the line which has 'data11'.then from that line, i need to trace back and find out the immediate line which has the same timestamp of that grepped line. for eg: log file: ----------- Process - data Process - datavalue - 2345 Process - data Process - data Process... (9 Replies)
Discussion started by: Sharmila_P
9 Replies
Login or Register to Ask a Question