Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Search backwards to certain string

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Search backwards to certain string
# 1  
Old 03-26-2012
Search backwards to certain string
Hi,
I'm using the following to do a backwards search of a file for a string
Code:
sed s/^M//g FILE | nawk 'c-->0;$0~s{if(b)for(c=b+1;c>1;c--)print r[(NR-c+1)%b];print;c=a}b{r[NR%b]=$0}' b=10 a=0 s="9005"|grep "policy "|sort -u |awk '{print $4}'|cut -c2-10

My issue is that because I'm looking back 10 lines it's pulling in more data than I want. The 10 lines is including lines with the word policy for other policies where I'm only interested in the first occurrence of policy in the reverse search.

So for example my string 9005 is located in 2 different parts of the file and the first occurrence works fine (because there's no line in the preceding 10 containing policy) but the second occurrence is pulling in 2 other lines other than the one I want.

I'm wondering how do I break out of the search when the first occurrence of policy is reached for each 9005 or alternatively instead of searching back 10 lines search back to the word policy for each 9005 ?

Thanks in advance.

Moderator's Comments:
Mod Comment Please use code tags!
Last edited by zaxxon; 03-26-2012 at 06:41 AM.. Reason: code tags
# 2  
Old 03-26-2012
Hi SaltyDog,

I don't understand your explanation, but to search for a word backwards, use:
Code:
$ tac infile | sed -n '/word/ { p ; q }'

This User Gave Thanks to birei For This Post:
SaltyDog
# 3  
Old 03-26-2012
Os is Sun and tac is not available.
# 4  
Old 03-26-2012
Perl, ok? You may have to install "File::ReadBackwards" module.

Code:
#!/usr/bin/perl
use File::ReadBackwards;
 
$x = File::ReadBackwards -> new('inputfile.txt');

while ( defined($line = $x->readline) )
{
  if (/pattern/) { print "$line\n" }
}

This User Gave Thanks to balajesuri For This Post:
SaltyDog
# 5  
Old 03-26-2012
i m not clear about your inputfile but maybe try this
Code:
# awk 'NR>9005-10&&NR<=9005&&$0~/policy/{x=$0}END{print x}' infile

This User Gave Thanks to ygemici For This Post:
SaltyDog
# 6  
Old 03-26-2012
Hi.

Providing representative samples of your data and expected output invites fast and accurate responses. Otherwise the answers either will use no data, or will use individual and / or eccentric datasets. Here is how I interpreted your question.

Nonstandard utility glark has options for this kind of task:
Code:
#!/usr/bin/env bash

# @(#) s1	Demonstrate extraction of lines for patterns within n lines of each other.
# See: http://www.incava.org/projects/glark/

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C glark

FILE=${1-data1}

pl " Input data file $FILE:"
cat $FILE

pl " Results:"
glark --text-color off --and 2 2009 policy $FILE

exit 0

producing sets of lines for 2009 and policy within 2 lines of one another:
Code:
% ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0.8 (lenny) 
bash GNU bash 3.2.39
glark version 1.8.0

-----
 Input data file data1:
apple
  policy
banana
cherry
date
  2009
fig
grape
kiwi
lemon
mango
nectarine
  policy
orange
  2009
peach
rhubarb

-----
 Results:
   13   policy
   14 orange
   15   2009

The glark command was in the Debian repository. Otherwise, see the web page noted in the script for more examples and downloads. The glark code is in ruby, so that needs to be available.

Best wishes ... cheers, drl
This User Gave Thanks to drl For This Post:
SaltyDog
# 7  
Old 03-26-2012
Unfortunately we don't have ruby/glark/tac etc installed which is why I was looking towards an sed/awk solution.

Sorry if the question was more confusing than necessary.

Basically here's some sample data in a file :
Code:
CS02010002 Policy 9999998599
CS13000008 Tax processing was done for 17/03/2012.
CS95869005 No BC record found. Please review urgently
CS02010002 Policy 9999998599
SS00200001 Change of adress processed
CS13000008 Tax processing was done for 18/03/2012.
CS02010002 Policy 9999999609
CS02010002 Policy 9999999619
CS02010002 Policy 9999999629
CS43500005 Payout Number A0002 is being processed now.
CS43500005 Payout Number A0003 is being processed now.
CS02010002 Policy 9999999639
CS43500005 Payout Number A0001 is being processed now.
CS02010002 Policy 9999999759
CS02010002 Policy 9999999899
CS43500005 Payout Number A0003 is being processed now.
CS13000008 Tax processing was done for 17/03/2012.
CS95869005 No BC record found. Please review urgently

The output I'm looking for is
Code:
9999998599
9999999899

corresponding to the previous policy reference before the "CS95869005 No BC record found. Please review urgently" line

but when I run the command
Code:
 sed s/^M//g test | nawk 'c-->0;$0~s{if(b)for(c=b+1;c>1;c--)print r[(NR-c+1)%b];print ;c=a}b{r[NR%b]=$0}' b=10 a=0 s="9005"|grep "Policy "|sort -u |awk '{print $3}'


I get
Code:
9999998599
9999999619
9999999629
9999999639
9999999759
9999999899

I understand why I'm getting the extra policy numbers (due to the b=10) but I can't shorten the gap as I don't know how many lines will be between the message "CS95869005 No BC record found. Please review urgently" and the previous "CS02010002 Policy " message.

This is why I was looking for a stop at the first occurrence in the backwards search or something to that effect.
Last edited by SaltyDog; 03-26-2012 at 02:02 PM.. Reason: Please use code tags for data and code samples, thank you
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to use a grep search to search for a specific string within multiple directories?

Lets say I have a massive directory which is filled with other directories all filled with different c++ scripts and I want a listing of all the scripts that contain the string: "this string". Is there a way to use a grep search for that? I tried: grep -lr "this string" * but I do not... (3 Replies)
Discussion started by: Circuits
3 Replies

2. Shell Programming and Scripting

Sed: find and replace backwards, until string

Some help please: Need to find string ||(everything in front of it)B0300|| and replace it with ||0|| globally In: 16112121||||0||0||0||0||0||52||52||0||0||0||0||1507200053342B0300||1507200053342B0300||0||0||0||0700 Out: 16112121||||0||0||0||0||0||52||52||0||0||0||0||0||0||0||0||0||0700 ... (4 Replies)
Discussion started by: drbiloukos
4 Replies

3. Shell Programming and Scripting

Search string within a file and list common words from the line having the search string

Hi, Need your help for this scripting issue I have. I am not really good at this, so seeking your help. I have a file looking similar to this: Hello, i am human and name=ABCD. How are you? Hello, i am human and name=PQRS. I am good. Hello, i am human and name=ABCD. Good bye. Hello, i... (12 Replies)
Discussion started by: royzlife
12 Replies

4. Shell Programming and Scripting

Search several string and convert into a single line for each search string using awk command AIX?.

I need to search the file using strings "Request Type" , " Request Method" , "Response Type" and by using result set find the xml tags and convert into a single line?. below are the scenarios. Cat test Nov 10, 2012 5:17:53 AM INFO: Request Type Line 1.... (5 Replies)
Discussion started by: laknar
5 Replies

5. Programming

How to search a file based on a time stamp backwards 10 seconds

Hi all, I'm after some help with this small issue which i'm struggling to work out a fix for. I have a file that contains records that all have a time stamp for each individual record, i need to search the file for a specific time stamp and then search back 10 seconds to see if the number... (2 Replies)
Discussion started by: sp3arsy
2 Replies

6. Shell Programming and Scripting

search for string and replace backwards

I'm new to Unix scripting and I'm not sure if this can be done. Example: search (grep) in a file for 'Control ID' and then replace with 4 blanks 7 bytes before 'Control ID. input "xxxxxx1234xxxxxxxControl IDxxxxxx" output: "xxxxxx xxxxxxxControl IDxxxxxx" thanks! (7 Replies)
Discussion started by: jbt828
7 Replies

7. Shell Programming and Scripting

How to search backwards in a log file by timestamp of entries?

Hello. I'm not nearly good enough with awk/perl to create the logfile scraping script that my boss is insisting we need immediately. Here is a brief 3-line excerpt from the access.log file in question (actual URL domain changed to 'aaa.com'): 209.253.130.36 - - "GET... (2 Replies)
Discussion started by: kevinmccallum
2 Replies

8. Shell Programming and Scripting

search backwards relative to a string

Hi, I have to search for first occurenceof string str1 in a file(>5GB). Now, after I have that , I have to search backwards from that offset till I get another string str2. I should also be able to get the new string str2's offset. Similarly, I look for last occurence of str1 and then... (1 Reply)
Discussion started by: finder255
1 Replies

9. Shell Programming and Scripting

Search backwards

Hi, I have a variable , lets say a=/disk1/net/first.ksh i need to grep "first.ksh" everytime "a" gets changed dynamically and i do not know how many '"/" are there in my variable. Can somebody help me out. (9 Replies)
Discussion started by: giri_luck
9 Replies

10. Shell Programming and Scripting

Perl: Search for string on line then search and replace text

Hi All, I have a file that I need to be able to find a pattern match on a line, search that line for a text pattern, and replace that text. An example of 4 lines in my file is: 1. MatchText_randomNumberOfText moreData ReplaceMe moreData 2. MatchText_randomNumberOfText moreData moreData... (4 Replies)
Discussion started by: Crypto
4 Replies
Login or Register to Ask a Question