Search: find current line, then search back


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Search: find current line, then search back
# 1  
Old 12-13-2013
Search: find current line, then search back

Hello.

I want to find a line that has "new = 0" in it, then search back based on field $4 ([Snapshot1690384351]) in the current line, and find the first line that has field $4 and "last fetch"

Grep or Awk preferred.

Here is what the data looks like:

Code:
2013-12-12 12:10:30,117 TRACE [Snapshot1690384351] [com.xxx.xxx] last fetch: Thu Dec 12 11:46:36 CST 2013, Files in root:
...
...
...
2013-12-12 12:12:17,698 DEBUG [Snapshot1690384351] [com.xxx.xxx] /tmp/snapshottmp6158934693853684856.txt diag stats, total = 238439, new = 0

Thanks in advance.

Moderator's Comments:
Mod Comment Use code tags please, see PM.

Last edited by zaxxon; 12-13-2013 at 11:21 AM.. Reason: code tags
# 2  
Old 12-13-2013
show more data and what output do you expect ?
# 3  
Old 12-13-2013
Use tac to print file in reverse and search using awk:
Code:
tac file | awk -F'[][]' '/new = 0/{v=$2;next}v==$2'

This User Gave Thanks to Yoda For This Post:
# 4  
Old 12-13-2013
Not a one-liner, but it does the job:

Code:
#!/bin/bash
#
#

# check command-line for file
if [ $# -ne 1 ]
then
    echo "Usage: ${0##*/} <file>"
    exit 1
fi

# store the filename
f=$1

# get current working directory and
# change to it
cwd=$(pwd)
cd $cwd

# retrieve the 4th field of all the lines
# with the 'new = 0' pattern and store in
# an array
declare -a PATTERN
PATTERN=( $(awk '/new = 0/{print $4}' $f) )

# bail if no matches found
if [ ! ${PATTERN[@]} ]
then
    echo "No matches found in $f."
    exit 1
fi

# iterate the array and find the matching lines
# based on the stored pattern
for p in ${PATTERN[@]}
do
    # let's escape the brackets in the pattern
    p=$(echo $p | sed 's#\([]/&[]\)#\\\1#g')

    # now build the search pattern
    srchPattern="$p.*last fetch"

    # search for the pattern in the file
    # with last fetch in the line
    awk "/$srchPattern/{print}" $f
done

# done
exit 0

getLast.sh /tmp/file.txt
2013-12-12 12:10:30,117 TRACE [Snapshot1690384351] [com.xxx.xxx] last fetch: Thu Dec 12 11:46:36 CST 2013, Files in root:

9573169c82d3bef7ecabfd699930ed75
# 5  
Old 12-13-2013
If last fetch,new=0 pairs never overlap, the problem is reduced to a trivial, sequential, single-pass solution; store a last fetch when encountered, compare the key of the last stored line to the current line if the current line matches new=0. Unfortunately, since the OP provides nothing but a scant two lines of sample data, it is unknown if this is a valid assumption.

Regards,
Alister
# 6  
Old 12-13-2013
With
Code:
grep -E "new = 0|last fetch" file | sort -k4,4
2013-12-12 12:10:30,117 TRACE [Snapshot1690384351] [com.xxx.xxx] last fetch: Thu Dec 12 11:46:36 CST 2013, Files in root:
2013-12-12 12:12:17,698 DEBUG [Snapshot1690384351] [com.xxx.xxx] /tmp/snapshottmp6158934693853684856.txt diag stats, total = 238439, new = 0
2013-12-12 12:10:30,117 TRACE [Snapshot1690384352] [com.xxx.xxx] last fetch: Thu Dec 12 11:46:36 CST 2013, Files in root:
2013-12-12 12:12:17,698 DEBUG [Snapshot1690384352] [com.xxx.xxx] /tmp/snapshottmp6158934693853684856.txt diag stats, total = 238439, new = 0

you'd always have the corresponding lines in pairs.
# 7  
Old 12-13-2013
Quote:
Originally Posted by in2nix4life
Not a one-liner, but it does the job
I disagree. One of the few things we know about the data is that a matching pair is ordered. Your code does not guarantee that a last fetch precedes its new = 0 counterpart.


Quote:
Originally Posted by in2nix4life
Code:
# get current working directory and
# change to it
cwd=$(pwd)
cd $cwd

Why change to the current directory? It's already the current directory. Am I missing something?

There is actually a bug in that cd. If the current directory is foo bar, and if there exists a directory named foo, cd $cwd will change to an unintended directory, because the variable expansion isn't double-quoted.


Quote:
Originally Posted by in2nix4life
Code:
for p in ${PATTERN[@]}
do
    ...
    awk "/$srchPattern/{print}" $f
done

If there are P patterns, P+1 passes will be necessary. Hopefully it's a small dataset with few patterns. Your approach could be reimplemented to always finish in 2 passes.


Quote:
Originally Posted by in2nix4life
Code:
    p=$(echo $p | sed 's#\([]/&[]\)#\\\1#g')

Escaping characters for a regular expression engine is a bug-prone procedure that yields brittle solutions.

A case in point: sed 's#\([]/&[]\)#\\\1#g' will add a backslash before each ampersand. Since ampersands are not an AWK ERE metacharacter, \& is an undefined sequence. An AWK implementation is allowed to either silently ignore the backslash or it can choose to abort or it can do ... whatever it wants.

I prefer to avoid this type of manual escaping as much as possible.

Regards,
Alister
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to restrict Find only search the current directory?

hello, all I have googled internet, read the man page of Find, searched this forum, but still could not figure out how. My current directory is: little@wenwen:~$ pwd /home/little little@wenwen:~$ I want to use find command to list the files in my current directory, how should i write... (3 Replies)
Discussion started by: littlewenwen
3 Replies

2. Shell Programming and Scripting

Search several string and convert into a single line for each search string using awk command AIX?.

I need to search the file using strings "Request Type" , " Request Method" , "Response Type" and by using result set find the xml tags and convert into a single line?. below are the scenarios. Cat test Nov 10, 2012 5:17:53 AM INFO: Request Type Line 1.... (5 Replies)
Discussion started by: laknar
5 Replies

3. UNIX for Dummies Questions & Answers

Restricting a Find search to the current directory only

Hi All, I am trying to delete file (with a mtime older than 2 days) from the current directory ONLY using: find . -daystart -maxdepth 1 -mtime 2 -exec rm {} \; but this doesn't seem to work it is still find files in subdirectories which I don't want to delete. Please can anyone offer... (2 Replies)
Discussion started by: daveu7
2 Replies

4. Shell Programming and Scripting

perl search and replace - search in first line and replance in 2nd line

Dear All, i want to search particular string and want to replance next line value. following is the test file. search string is tmp,??? ,10:1 "???" may contain any 3 character it should remain the same and next line replace with ,10:50 tmp,123 --- if match tmp,??? then... (3 Replies)
Discussion started by: arvindng
3 Replies

5. Shell Programming and Scripting

search for difference and find the above line

hello All, i have to two files package.today and package.yesterday , the extension of the files says what day the file belongs to . contents of the file change little bit everyday . i am taking only the package and sub package from the files awk '/^/{print $0}' Packages.today 82 ... (0 Replies)
Discussion started by: posner
0 Replies

6. Shell Programming and Scripting

How to use sed to search for string and Print previous two lines and current line

Hello, Can anybody help me to correct my sed syntax to find the string and print previous two lines and current line and next one line. i am using string as "testing" netstat -v | sed -n -e '/test/{x;2!p;g;$!N;p;D;}' -e h i am able to get the previous line current line next line but... (1 Reply)
Discussion started by: nmadhuhb
1 Replies

7. Shell Programming and Scripting

search directory-find files-append at end of line

Hi, I have a command "get_data" with some parameters in few *.text files of a directory. I want to first find those files that contain this command and then append the following parameter to the end of the command. example of an entry in the file :- get_data -x -m50 /etc/web/getid this... (1 Reply)
Discussion started by: PrasannaKS
1 Replies

8. Shell Programming and Scripting

search for the matched pattern by tracing back from the line

Hi, I want to grep the line which has 'data11'.then from that line, i need to trace back and find out the immediate line which has the same timestamp of that grepped line. for eg: log file: ----------- Process - data Process - datavalue - 2345 Process - data Process - data Process... (9 Replies)
Discussion started by: Sharmila_P
9 Replies

9. Shell Programming and Scripting

Perl: Search for string on line then search and replace text

Hi All, I have a file that I need to be able to find a pattern match on a line, search that line for a text pattern, and replace that text. An example of 4 lines in my file is: 1. MatchText_randomNumberOfText moreData ReplaceMe moreData 2. MatchText_randomNumberOfText moreData moreData... (4 Replies)
Discussion started by: Crypto
4 Replies

10. UNIX for Dummies Questions & Answers

Question about Restricting Search path of FIND to current directory

Hi, By default FIND command searches for matching files in all the subdirectories within the specified path. Is there a way to restrict FIND command's search path to only the specified directory and NOT TO scan its subdirectories. Any help would be more than appreciated. Thanks and Regards (2 Replies)
Discussion started by: super_duper_guy
2 Replies
Login or Register to Ask a Question

Featured Tech Videos