Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Grep log file to get line above matching pattern

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Grep log file to get line above matching pattern
# 1  
Old 09-21-2015
Grep log file to get line above matching pattern
Hi,

I have a log file that looks like this

Code:
    "delete" : {
      "_type" : "cl",
      "_id" : "1000600000000562636",
      "_version" : 1,
      "status" : 200,
      "found" : false
    }
  }, {
    "delete" : {
      "_type" : "cl",
      "_id" : "1000600000000562643",
      "_version" : 1,
      "status" : 200,
      "found" : false
    }
  }, {
    "delete" : {
      "_type" : "cl",
      "_id" : "1000600000000562648",
      "_version" : 1,
      "status" : 404,
      "found" : false
    }
  } ]
}

For every delete block which has a status of 404 I need to get the id from that block.

So output of above log would be -1000600000000562648

Using grep I can find number of errors
Code:
grep  "\"status\" : 404" log_file | wc -l

But how do I get the value of id?

Thanks

---------- Post updated at 11:55 AM ---------- Previous update was at 11:51 AM ----------

Sorry I had a brain freeze forgot
Code:
grep -B

# 2  
Old 09-21-2015
An awk solution:-
Code:
awk -F: '
        /_id/ {
                id = $2
                gsub(/[", ]/, X, id)
                next
        }
        /status/ {
                st = $2
                gsub(/[, ]/, X, st)
                if ( st+0 == 404 )
                        print id
        }
' log_file

# 3  
Old 09-21-2015
Quick not dirty (and the same idea as Yoda's)
Code:
awk '$1~/_id/ {id=$NF} $1~/status/ && $NF~/404/ {print id}' log_file

# 4  
Old 09-21-2015
Hello Wahi80,

Following may help you too here. It will take care of following points too.
I- if any order is missing in the Input_file then it wouldn't print that specific ID and status value, let's say we have following Input_file:
Code:
cat Input_file
   "delete" : {
      "_type" : "cl",
      "_id" : "1000600000000562636",
      "_version" : 1,
      "status" : 200,
      "found" : false
    }
  }, {
    "delete" : {
      "_type" : "cl",
      "_version" : 1,
      "status" : 404,
      "found" : false
    }
  }, {
    "delete" : {
      "_type" : "cl",
      "_id" : "1000600000000562648",
      "_version" : 1,
      "status" : 404,
      "found" : false
    }
  } ]
}

Where above we can see 2nd block doesn't have any id. So following code may help then.
Code:
awk '/\"delete\"/{ID="";while($0 !~ /\}/){if($0 ~ /\"_id\"/){gsub(/\"|\,/,X,$NF);ID=$NF};if(($1 ~ /\"status\"/) && ($NF ~ /404\,/)){sub(/\,/,X,$NF);if($NF){if(ID ~ /[0-9]/){print "ID=" OFS  ID;}}};getline;}}' Input_file

Output will be as follows.
Code:
ID= 1000600000000562648

ii- If id value is NULL or not equal to digits then it will not print it as follows. Let's say following is the Input_file:
Code:
cat Input_file
    "delete" : {
      "_type" : "cl",
      "_id" : "",
      "_version" : 1,
      "status" : 404,
      "found" : false
    }
  }, {
    "delete" : {
      "_type" : "cl",
      "_id" : "1000600000000562648",
      "_version" : 1,
      "status" : 400,
      "found" : false
    }
  }, {
    "delete" : {
      "_type" : "cl",
      "_id" : "1000600000000562648",
      "_version" : 1,
      "status" : 404,
      "found" : false
    }
  } ]
}

Then same above code will give following output.
Code:
awk '/\"delete\"/{ID="";while($0 !~ /\}/){if($0 ~ /\"_id\"/){gsub(/\"|\,/,X,$NF);ID=$NF};if(($1 ~ /\"status\"/) && ($NF ~ /404\,/)){sub(/\,/,X,$NF);if($NF){if(ID ~ /[0-9]/){print "ID=" OFS  ID;}}};getline;}}' Input_file
ID= 1000600000000562648

Thanks,
R. Singh
Last edited by RavinderSingh13; 09-21-2015 at 03:37 PM..
# 5  
Old 09-21-2015
Code:
$ perl -nle 'push @B, $_; shift @B if @B>3; /status.*404/ and print $B[0] =~ /(\d+)/' wahi80.file
1000600000000562648
1000600000000562649
1000600000000562650

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Grep file starting from pattern matching line

I have a file with a list of references towards the end and want to apply a grep for some string. text .... @unnumbered References @sp 1 @paragraphindent 0 2017. @strong{Chalenski, D.A.}; Wang, K.; Tatanova, Maria; Lopez, Jorge L.; Hatchell, P.; Dutta, P.; @strong{Small airgun... (1 Reply)
Discussion started by: kristinu
1 Replies

2. UNIX for Dummies Questions & Answers

Grep -v lines starting with pattern 1 and not matching pattern 2

Hi all! Thanks for taking the time to view this! I want to grep out all lines of a file that starts with pattern 1 but also does not match with the second pattern. Example: Drink a soda Eat a banana Eat multiple bananas Drink an apple juice Eat an apple Eat multiple apples I... (8 Replies)
Discussion started by: demmel
8 Replies

3. Shell Programming and Scripting

Grep pattern after specific line number in a file

Hi guys, I am running a while loop in a script ro read a file line by line. Now I want to run a grep only on the lines below the line I am that is being read by the while loop. Eg: If my while loop is on line 4 of the file, the grep only runs below line 4 and does not include line 1,2... (3 Replies)
Discussion started by: Junaid Subhani
3 Replies

4. Shell Programming and Scripting

Sed: printing lines AFTER pattern matching EXCLUDING the line containing the pattern

'Hi I'm using the following code to extract the lines(and redirect them to a txt file) after the pattern match. But the output is inclusive of the line with pattern match. Which option is to be used to exclude the line containing the pattern? sed -n '/Conn.*User/,$p' > consumers.txt (11 Replies)
Discussion started by: essem
11 Replies

5. Shell Programming and Scripting

Scan of log file in Linux for entries in last 15 minutes for matching a pattern

Is there any way I can do scan of log file in Linux, where the log file entries for last 15 minutes can be searched for a particular pattern. The log file entries are in below format. 2014-01-27T23:08:53.924-0500 LDAP authentication error 2014-01-27T23:08:53.934-0500 LDAP authentication... (4 Replies)
Discussion started by: anandrudran
4 Replies

6. UNIX for Dummies Questions & Answers

Issues while pattern matching using grep

Hi, I have a file f1 wi the following data f1.txt ======== Report ID Report Name ----------------------------------------------------------------- Post Requests : 2 Post successes : 2 ============================================= I need to search for the... (2 Replies)
Discussion started by: RP09
2 Replies

7. Shell Programming and Scripting

HELP: Shell Script to read a Log file line by line and extract Info based on KEYWORDS matching

I have a LOG file which looks like this Import started at: Mon Jul 23 02:13:01 EDT 2012 Initialization completed in 2.146 seconds. -------------------------------------------------------------------------------- -- Import summary for Import item: PolicyInformation... (8 Replies)
Discussion started by: biztank
8 Replies

8. Shell Programming and Scripting

Grep the word from pattern line and update in subsequent lines till next pattern line reached

Hi, I have got the below requirement. please suggest. I have a file like, Processing Item is: /data/ing/cfg2/abc.txt /data/ing/cfg3/bgc.txt Processing Item is: /data/cmd/for2/ght.txt /data/kernal/config.klgt.txt I want to process the above file to get the output file like, ... (5 Replies)
Discussion started by: rbalaj16
5 Replies

9. UNIX for Dummies Questions & Answers

grep only word matching the pattern

Hi gurus, A file contains many words in format "ABC.XXXX.XXXX.X.GET.LOG" (X->varying). Now my shell script want this list (only words in formatABC.XXXX.XXXX.X.GET.LOG ) to continue the process. Pls help me. Thanks, Poova. (8 Replies)
Discussion started by: poova
8 Replies

10. Shell Programming and Scripting

comment/delete a particular pattern starting from second line of the matching pattern

Hi, I have file 1.txt with following entries as shown: 0152364|134444|10.20.30.40|015236433 0233654|122555|10.20.30.50|023365433 ** ** ** In file 2.txt I have the following entries as shown: 0152364|134444|10.20.30.40|015236433 0233654|122555|10.20.30.50|023365433... (4 Replies)
Discussion started by: imas
4 Replies
Login or Register to Ask a Question