How to dynamically fetch lines after a match?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to dynamically fetch lines after a match?
# 1  
Old 08-13-2014
Debian How to dynamically fetch lines after a match?

Hi Friends,

How to fetch current hour data from a log file, given below?

I want all lines after the match "Wed Aug 13 16:"

I have tried below command, but not working. If I put exact string, then it is working.
Code:
cat /iscp/user/monitor/ORA_errors |awk '/`date +%h" "%d" "%h`/,printed==999 { ++printed; print; }'

Input file:
Code:
Wed Aug 13 13:03:22 2014
Wed Aug 13 13:03:27 2014
asdd
Wed Aug 13 13:03:28 2014
Wed Aug 13 13:07:24 2014
Wed Aug 13 13:38:06 2014
rrrr
Wed Aug 13 14:00:08 2014
Wed Aug 13 14:00:09 2014
Wed Aug 13 14:00:24 2014
Wed Aug 13 14:08:43 2014
Wed Aug 13 14:30:00 2014
Wed Aug 13 14:30:03 2014
4567
Wed Aug 13 14:39:20 2014
Wed Aug 13 14:47:48 2014
Wed Aug 13 15:00:09 2014
Wed Aug 13 15:00:09 2014
Error 5
Wed Aug 13 15:00:20 2014
Wed Aug 13 15:11:10 2014
Wed Aug 13 15:43:49 2014
Wed Aug 13 16:00:08 2014
Error
Wed Aug 13 16:00:09 2014
Wed Aug 13 16:00:13 2014
Wed Aug 13 16:16:19 2014
Error
Wed Aug 13 16:51:58 2014
Wed Aug 13 17:00:09 2014
Wed Aug 13 17:00:09 2014

Output required:
Code:
Wed Aug 13 16:00:08 2014
Error
Wed Aug 13 16:00:09 2014
Wed Aug 13 16:00:13 2014
Wed Aug 13 16:16:19 2014
Error
Wed Aug 13 16:51:58 2014
Wed Aug 13 17:00:09 2014
Wed Aug 13 17:00:09 2014


Last edited by Franklin52; 08-13-2014 at 09:38 AM.. Reason: Please use code tags
# 2  
Old 08-13-2014
`date +%h" "%d" "%h` will not be executed inside the single quotes. Use the -v option to pass variables into awk, and then the match () builtin function.
# 3  
Old 08-13-2014
Hi.. can you please share sample command???
# 4  
Old 08-13-2014
Code:
awk -v DT="$(date +%h" "%d" "%h)" 'match ($0, DT) {...}' ORA_errors

# 5  
Old 08-14-2014
If these don't help, could you consider a two step process?
  1. Find the first match that you want to split on (being very specific to ensure no duplicates)
  2. Split the file into two output files on that specific marker
Would that suit or is the input file so huge that splitting it up (leaving the original and creating the two parts) would fill your filesystem?



Robin
# 6  
Old 08-14-2014
Here is a two-step process:

First goal: search for a line matching a certain expression and print all lines from then on to the end of the file. The following does this:

Code:
sed -n '/<regexp>/,$ p' /path/to/inputfile

If you, for instance, replace "<regexp>" with "foobar", then all lines, starting with the first occurrence of "foobar", to the end are being printed.

Second goal: get the correct regexp into this expression somehow. This is easy:

Code:
regexp="foobar"
sed -n '/'"$regexp"'/,$ p' /path/to/inputfile

Notice that first the single-quoted string is closed, then a double-quoted string is openend in which the variable is expanded, this is closed afterwards and then another single-quoted string is opened. This makes sure that, regardless of what is in "$regexp", the input string to sed remains as one string.

To fill the variable "$regexp" with the right content is left as an exercise to the interested reader. Note, that you may need escaping if the expression contains wildcard characters special to "sed", like "*", ".", etc..

One last thing: you should NOT USE BACKTICKS! Instead of

Code:
regexp=`date +"someformat"`

use

Code:
regexp="$(date +"someformat")"


I hope this helps.

bakunin
This User Gave Thanks to bakunin For This Post:
# 7  
Old 08-14-2014
I'm completely lost here. The date command you used in your awk script:
Code:
date +%h" "%d" "%h

will give you something like:
Code:
Aug 14 Aug

which doesn't look anything at all like the:
Code:
Wed Aug 13 16:

you said you wanted to match. And, you never specified what that string represents in relation to the current time. If you're trying to print every line in your log file starting with the 1st line that matches the current hour of the current day, that would be something like:
Code:
awk -v ts="$(date +'^%a %h %d %H:')" '$0 ~ ts{p=1}p' /iscp/user/monitor/ORA_errors

or (since there are no slash characters in the date format you're trying to match which bakunin warned you about):
Code:
sed "/$(date +'^%a %h %d %H:')/,\$p" /iscp/user/monitor/ORA_errors

Note that if the day in the timestamps in your log file has a leading space instead of a leading 0 for the 1st nine days of a month, you'll need to change the %d in the date command format string to %e.

Last edited by Don Cragun; 08-14-2014 at 02:58 PM.. Reason: fix doubled single quote
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Fetch the lines which contains special characters

Hi All, My source file contains special characters(Latin characters).I need to fetch only the lines which contains the special characters. The problem is i don't know which all latin/special characters can come in the source. Is there anyway to extract the lines which contain letters other... (3 Replies)
Discussion started by: joe!!
3 Replies

2. Shell Programming and Scripting

Match first column entries precisely and fetch whatever in front of it

Hi all I have 2 files: first file AABC TTYP JKBH CVBN NHJK KJHM Second file is AABC,XCYU,JUHD Alllele1 GACXT It is approved study TTYP,JKBH Allele2 PPRD It is clinical trial study JKBH Allele2 PPRD ... (5 Replies)
Discussion started by: Priyanka Chopra
5 Replies

3. Shell Programming and Scripting

Match columns and fetch whatever in front of it

Hi Solved these kind of issues using these codes But these are not wrking for my attached files can anybody check........ awk 'NR==FNR{X++;next}{if(X){print}}' file1 file2 awk 'NR==FNR{X=$0;next}{n=split($1,P," ");sub($1,"",$0);for(i=1;i<=n;i++){if(X]){print P,$0}}}' file1 FS="\t" file2 ... (6 Replies)
Discussion started by: Priyanka Chopra
6 Replies

4. Shell Programming and Scripting

Match words and fetch data in front of it in second column

Hi all, I have 2 files one file contain data like this in one column AST3 GSTY4 JST3 second file containign data like this in 2 columns AST3(PAXXX),GSTY4(PAXXY) it is used in diabetes KST4 it is used in blood... (6 Replies)
Discussion started by: manigrover
6 Replies

5. Shell Programming and Scripting

Match the word or words and fetch the entries

Hi all, I have 7 words Now I have 1 file which contain data in large number of rows and columns and 6th column contain any of these words or may be more than one words among above 7 words: I want script should search for the above mentioned 7 words in the 6th column ... (9 Replies)
Discussion started by: manigrover
9 Replies

6. Shell Programming and Scripting

match sentence and word adn fetch similar words in alist

Hi all, I have ot match sentence list and word list anf fetch similar words in a separate file second file with 2 columns So I want the output shuld be 2 columns like this (3 Replies)
Discussion started by: manigrover
3 Replies

7. UNIX for Dummies Questions & Answers

awk display the match and 2 lines after the match is found.

Hello, can someone help me how to find a word and 2 lines after it and then send the output to another file. For example, here is myfile1.txt. I want to search for "Error" and 2 lines below it and send it to myfile2.txt I tried with grep -A but it's not supported on my system. I tried with awk,... (4 Replies)
Discussion started by: eurouno
4 Replies

8. Shell Programming and Scripting

fetch last line no form file which is match with specific pattern by grep command

Hi i have a file which have a pattern like this Nov 10 session closed Nov 10 Nov 9 08:14:27 EST5EDT 2010 on tty . Nov 10 Oct 19 02:14:21 EST5EDT 2010 on pts/tk . Nov 10 afrtetryytr Nov 10 session closed Nov 10 Nov 10 03:21:04 EST5EDT 2010 Dec 8 Nov 10 05:03:02 EST5EDT 2010 ... (13 Replies)
Discussion started by: Himanshu_soni
13 Replies

9. Shell Programming and Scripting

Grep and fetch subsequent lines also

Hi, I need to grep a pattern and fetch subsequent lines till end of the data-set. E.g., i have a file like: AA 1111 23 34 BB 45 56 78 CC 22 44 AA 2222 78 34 56 BB 22 56 67 68 23 CC 56 78 DD 33 55 77 AA 3333 46 BB 58 79 In above file i have 3-data sets where each set starts with... (6 Replies)
Discussion started by: prvnrk
6 Replies

10. Shell Programming and Scripting

Fetch the rows with match string on a fixed lenth text file - NO delimiters

Hi I am trying to fetch the rows with match string "0000001234" Input file looks like below: 09 0 XXX 0000001234 Z 1 09 0 XXX 0000001234 Z 1 09 0 XXX 0000001234 Z 1 09 0 XXX 0000001234 Z 1 09 0 XXX 0000001234 Z 1... (6 Replies)
Discussion started by: nareshk
6 Replies
Login or Register to Ask a Question