Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

piping from grep to awk without intermediate files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting piping from grep to awk without intermediate files
# 1  
Old 02-16-2012
Question piping from grep to awk without intermediate files
I am trying to extract the file names alone, for example "TVLI_STATS_NRT_XLSTWS03_20120215_132629.csv", from below output
which was given by the grep.

Code:
sam:/data/log: grep "C10_Subscribe.000|subscribe|newfile|" PDEWG511_TVLI_JOB_STATS.ksh.201202*
 
Output: 
 
PDEWG511_TVLI_JOB_STATS.ksh.20120214_1331.log:nazeer|Wed Feb 15 13:28:04 
2012|C10_Subscribe.000|subscribe|newfile|/opt/etl_sources/edw/stats/CF/TVLI_STATS_NRT_XLSTWS03_20120215_132629.csv|
PDEWG511_TVLI_JOB_STATS.ksh.20120212_1330.log:nazeer|Sun Feb 12 22:35:27 
2012|C10_Subscribe.000|subscribe|newfile|/opt/etl_sources/edw/stats/CF/TVLI_STATS_NRT_XLSTWS03_20120212_223336.csv|
PDEWG511_TVLI_JOB_STATS.ksh.20120212_1330.log:nazeer|Sun Feb 12 22:37:25 
2012|C10_Subscribe.000|subscribe|newfile|/opt/etl_sources/edw/stats/CF/TVLI_STATS_NRT_XLSTWS03_20120212_223527.csv|
PDEWG511_TVLI_JOB_STATS.ksh.20120212_1330.log:nazeer|Sun Feb 12 22:38:48 
2012|C10_Subscribe.000|subscribe|newfile|/opt/etl_sources/edw/stats/CF/TVLI_STATS_NRT_XLSTWS03_20120212_223725.csv|
PDEWG511_TVLI_JOB_STATS.ksh.20120212_1330.log:nazeer|Sun Feb 12 22:40:45 
2012|C10_Subscribe.000|subscribe|newfile|/opt/etl_sources/edw/stats/CF/TVLI_STATS_NRT_XLSTWS03_20120212_223848.csv|
PDEWG511_TVLI_JOB_STATS.ksh.20120212_1330.log:nazeer|Sun Feb 12 22:42:42 
2012|C10_Subscribe.000|subscribe|newfile|/opt/etl_sources/edw/stats/CF/TVLI_STATS_NRT_XLSTWS03_20120212_224046.csv|
PDEWG511_TVLI_JOB_STATS.ksh.20120212_1330.log:nazeer|Sun Feb 12 22:45:50 
2012|C10_Subscribe.000|subscribe|newfile|/opt/etl_sources/edw/stats/CF/TVLI_STATS_NRT_XLSTWS03_20120212_224415.csv|
PDEWG511_TVLI_JOB_STATS.ksh.20120212_1330.log:nazeer|Sun Feb 12 22:47:30 
2012|C10_Subscribe.000|subscribe|newfile|/opt/etl_sources/edw/stats/CF/TVLI_STATS_NRT_XLSTWS03_20120212_224550.csv|

I tried to pipe the output of the grep to awk as below which didn't worked out Smilie
Code:
 
grep "C10_Subscribe.000|subscribe|newfile|" PDEWG511_TVLI_JOB_STATS.ksh.201202* | awk -F"|" '{ print $5 }' | awk -F"/" '{ 
print $6 }' > demo.txt

I am little aware that awk needs a file to look for the field seperator.
Is there any way I can achieve my task without creating intermediate files??
# 2  
Old 02-16-2012
MySQL
When i execute your command i am getting the below output.
Quote:
CF
CF
CF
CF
CF
CF
CF
CF
Its not mandatory to have file for awk.

Is this is the expected output ?

Thanks,
Kalai
This User Gave Thanks to kalpeer For This Post:
siteregsam
# 3  
Old 02-16-2012
Hi Kalai..
I need the field(file name) next to CF, for example TVLI_STATS_NRT_XLSTWS03_20120212_224550.csv|

For me the command didn't worked. Could u plz post the command u tried...
# 4  
Old 02-16-2012
MySQL
Just modified your command little,

Quote:
grep "C10_Subscribe.000|subscribe|newfile|" inp8* | awk -F"|" '{ print $5 }' | awk -F"CF/" '{ print $2 }'
Output:
Quote:
TVLI_STATS_NRT_XLSTWS03_20120215_132629.csv
TVLI_STATS_NRT_XLSTWS03_20120212_223336.csv
TVLI_STATS_NRT_XLSTWS03_20120212_223527.csv
TVLI_STATS_NRT_XLSTWS03_20120212_223725.csv
TVLI_STATS_NRT_XLSTWS03_20120212_223848.csv
TVLI_STATS_NRT_XLSTWS03_20120212_224046.csv
TVLI_STATS_NRT_XLSTWS03_20120212_224415.csv
TVLI_STATS_NRT_XLSTWS03_20120212_224550.csv
TVLI_STATS_NRT_XLSTWS03_20120215_132629.csv
TVLI_STATS_NRT_XLSTWS03_20120212_223336.csv
TVLI_STATS_NRT_XLSTWS03_20120212_223527.csv
TVLI_STATS_NRT_XLSTWS03_20120212_223725.csv
TVLI_STATS_NRT_XLSTWS03_20120212_223848.csv
TVLI_STATS_NRT_XLSTWS03_20120212_224046.csv
TVLI_STATS_NRT_XLSTWS03_20120212_224415.csv
TVLI_STATS_NRT_XLSTWS03_20120212_224550.csv
Whats your platform ? uname -a
This User Gave Thanks to kalpeer For This Post:
siteregsam
# 5  
Old 02-16-2012
Or put all commands in one awk:
Code:
awk -F'[/|]' '/C10_Subscribe.000\|subscribe\|newfile\|/{print $(NF-1)}' infile

This User Gave Thanks to Scrutinizer For This Post:
siteregsam
# 6  
Old 02-16-2012
Code:
sed '/|$/!d;s:.*/::;s:|$::' infile

Code:
$ cat tst
PDEWG511_TVLI_JOB_STATS.ksh.20120214_1331.log:nazeer|Wed Feb 15 13:28:04 
2012|C10_Subscribe.000|subscribe|newfile|/opt/etl_sources/edw/stats/CF/TVLI_STATS_NRT_XLSTWS03_20120215_132629.csv|
PDEWG511_TVLI_JOB_STATS.ksh.20120212_1330.log:nazeer|Sun Feb 12 22:35:27 
2012|C10_Subscribe.000|subscribe|newfile|/opt/etl_sources/edw/stats/CF/TVLI_STATS_NRT_XLSTWS03_20120212_223336.csv|
PDEWG511_TVLI_JOB_STATS.ksh.20120212_1330.log:nazeer|Sun Feb 12 22:37:25 
2012|C10_Subscribe.000|subscribe|newfile|/opt/etl_sources/edw/stats/CF/TVLI_STATS_NRT_XLSTWS03_20120212_223527.csv|
PDEWG511_TVLI_JOB_STATS.ksh.20120212_1330.log:nazeer|Sun Feb 12 22:38:48 
2012|C10_Subscribe.000|subscribe|newfile|/opt/etl_sources/edw/stats/CF/TVLI_STATS_NRT_XLSTWS03_20120212_223725.csv|
PDEWG511_TVLI_JOB_STATS.ksh.20120212_1330.log:nazeer|Sun Feb 12 22:40:45 
2012|C10_Subscribe.000|subscribe|newfile|/opt/etl_sources/edw/stats/CF/TVLI_STATS_NRT_XLSTWS03_20120212_223848.csv|
PDEWG511_TVLI_JOB_STATS.ksh.20120212_1330.log:nazeer|Sun Feb 12 22:42:42 
2012|C10_Subscribe.000|subscribe|newfile|/opt/etl_sources/edw/stats/CF/TVLI_STATS_NRT_XLSTWS03_20120212_224046.csv|
PDEWG511_TVLI_JOB_STATS.ksh.20120212_1330.log:nazeer|Sun Feb 12 22:45:50 
2012|C10_Subscribe.000|subscribe|newfile|/opt/etl_sources/edw/stats/CF/TVLI_STATS_NRT_XLSTWS03_20120212_224415.csv|
PDEWG511_TVLI_JOB_STATS.ksh.20120212_1330.log:nazeer|Sun Feb 12 22:47:30 
2012|C10_Subscribe.000|subscribe|newfile|/opt/etl_sources/edw/stats/CF/TVLI_STATS_NRT_XLSTWS03_20120212_224550.csv|

Code:
$ sed '/|$/!d;s:.*/::;s:|$::' tst
TVLI_STATS_NRT_XLSTWS03_20120215_132629.csv
TVLI_STATS_NRT_XLSTWS03_20120212_223336.csv
TVLI_STATS_NRT_XLSTWS03_20120212_223527.csv
TVLI_STATS_NRT_XLSTWS03_20120212_223725.csv
TVLI_STATS_NRT_XLSTWS03_20120212_223848.csv
TVLI_STATS_NRT_XLSTWS03_20120212_224046.csv
TVLI_STATS_NRT_XLSTWS03_20120212_224415.csv
TVLI_STATS_NRT_XLSTWS03_20120212_224550.csv

Last edited by ctsgnb; 02-16-2012 at 04:41 AM..
This User Gave Thanks to ctsgnb For This Post:
siteregsam
# 7  
Old 02-16-2012
MySQL
Thanks to all...

I made a silly mistake in the command. I had wrongly specified the field position as 6 instead of 7 in awk Smilie

After correcting it, the command worked.
Sorry for the inconvenience.

---------- Post updated at 03:49 AM ---------- Previous update was at 03:48 AM ----------

Quote:
Originally Posted by siteregsam
Thanks to all...

I made a silly mistake in the command. I had wrongly specified the field position as 6 instead of 7 in awk Smilie

After correcting it, the command worked.
Sorry for the inconvenience.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. OS X (Apple)

Piping to grep with pbpaste

cat file 1 aaa 2 bbb 3 ccc 4 ddd In TextEdit, I then copy the characters “ccc” to the clipboard. The problem is that the following command gives no output: bash-3.2$ pbpaste | grep - file Desired output: 3 ccc What should the syntax be for that command? I am using MacOS El... (3 Replies)
Discussion started by: palex
3 Replies

2. Shell Programming and Scripting

Piping through grep/awk prevents file write

So, this is weird... I'm running this command: iotop -o -P -k -bt -d 5 I'd like to save the output relelvant to rsyslogd to a file, so I do this: iotop -o -P -k -bt -d 5 | grep rsyslogd >> /var/log/rsyslogd Nothing is written to the file! I can write the full output to the file: ... (2 Replies)
Discussion started by: treesloth
2 Replies

3. UNIX for Dummies Questions & Answers

Piping grep into awk, read the next line using grep

Hi, I have a number of files containing the information below. """"" Fundallinfo 6.3950 14.9715 14.0482 """"" I would like to grep for Fundallinfo and use it to read the next line? I ideally would like to read the three numbers that follow in the next line and... (2 Replies)
Discussion started by: Paul Moghadam
2 Replies

4. Shell Programming and Scripting

initializing loop to delete intermediate output files

Hi, I am running a script which produces a number of intermediate output files for each time step. is there a way to remove these intermediate files and just retain the final output at every end of the loop, like sort of an initialization process? this the inefficient way i do it. for i in... (3 Replies)
Discussion started by: ida1215
3 Replies

5. Ubuntu

Piping with grep

Hi everybody, I have a big file with blast results (if you know what this means, otherwise look at it just as a text file with a specific form). I am trying to extract some ids from within this file, which have certain parameters. For example, some Of my IDs have the term 'No hit results'... (6 Replies)
Discussion started by: frymor
6 Replies

6. Ubuntu

Piping with grep

Hi everybody, I have a big file with blast results (if you know what this means, otherwise look at it just as a text file with a specific form). I am trying to extract some ids from within this file, which have certain parameters. For example, some Of my IDs have the term 'No hit results'... (1 Reply)
Discussion started by: frymor
1 Replies

7. Shell Programming and Scripting

Piping STDOUT as pattern to grep or sed

$>cat file.txt 123 d3 234 abc 3 zyf 23 124 def 8 ghi kz0 ... ... I have the following output on the screen through <some command>. $> <some command> abc def ghi ... ... I have to search for each of these patterns in the file.txt and print the lines in file.txt matching the... (4 Replies)
Discussion started by: VNR
4 Replies

8. Shell Programming and Scripting

How to Avoid intermediate files when pipe does nt work

problem with piping one output to another.Would like to avoid the intermediate file creation.The piping does nt work on places where files have been created and goes in an endless loop. sed -e "s/^\.\///g" $LINE1| sed -e "s/_\(\)/kkk\1/g" > $file1 tr -s '_' ' ' < $file1| \ sort -n -k... (1 Reply)
Discussion started by: w020637
1 Replies

9. UNIX for Dummies Questions & Answers

Piping GREP

Hi, I need to use a double grep so to speak. I need to grep for a particular item say BOB and then for each successful result I need to grep for another item say SMITH. I tried grep "BOB" filename | grep "SMITH" but it does not seem to work. I can achieve my desired result using an... (12 Replies)
Discussion started by: mojoman
12 Replies

10. Shell Programming and Scripting

question about grep, cut, and piping

Howdy folks, I am fairly new to scripting but have lost of expirience in c++, pascal, and a few other. I am trying to complete a file search script that is sent a file name containing data to search that is arranged like this "id","name","rating" "1","bob","7" etc and an argument to... (1 Reply)
Discussion started by: dyrt
1 Replies
Login or Register to Ask a Question