Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

[awk] grep a part of filename as entry file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting [awk] grep a part of filename as entry file
# 1  
Old 09-03-2012
[awk] grep a part of filename as entry file
hi all,

i need to combine these files into one csv file.

Code:
Bounce_Mail_Event_Daily_Report_01_Jul_2012.csv
Bounce_Mail_Event_Daily_Report_02_Jul_2012.csv
Bounce_Mail_Event_Daily_Report_03_Jul_2012.csv 
Bounce_Mail_Event_Daily_Report_04_Jul_2012.csv
Bounce_Mail_Event_Daily_Report_05_Jul_2012.csv
...
Bounce_Mail_Event_Daily_Report_31_Jul_2012.csv

sample of content file (Bounce_Mail_Event_Daily_Report_02_Jul_2012.csv):
Code:
Bounce Mail Daily Event Report
08/24/2012,2:44 PM
Device Hostname,Device Address,Count
server1.example.com,10.65.1.19,5053

the problem is i should get the date from file name as the timestamp and get the 4th line as it entry.

so the result will be:
Code:
01_Jul_2012,server1.example.com,10.65.1.19,3083
02_Jul_2012,server1.example.com,10.65.1.19,5053
...
31_Jul_2012,server1.example.com,10.65.1.19,4838

i could not use date from the file (line 2), because it is the date when csv file created not the date as mention on filename.

Does anyone have any suggestions how to do it with awk?

thank you.
# 2  
Old 09-03-2012
Use below code to give date from the file name into the file entry...

As you have not clearly mentioned how you want the output of your csv, You can just add what you want to add(just grep after sed).

Code:
for i in *.csv
do
Date_var=$(echo "$i" | awk -F "[_.]" '{ print $6,$7,$8 }' OFS="_" )
sed 's/server1.example.com/'$Date_var',server1.example.com/1'   
done

If you are getting your desired output or need to add some specific data just add after the sed....
Code:
sed 's/server1.example.com/'$Date_var',server1.example.com/1'  | grep "erver1.example.com" >> final_output.csv

This User Gave Thanks to pamu For This Post:
makan
# 3  
Old 09-03-2012
Code:
for i in *.csv ; do 
       echo $(basename $i .csv | cut -d_ -f6-),$(tail -1 $i)
done  > output.csv

This User Gave Thanks to mirni For This Post:
makan
# 4  
Old 09-03-2012
Try (not tested):
Code:
awk 'FILENAME!=prevfile{
match(FILENAME,/[0-9]{2}_[A-Z][a-z]{2}_[0-9]{4}/)
dt=substr(FILENAME,RSTART,RLENGTH)
prevfile=FILENAME}
FNR>=4{print dt "," $0}' Bounce_Mail_Event_Daily_Report_??_Jul_2012.csv > Bounce_Mail_Event_Monthly_Report_Jul_2012.csv

Last edited by elixir_sinari; 09-03-2012 at 03:41 AM..
This User Gave Thanks to elixir_sinari For This Post:
makan
# 5  
Old 09-03-2012
Code:
 
nawk -F"[_.]" 'BEGIN{fdate=substr(FILENAME,length(FILENAME)-14,11)}NR==4{$0=fdate","$0;print}' Bounce_Mail_Event_Daily_Report*.csv

try with awk, if you dont have nawk.
# 6  
Old 09-03-2012
Quote:
Originally Posted by itkamaraj
Code:
 
nawk -F"[_.]" 'BEGIN{fdate=substr(FILENAME,length(FILENAME)-14,11)}NR==4{$0=fdate","$0;print}' Bounce_Mail_Event_Daily_Report*.csv

try with awk, if you dont have nawk.
The BEGIN block will be executed before reading the arguments. So, FILENAME will have null value (on many awk implementations including gawk).
The FS is not required.
Last edited by elixir_sinari; 09-03-2012 at 04:24 AM..
# 7  
Old 09-03-2012
Code:
$ nawk 'BEGIN{print FILENAME}' a.txt
a.txt

FILENAME - The name of the current input file. If no files are specified on the command line,the value of FILENAME is "-". However, FILENAME is undefined inside the BEGIN block (unless set by getline).

so, iterating all the csv files, you can use for loop

Code:
 
for i in *.csv
do
nawk -F"[_.]" 'BEGIN{fdate=substr(FILENAME,length(FILENAME)-14,11)}NR==4{$0=fdate","$0;print}' $i
done

This User Gave Thanks to itkamaraj For This Post:
makan
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to grep for first entry in a file?

Hello friends, I have a question. Sometimes I have to search for an entry in a file that is repeated thousands of times. Can you tell me how to search so that i get limited results? For example: file: myfile.txt grep "hello world" myfile.txt this above grep will generate 5000... (4 Replies)
Discussion started by: DallasT
4 Replies

2. Shell Programming and Scripting

Grep/awk part of info of a column

I have a question that I am at a loss to solve. I have 3 column tab-separated data, such as: abs nmod+n+n-commitment-n 349.200023 abs nmod+n+n-a-commitment-n 333.306429 abs into+ns-j+vn-pass-rb-divide-v 295.57316 abs nmod+n+ns-commitment-n 182.085018 abs nmod+n+n-pledge-n ... (2 Replies)
Discussion started by: owwow14
2 Replies

3. Shell Programming and Scripting

Use grep/awk to remove part of column

hi all, how can i use grep or awk to clean the following input data: n<>the<>96427210 861521305 123257583 n<>obj<>79634223 861521305 79634223 n<>nmod<>68404733 861521305 68422718 where the desired results is to remove all non-numeric characters?: 96427210 861521305 123257583 ... (5 Replies)
Discussion started by: owwow14
5 Replies

4. Shell Programming and Scripting

Need to grep certain entry out of log file

This one is a bit too challenging for me... Hopefully you guys can help. Let's say I have a log file called: "$MW_HOME/user_projects/domains/IDMDomain/servers/wls_ods?/logs/wls_ods1-diagnostic.log" In this log file I want to search for "DIP-10219". When I execute this $ cat... (7 Replies)
Discussion started by: exm
7 Replies

5. Shell Programming and Scripting

Diff between grep .* file name and grep '.*' filename

Hi, Can anyone let me know what is difference between grep .* foo.c grep '.*' foo.c I am not able to understand what is exact difference. Thanks in advance (2 Replies)
Discussion started by: SasDutta
2 Replies

6. Programming

to extract all the part of the filename before a particular word in the filename

Hi All, Thanks in Advance I am working on a shell script. I need some assistance. My code: if then set "subscriber" "promplan" "mapping" "dedicatedaccount" "faflistSub" "faflistAcc" "accumulator"\ "pam_account"; for i in 1 2 3 4 5 6 7 8;... (0 Replies)
Discussion started by: aealexanderraj
0 Replies

7. UNIX for Dummies Questions & Answers

to extract all the part of the filename before a particular word in the filename

Hi All, Thanks in Advance I am working on a shell script. I need some assistance. My Requirement: 1) There are some set of files in a directory like given below OTP_UFSC_20120530000000_acc.csv OTP_UFSC_20120530000000_faf.csv OTP_UFSC_20120530000000_prom.csv... (0 Replies)
Discussion started by: aealexanderraj
0 Replies

8. UNIX Desktop Questions & Answers

How to grep for password file entry

How would I grep for password file entry without using grep 'username' /etc/passwd? perhaps with who? I want to create alias that will find the password file entry regardless of the user who is using it. Thanks (4 Replies)
Discussion started by: alis
4 Replies

9. UNIX for Dummies Questions & Answers

grep for password file entry

How would I grep for password file entry without using grep 'username' /etc/passwd? perhaps with who? I want to create alias that will find the password file entry regardless of the user who is using it. I am trying to get the same exact line from the file entry like: Name : Password : UserID... (7 Replies)
Discussion started by: alis
7 Replies

10. Shell Programming and Scripting

grep and awk showing filename in loop

I am attempting to grep a list of files for a string an and then only extract the 3rd and 4th field of from the line. That's easy. But I want to prefix the line with the filename that the information came from. for filename in `ls -1 *.txt' do grep search_text $filename | awk '{print $3"... (5 Replies)
Discussion started by: sjohns6
5 Replies
Login or Register to Ask a Question