Shell Programming and Scripting

hi

i need to extract lines from multiple files to a csv file.
for example, i have these 3 files

file1.txt

Code:

date:29dec1980
caller:91245824255
called:8127766

file2.txt

Code:

date:11apr2014
caller:9155584558
called:8115478

file3.txt

Code:

date:25jun2015
caller:445225552
called:8117485

i need grep/awk to read these files and create a csv like this

output.csv

Code:

file,date,caller,called
file1.txt,29dec1980,91245824255,8127766
file2.txt,11apr2014,9155584558,8115478
file3.txt,25jun2015,445225552,8117485

i got this far

Code:

awk '/date/||/caller/||/called/ {print}' ORS=',' test01 > test012.csv

any idea?
thanks

Try:

Code:

awk '{print FILENAME,$2,$4,$6}' FS=':|\n' OFS=, RS= file*.txt

thanks Scrutinizer

is there a way to use awk recursively ?

You are welcome, yes you could do that, for what purpose?

the final idea is
i have 12 millions call recorded files. each file got a header with details of the call. files are in folders. for example, a file created at 20hr25min36sec on the 2nd of march 2010 will be in d:\2010\03\02\202536.55254.audio

i m hoping to run awk to find details like caller/called/date and create a csv file for my 12 millions files.

once i got the csv, i will need to categories the calls per department then copy files per department to a specific folder

thanks for reading

How are we supposed to determine the department from the data you have shown us?

Will there be any files in your audio directories (e.g., d:\2010\03\02\202536.55254.audio) other than .txt files in the format you showed us in post #1 in this thread?

Will filenames in the audio directories be unique across all audio directories or could the last component of the pathnames appear in multiple audio directories?

Do you want the full pathname of your files in the CSV output file or just the last component of the pathname?

Do you want one CSV file with data from all 12 million files, or do you want one CSV file for each department?

How many departments are there?

Hi Don,

below my answers

How are we supposed to determine the department from the data you have shown us?
Yes, the ""called" data will tells me what department

Will there be any files in your audio directories (e.g., d:\2010\03\02\202536.55254.audio ) other than .txt files in the format you showed us in post #1 in this thread?
no, every file will be *.audio. they can be read by notepad

Will filenames in the audio directories be unique across all audio directories or could the last component of the pathnames appear in multiple audio directories?
No they are unique. a real example is 20160317T085058.927444.audio

Do you want the full pathname of your files in the CSV output file or just the last component of the pathname?
the full pathname. a real example is E:\unarchived\20160331\20160331T074846.793023.audio

Do you want one CSV file with data from all 12 million files, or do you want one CSV file for each department?
one csv is enough, i can split afterward

How many departments are there?
Only 3.

many thanks