Redirect all logs files contents into a single log file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Redirect all logs files contents into a single log file
# 1  
Old 04-19-2012
Redirect all logs files contents into a single log file

Hi ,

I have a Data cleansing process which creates different log file for each step , when the process runs it creates following log files in below order:

p1_tranfrmr_log.txt
p1_tranfrmr_stats.txt
p2_globrtr_log.txt
p2_globrtr_stats.txt
p3_cusparse_log.txt
p3_cusparse_stats.txt
'
'
'
'
'
'
'
'
'
p41_drtranfrmr_log.txt
p41_drtranfrmr_stats.txt
p42_tranfrmr_log.txt
p42_tranfrmr_log.txt

I want to redirect all the contents of above 'log' and 'stats' of each process ( tranfrmr,globrtr, cusparse, drtranfrmr etc ) into a single file
in the above p* order ( p1,p2,p3.....p42)

i need a shell script which merges the contents of several file into a single file.

Thanks,
Sonu
# 2  
Old 04-19-2012
Code:
cat p*.txt >>accumulatedlog.txt

# 3  
Old 04-19-2012
Hi,

I tried executing the above commands but I need a 1 line space after merging the each contents of each file,

Thanks
sonu
# 4  
Old 04-19-2012
Then step through them adding a line between concatonations.
Code:
for i in p*.txt; do
   cat $i >>accumulated_logs.txt
   echo >>accumulated_logs.txt
done

# 5  
Old 04-19-2012
Code:
 
for i in p*.txt
do
    cat "$i" >> accumulatedlog.txt
    echo "" >> accumulatedlog.txt
done

# 6  
Old 04-19-2012
Code:
awk 'FNR==1 && NR!=1{print ""}1' p*.txt > accumulated_logs.txt

# 7  
Old 04-19-2012
Hi,

I tried doing above steps but it is not merging in sorted order.

For eg :it is merging the contents in below random order of file

p41_drtranfrmr_log.txt
p41_drtranfrmr_stats.txt

p3_cusparse_log.txt
p3_cusparse_stats.txt

p1_tranfrmr_log.txt
p1_tranfrmr_stats.txt

p42_tranfrmr_log.txt
p42_tranfrmr_log.txt

Thanks
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Output file name and file contents of multiple files to a single file

I am trying to consolidate multiple information files (<hostname>.Linux.nfslist) into one file so that I can import it into Excel. I can get the file contents with cat *Linux.nfslist >> nfslist.txt. I need each line prefaced with the hostname. I am unsure how to do this. --- Post updated at... (5 Replies)
Discussion started by: Kentlee65
5 Replies

2. Shell Programming and Scripting

Paste 2 single column files to a single file

Hi, I have 2 csv/txt files with single columns. I am trying to merge them using paste, but its not working.. output3.csv: flowerbomb everlon-jewelry sofft steve-madden dolce-gabbana-watchoutput2.csv: http://www1.abc.com/cms/slp/2/Flowerbomb http://www1.abc.com/cms/slp/2/Everlon-Jewelry... (5 Replies)
Discussion started by: ajayakunuri
5 Replies

3. Shell Programming and Scripting

Tailing logs from different files into one single file

Hi Please help me in finding a solution for tailing multiple log files and writing all of them into one common file. I have 4 log files with same name in 4 different folders. Whenever I post a Request - any one of these 4 log files gets updated with some log detail in the below format : ... (5 Replies)
Discussion started by: nisav
5 Replies

4. Shell Programming and Scripting

Perl script to Merge contents of 2 different excel files in a single excel file

All, I have an excel sheet Excel1.xls that has some entries. I have one more excel sheet Excel2.xls that has entries only in those cells which are blank in Excel1.xls These may be in different workbooks. They are totally independent made by 2 different users. I have placed them in a... (1 Reply)
Discussion started by: Anamika08
1 Replies

5. Debian

Logrotate truncated my log files to 0 bytes and no logs are written

Hi, Yesterday I installed and configured logrotate on my Debian machine. I was expecting this to run at 06:25 in the morning and it actually did. All my old logs were compressed and zipped but the new logs were all with size equal to 0 bytes. Processes, while still running ok, they were not... (2 Replies)
Discussion started by: pmatsinopoulos
2 Replies

6. Shell Programming and Scripting

Parsing log files, displaying logs between specific dates

Sorry, couldn't really think of a simple subject/title. So, I have a log file, and the dates are displayed like so: 2009-03-05 02:49:44 So the first and second field are the date/time. I can change them into a unix timestamp easily with: date -d "2009-03-05 02:49:44" +%s However,... (17 Replies)
Discussion started by: Rhije
17 Replies

7. Shell Programming and Scripting

Search a pattern in a file with contents in a single line

Hi all I am searching for a pattern in a file . The file content is in a single line.If am doing a grep or sed for the a particular pattern am getting whole file. I want the result in different lines. attaching the file for reference search pattern "/xxxxxx/hhhh/tttttttt/sss/" and... (4 Replies)
Discussion started by: sparks
4 Replies

8. Shell Programming and Scripting

not able to redirect the logs of a singl date in one system

Hi All, I have around 15 servers. I need to check for the error in /var/adm/messages in 15 servers of current date everyday and log it in one server. rsh is configured in all servers. The command I am using to accomplish this in shell script is rsh <remote sever> grep 'Jun 17'... (2 Replies)
Discussion started by: partha_bhunia
2 Replies

9. HP-UX

how to redirect the growing contents of log file to another file in unix

how to redirect the growing contents of log file to another file in unix (2 Replies)
Discussion started by: megh
2 Replies

10. Shell Programming and Scripting

how to divide single large log file into multiple files.

Can you please help me with writing script for following purpose. I have to divide single large web access log file into multiple log files based on dates inside the log file. For example: if data is logged in the access file for jan-10-08 , jan-11-08 , Jan-12-08 then make small log file... (1 Reply)
Discussion started by: kamleshm
1 Replies
Login or Register to Ask a Question