Tailing logs from different files into one single file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Tailing logs from different files into one single file
# 1  
Old 11-07-2013
Tailing logs from different files into one single file

Hi

Please help me in finding a solution for tailing multiple log files and writing all of them into one common file.

I have 4 log files with same name in 4 different folders.
Whenever I post a Request - any one of these 4 log files gets updated with some log detail in the below format :

blah|blah|blah|blah|blah

ex:
Code:
/Top/Folder1/logfile.log
/Top/Folder2/logfile.log
/Top/Folder3/logfile.log
/Top/Folder4/logfile.log

Logging can happen in any of the above 4 logfile.log instances each time a request is sent to the Server.

I check if logging happens or not by tailing all the above 4 instances in 4 different windows which is very ineffective.

I want to make this simple by writing some script that will look up if any of the log files get updated whenever a request is sent & then write that line into one common file.

ex :
Code:
tail -f /Top/Folder1/logfile.log >> file.log
tail -f /Top/Folder2/logfile.log >> file.log
tail -f /Top/Folder3/logfile.log >> file.log
tail -f /Top/Folder4/logfile.log >> file.log

I want to tail only that common file each time to capture my Logs.

Thanks
S

Last edited by vbe; 11-07-2013 at 09:58 AM..
# 2  
Old 11-07-2013
one method can be to writer the script which look the modification time and if it is different from the last modified time then tail last four line of that log file into common log file.
# 3  
Old 11-07-2013
Are you saying that your tail -f commands are ineffective? Is it that you actually need to put al of these into the background so that they are all running:-
Code:
tail -f /Top/Folder1/logfile.log >> file.log &
tail -f /Top/Folder2/logfile.log >> file.log &
tail -f /Top/Folder3/logfile.log >> file.log &
tail -f /Top/Folder4/logfile.log >> file.log &

What output are you getting in file.log and what's not?



Robin
Liverpool/Blackburn
UK
# 4  
Old 11-07-2013
Have you considered multitail ?

Regards
Peasant.
# 5  
Old 11-07-2013
Sorry, I meant to say that tailing all the 4 logs in 4 different windows is not an efective way to verify.

Yes I want to put them in Background and Only tail file.log
Is there a way I can write these tail cmds in one script & execute that script whenever I want

Sorry I am new to this & trying to learn

---------- Post updated at 11:31 AM ---------- Previous update was at 11:30 AM ----------

Quote:
Originally Posted by Peasant
Have you considered multitail ?

Regards
Peasant.
No I haven't. Could you please help me how to ?

---------- Post updated at 12:09 PM ---------- Previous update was at 11:31 AM ----------

Thanks All, the following works the way I want it
Code:
tail -f /Top/Folder1/logfile.log -f /Top/Folder2/logfile.log -f /Top/Folder3/logfile.log -f /Top/Folder4/logfile.log

BUT is there a way to tail all file.log files inside ALL the sub folders under /Top in just one line :

Code:
tail -f  /Top/*/file.log

* --> All subfolders under /Top

Last edited by Scott; 11-07-2013 at 01:12 PM.. Reason: Code tags, please
# 6  
Old 11-07-2013
Well, you could do it in two steps:-
Code:
for file in /Top/*/logfile.log
do
   logfiles="$logfiles -f $file"
done

if [ ! -z "$logfiles" ]  # Just in case none are found
then
   tail $logfiles        # Implicit -f already in the $logfiles variable
fi

If the file filename expansion becomes too long, you may hit problems and have to convert it a little.


Does this do what you need?



Robin
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Output file name and file contents of multiple files to a single file

I am trying to consolidate multiple information files (<hostname>.Linux.nfslist) into one file so that I can import it into Excel. I can get the file contents with cat *Linux.nfslist >> nfslist.txt. I need each line prefaced with the hostname. I am unsure how to do this. --- Post updated at... (5 Replies)
Discussion started by: Kentlee65
5 Replies

2. Shell Programming and Scripting

Paste 2 single column files to a single file

Hi, I have 2 csv/txt files with single columns. I am trying to merge them using paste, but its not working.. output3.csv: flowerbomb everlon-jewelry sofft steve-madden dolce-gabbana-watchoutput2.csv: http://www1.abc.com/cms/slp/2/Flowerbomb http://www1.abc.com/cms/slp/2/Everlon-Jewelry... (5 Replies)
Discussion started by: ajayakunuri
5 Replies

3. Shell Programming and Scripting

Concatenating 3 files into a single file

I have 3 files File1 C1 C2 c3 File 2 C1 c2 c3 File 3 C1 c2 c3 Now i want to have File1 as C1 c2 c3 I File2 as C1 c2 c3 O File3 as c1 c2 c3 D and these 3 files should be concatenated into a single file how can it be done in unix script? (3 Replies)
Discussion started by: Codesearcher
3 Replies

4. Shell Programming and Scripting

Redirect all logs files contents into a single log file

Hi , I have a Data cleansing process which creates different log file for each step , when the process runs it creates following log files in below order: p1_tranfrmr_log.txt p1_tranfrmr_stats.txt p2_globrtr_log.txt p2_globrtr_stats.txt p3_cusparse_log.txt p3_cusparse_stats.txt ' '... (8 Replies)
Discussion started by: sonu_pal
8 Replies

5. UNIX for Dummies Questions & Answers

tailing a file which contains Control chracters

Hi. I have a log file which gets updated by a java process and it uses ASCII STX and ETX characters (i.e CTRL-B and CTRL-C characters) to demarcate each XML message logged. so the format of the file is something like STX XML_MESSAGE1 .. .. ETX STX XML_MESSAGE2 .. .. ETX each XML... (4 Replies)
Discussion started by: gregoryp
4 Replies

6. Shell Programming and Scripting

Tailing last modified part of log file

I have a log file which contains data like this This log file is updated twice a day at 7am and 6pm, I want a script(which i will make run at 7:10am and 6:10pm) which should fetch only the last appended lines since last update.. I mean.. if i execute the script at 7.10am 3/3/2010 it... (4 Replies)
Discussion started by: user__user3110
4 Replies

7. Shell Programming and Scripting

Tailing new log file & echo the string on console

Guys, I do have a script that runs to take the server out from network, after running the script it is writing the new log file{outFile} in to directory . Now what i need is my script should tail the last modified file{outFile} & search the string {Server Status} ans should echo the same at the... (0 Replies)
Discussion started by: raghunsi
0 Replies

8. Shell Programming and Scripting

How to put multiple logs in single line?

Hi, I have a script which generates a log file on demand by redirecting the output to a log file. But the requirement is : echo A echo B echo C should print A B C not like A B C and this has to be handled in script only..not in the shell level (while running the script only) .... (5 Replies)
Discussion started by: bhaskar_m
5 Replies

9. Shell Programming and Scripting

Tailing 2 or more log files simultaneously PERL

Hi, I am having issue where I have to tail 3 log files continuously (forever) and while I am reading the files , parse them and shove the data into DB. I can do this with one file totally fine but how can I read 3 files at the same time? I am not really looking for code (but would be nice) but... (3 Replies)
Discussion started by: Dabheeruz
3 Replies

10. UNIX for Dummies Questions & Answers

tailing logs

Hi I'd like to achieve the ff functionality; tail -f log | grep keyword ...... and then perform a function. That is, I like to tail a log and when a certain keyword appears I then want my script to play an audio file for example. Any ideas?? Cheers M (1 Reply)
Discussion started by: squeakywheel
1 Replies
Login or Register to Ask a Question