How to tail real time file Generated every hour? Post: 302632209

Sponsored Content

Top Forums Shell Programming and Scripting How to tail real time file Generated every hour? Post 302632209 by methyl on Sunday 29th of April 2012 01:42:19 PM

04-29-2012

Registered User

Original post compared with suggested change from @spacebar:

Quote:

# Original post
while true; do newdate=$(date '+%Y%m%d') ; nowdate=$(date '+%Y%m%d-%H.log'); tail -f /usr/home/ABCD/atimo/log/$newdate/option/OP2/GatewayTransaction.$nowdate ;sleep 5 ; done ;

# Suggested change from @spacebar
while true; do newdate=`date '+%Y%m%d'`; nowdate=`date '+%Y%m%d-%H.log'`; tail -f /usr/home/ABCD/atimo/log/$newdate/option/OP2/GatewayTransaction.$nowdate; sleep 5; done;

The O/P stated that the original problem was "When creating a new file scripts not working".
Both of the above command sequences have this problem (the suggested change just makes an irrelevant change to the date command). They will both go wrong when the application starts a new file because the "tail -f" will lose context.

The big problem is because the script is using "tail -f".
There is no "quick and easy" way round this script design issue except to avoid "tail -f".

My usual approach to log file monitoring is to issue "wc -l" to find out the number of lines in the logfile, record that number in a workfile, then after the time interval calculate the new number of lines, calculate the difference and then issue "tail -n". This is a simplified view because we need to take account of the application starting a new file and will utililise the timestamp on our record file to detect the new file. The simplest approach to the "first time" condition is to do nothing except record the current state. This means that the logfile monitoring script needs to start running exactly one time increment earlier than the first report.

Ps. It always helps to know what Operating System and version you are running and what Shell you prefer.

Last edited by methyl; 04-29-2012 at 03:31 PM.. Reason: correct some ambiguity

methyl

View Public Profile for methyl

Find all posts by methyl

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Real time log file redirect

Hi all, i would like to write the shell script program, it can monitor the access_log "real time" when the access_log writing the line contain "abcdef" the program will be "COPY" this line into a file named "abcdef.txt", do the same thing if the contain "123456" "COPY" it into a file named...

2. UNIX for Dummies Questions & Answers

Live/real-time text-file updates in terminal

I want to have a terminal open and have something like a "repeating cat" command running in it for a certain text file (in particular /var/log/system.log). So my terminal will scan or cat the text file every so often or whenever the text file system.log gets written to by the system, it will...

3. Shell Programming and Scripting

How to tail -f real time file.

How to tail -f real time file. I want to tail file created last time. The server is gen new file Always. . An example file. -rw-r--r-- 1 shinnie tiituck 251M Oct 18 05:39 20111018_00.log -rw-r--r-- 1 shinnie tiituck 251M Oct 18 11:18 20111018_01.log -rw-r--r-- 1 shinnie tiituck...

4. Shell Programming and Scripting

How to convert 24 hour time to 12 hour timing?

Hi friends, I want to convert 24 hour timing to 12 hour please help me... my data file looks like this.. 13-Nov-2011 13:27:36 15.32044 72.68502 13-Nov-2011 12:08:31 15.31291 72.69807 16-Nov-2011 01:16:54 15.30844 72.74028 15-Nov-2011 20:09:25 15.35096 ...

5. Shell Programming and Scripting

How to pick only the files which are generated in one hour?

Hello Masters, I need one help. I want to copy the files which are continuously generating on one server. But this would be on hourly basis. e.g. -rw-rw-r-- 1 akore akore 0 Feb 12 03:20 test1.log -rw-rw-r-- 1 akore akore 0 Feb 12 03:42 test2.log -rw-rw-r-- 1 akore akore 0 Feb 12 04:22...

6. Shell Programming and Scripting

How do i use tail & grep in real time here??

Hi I have a file which is updated very frequently. Where in i wanted to use tail -f command in the script and wanted to grep for a particular word. But the issue is when i use tail -f filename|grep "word" ... it will show me blank until the word is found in the real time. if it shows...

7. Shell Programming and Scripting

Grep in a log file within a time range (hour)

Hi, im trying to write a grep script that returns me the last inputs added in the last hour in the log file. Literally i have nothing yet but: grep 'Line im looking for' LOGFILE.log | tail -1 this only gives me the last input, but no necessarily from the last hour. Help Please.

8. Shell Programming and Scripting

Archiving or removing few data from log file in real time

Hi, I have a log file that gets updated every second. Currently the size has grown to 20+ GB. I need to have a command/script, that will try to get the actual size of the file and will remove 50% of the data that are in the log file. I don't mind removing the data as the size has grown to huge...

LEARN ABOUT DEBIAN

logtop

LOGTOP(1)						      General Commands Manual							 LOGTOP(1)

NAME

       logtop - Realtime log line rate analyser

SYNOPSIS

       logtop [OPTIONS]

DESCRIPTION

	logtop is a System Administrator tool analyzing line rate on stdin.
	It reads on stdin and print a constantly updated result
	displaying, in columns:
	Line number, count, frequency, and the actual line.

	 $ tail -f FILE | logtop
	is the friendly version of:
	 $ watch 'tail FILE | sort | uniq -c | sort -gr'

OPTIONS

       -s, --size=K
	      Only keep K lines in memory, instead of 10000.

       -q, --quiet
	      Do not display a live view of the data, only display a top at exit.

       -l, --line-by-line=K
	      Print result line by line, in a machine friendly format, K is the number of result to print per line.

		Line by line format is : [%d %f %s	]*

		  %d : Number of occurences
		  %f : Frequency of apparition
		  %s : String (Control chars replaced by dots.

       -i, --interval=K
	      Interval between graphical updates, in seconds. Defaults to 1.

       -h, --help
	      Show summary of options.

       -v, --version
	      Show version of program.

EXAMPLES

       Here are some logtop usage examples.

       tail -f cache.log | grep -o "HIT|MISS" | logtop

       Realtime hit / miss ratio on some caching software log file.

       tail -f access.log | cut -d' ' -f1 | logtop -s 10000

       Realtime most querying IPs on your server, as long as log lines in access.log starts with the client IP.

       tail -f access.log | cut -d' ' -f7 | logtop -s 10000

       Realtime most requested web pages in a NCSA like log file.

       cat auth.log | grep -v "CRON" | grep -o ": .*" | logtop -q -s 100000

       Display a one-shot simple analyse of your auth.log.

SEE ALSO

       watch(1)

AUTHOR

       logtop was written by Julien Palard.

       This manual page was written by Julien Palard <julien@palard.fr>, for the Debian project (and may be used by others).

								  April 16, 2011							 LOGTOP(1)