i have a logfile which is continuously being updated. I have built a script to check for a specific content and if it is found, it sends a string into a file. Here's the current script:
As you can see i am continuously reading from the "server.log" file and while reading each entry, if the "grep" command find the matched string it echoes to file, else it echos to the same file but with a different string (YES or NO).
However i noticed that when the end of the day is reached, i.e @ 23:59, the file "/home/nms/Disconnection_oldstack.txt" stops updating. I noticed that the "server.log" file closes and a new one is reopened at that time, here below:
i tried to add
in the beginning of the script so it sleeps if file is does not exist (since i thought that at the time of the file rotation it stops since the file is not found), but the same thing happened.
Am i missing something?
Note: The output redirection to file is necessary in this case so i need to stick to that!
If I understand what is happening, the process that builds the server.log file closes the file at 23:59. It then renames the file and open a new server.log file.
I know you tried to wait a few seconds; what happens if you try to wait longer?
While the old file may close and rename, perhaps the new file does not get created until the first event to cause something to be written to it.
Unsure on what part of that sript makes it continuously read. The code snippet seems to be a one-time pass-thru.
Elaborating on what rbatte1 already alluded to above: the man page for tail (GNU coreutils) 8.25 reads:
Quote:
With --follow (-f), tail defaults to following the file descriptor, which means that even if a tail'ed file is renamed, tail will continue to track its end. This default behavior is not desirable when you really want to track the actual name of the file, not the file descriptor (e.g., log rotation). Use --follow=name in that case. That causes tail to track the named file in a way that accommodates renaming, removal and creation.
Thanks for your feedback. I modified the script to use -F instead and increased the sleep time (although this might not be really needed since -F helps in cases rotated log files.
We'll see how it goes and keep you posted.
fyi the OS is a CentOS release 6.6
I'm not sure if you expoect lots of messages to rush through at any time, but that could be painful calling grep for each and every one one.
If the string you are checking is exactly what you show (no leading date string or other stuff) might it be better to code like this:-
If the string you are looking for is only part of the line, then this might be better:-
Both of these negate the need to make the external call to grep The second one uses variable substitution to cut off the string from the end of the line so if there is a leading timestamp, that will be put into variable test_line and the comparison done , which will then go to the then section. if the alert string is not in the line, then the whole line is put into test_line and the comparison will match so we go to the else section.
I did also wonder if you really want to be overwriting the output file each time your read a record too, as this could cause a heavy IO load to re-write a single line in the file many times over. If you keep the current state in a variable, you can also avoid the unnecessary IO and only re-write the file each time the state changes.
Perhaps this would do both:-
If you just want to append to the output file, put the output redirection on the whole loop, i.e. after the done and the input redirection.
I hope that this helps,
Robin
Last edited by rbatte1; 02-15-2017 at 10:31 AM..
Reason: Added the logfile definition
Hi,
This is a bit lengthy problem, i will try to keep explaining it simple.
I have got a file say file1 that contains the following in it,
------------------------------------------------------------------------ r201463 | ngupta@gmail.com | 2012-06-19 22:02:20 +0530 (Tue, 19 Jun 2012) |... (3 Replies)
Hi All,
I m very new to unix...i jus want to chk the content of file.
ma requirement is
if file has a content then
display it
else
dont display or something
pls specify which loop shalli use either for or while?? (20 Replies)
I have some users in one unix system and i want to search some files with specific to user and then i want to find some content inside that file so can u help me how we can implement it?
File location is as below.
/pools/home_unix/cmadireddy/work/models/model/
cmadireddy is user name.
now... (6 Replies)
Hi All,
I have the file in this format
**** Results Data ****
Time or Step
1
2
20
0.000000000e+00 0s 0s 0s
1.024000000e+00 Us 0s 0s
1.100000000e+00 1s 0s 0s
1.100000001e+00 1s 0s 1s
2.024000000e+00 Us Us 1s
2.024000001e+00 ... (7 Replies)
Hi
I have a large text file and I want to split its content into multiple flies.
this large file contains several blocks of codes separated by a comment line for each block.
this comment line represents a directory path
So, when separate these blocks each into a separate file, This output... (7 Replies)
Hi All,
I would like to print the content from the specific line of a file .
For example... i have file abc.txt which has 100 lines of code ,from this file i would like to print the content from 20,19,18th line......like that
Regards
Srikanth (4 Replies)
Hi, I have a file called fl_list consists of files i have to archive. I want to create a exception parm called except_parm, so if it finds the directory it will not archive these files and remove from fl_list.
$ cat fl_list
/apps/dev/ihub/ready/IA003B/IA003B_Deal_Header_yyyymmdd_hhmmss.txt... (1 Reply)
Hi ,
I have a file called "X" . the content of X are
X
--
abc
def
and i have a file called "Y" , the content of Y are
Y
--
erty
sdss
s
abc
sfs
def
I need to check if the content of file X is contained in Y.Only unix (7 Replies)
Hi,
I am trying to find the content of file using grep and find command and list only the file names
but i am getting entire file list of files in the directory
find . -exec grep "test" {} \; -ls
Can anyone of you correct this (2 Replies)