Archiving or removing few data from log file in real time Post: 302907380

Sponsored Content

Top Forums Shell Programming and Scripting Archiving or removing few data from log file in real time Post 302907380 by rbatte1 on Friday 27th of June 2014 12:17:45 PM

06-27-2014

Moderator

Surely cat filename|wc -l adds a process and therefore considerable extra time. Would wc -l filename not be quicker?

Anyhow, is the file being appended to as in-use all the time or is it separate operations. Consider these two (probably not exactly true, but just for an example)

Code:

for i in 1 2 3 4 5
do
   echo "Hello $i" >> filename
   sleep 5
done

versus

Code:

for i in 1 2 3 4 5
do
   echo "Hello $i"
   sleep 5
done >>filename

In the first, you have five discreet "open-append and close" operations. In the second you have one, so in the gaps between the echo statements, the file remains open. If you delete the data and write the file back, where does the subsequent output go? If you rename the file, then the output follows the old file.

Like Corona688 says, we need to know what is generating the messages. It may be that you have to stop that process whilst you manipulate the file, then restart it if there is no signal you can send to get it to switch logs.

Robin

rbatte1

View Public Profile for rbatte1

Visit rbatte1's homepage!

Find all posts by rbatte1

7 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

log users real time

hi.... how i can configurator a log file on real time....on unix solaris.... thanks a lot.... Best Regards...

2. Shell Programming and Scripting

Piped open not real-time - How would one handle live data?

When I run "/etc/myApp" I am presented with continuous output, just about once per second. However when I try to get the information in Perl via a piped open, it waits till the end to give me anything... my code: open (OUTPUT,"/etc/myApp |"); while (<OUTPUT>){ print $_; }...

3. Shell Programming and Scripting

Real time log file redirect

Hi all, i would like to write the shell script program, it can monitor the access_log "real time" when the access_log writing the line contain "abcdef" the program will be "COPY" this line into a file named "abcdef.txt", do the same thing if the contain "123456" "COPY" it into a file named...

4. HP-UX

HP-UX real time audit log writing

Hey all, I have a problem I was hoping to get some help on. So I have my two auditfiles, audfile1 and audfile2 that can be written to, I want to have the text version of them write to an NFS mount that I have set up. So i already know that i can do .secure/etc/audsp audfile1 > //nfsmount/folder/...

5. Shell Programming and Scripting

[solved] How to see log in real time?

Hi people I have a bash script with a line like this: python example.py >> log & But i can't see anything in the log file while python program is running only if the program ends seems to write the log file. "$ cat log" for example don't show anything until the program ends. Is there...

6. UNIX for Advanced & Expert Users

How to read a fast written log file at Real time speed?

Hello All, I am building a real time parser for a log file in my application. The log file is continuously written at a very fast pace and gets rolled over every 10 minutes. I have measured the speed and observed that around 1000 lines are written to it every second, each line about 30-40...

7. Shell Programming and Scripting

Log all the commands input by user at real time in /var/log/messages

Below is my script to log all the command input by any user to /var/log/messages. But I cant achieve the desired output that i want. PLease see below. function log2syslog { declare COMMAND COMMAND=$(fc -ln -0) logger -p local1.notice -t bash -i -- "$USER:$COMMAND" } trap...

LEARN ABOUT PHP

touch

TOUCH(3)								 1								  TOUCH(3)

touch - Sets access and modification time of file

SYNOPSIS

       bool touch (string  $filename, [int  $time = time()], [int  $atime])

DESCRIPTION

	Attempts  to set the access and modification times of the file named in the $filename parameter to the value given in $time. Note that the
       access time is always modified, regardless of the number of parameters.

	If the file does not exist, it will be created.

PARAMETERS

	      o $filename
		- The name of the file being touched.

	      o $time
		- The touch time. If $time is not supplied, the current system time is used.

	      o $atime
		- If present, the access time of the given filename is set to the value of $atime. Otherwise, it is set to the value passed to the
		$time parameter. If neither are present, the current system time is used.

RETURN VALUES

	Returns TRUE on success or FALSE on failure.

CHANGELOG

       +--------+---------------------------------------------------+
       |Version |						    |
       |	|						    |
       |	|		     Description		    |
       |	|						    |
       +--------+---------------------------------------------------+
       | 5.3.0	|						    |
       |	|						    |
       |	|  It  became  possible  to change the modification |
       |	| time of a directory under Windows.		    |
       |	|						    |
       +--------+---------------------------------------------------+
EXAMPLES

       Example #1

	      touch(3) example

	      <?php
	      if (touch($filename)) {
		  echo $filename . ' modification time has been changed to present time';
	      } else {
		  echo 'Sorry, could not change modification time of ' . $filename;
	      }
	      ?>

       Example #2

	      touch(3) using the $time parameter

	      <?php
	      // This is the touch time, we'll set it to one hour in the past.
	      $time = time() - 3600;

	      // Touch the file
	      if (!touch('some_file.txt', $time)) {
		  echo 'Whoops, something went wrong...';
	      } else {
		  echo 'Touched file with success';
	      }
	      ?>

NOTES

       Note

	      Note that time resolution may differ from one file system to another.

       Warning

	       Prior to PHP 5.3.0 it was not possible to change the modification time of a directory with this function under Windows.

PHP Documentation Group 														  TOUCH(3)