ksh - truncate a log in place, sed -i not available captain!


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting ksh - truncate a log in place, sed -i not available captain!
# 1  
Old 03-06-2008
ksh - truncate a log in place, sed -i not available captain!

Hi thereSmilie

Because of security requirements, It would be much better if I could truncate my logs in place using sed -i (or ?). I cant use the -i option on sed in my environment.

Can anyone help a DBA?
# 2  
Old 03-06-2008
If by truncate you mean set file to length zero:
Code:
> filename

Also, perl supports an -i option for edit in place.
# 3  
Old 03-06-2008
Here is a sample of the code I should have supplied to begin with:
## Truncate Log
LogMax=5000
LogLines=12
while [ "`wc -l $LOGFILE | awk '{print $1}'`" -gt "$LogMax" ]
do
sed "1,$LogLines d" < $LOGFILE
done

I am a perl newb (actually have not touched it yet - any hints?)
# 4  
Old 03-06-2008
It won't work without using a temporary file, as sed doesn't - out of security reasons - support in-place-editing. Neither does any other system-level command with the possible exception of editors like vi, but they do in fact create temp files too (vi does exactly that in /var/tmp) and move them over the old file at the end of the editing session.

At least you could eliminate the necessity for the loop and do it with a single temp-file:

Code:
typeset -i LogMax=5000
typeset -i LogLines=12
typeset -i LinesInFile=$(wc -l $LOGFILE)
typeset -i LinesToRemove=$(( LinesInFile - LogMax ))

if [ $LinesToRemove -gt 0 ] ; then
     (( LinesToRemove = ( ( LinesToRemove + LogLines - 1 ) / LogLines ) * LogLines ))
     tail -n $(( LinesInFile - LinesToRemove )) $LOGFILE > tempfile
     cat tempfile > $LOGFILE
fi

The reason for cat-ing the file back into $LOGFILE instead of moving it is to preserve the inode number (some processes don't like it being changed).

I hope this helps.

bakunin
# 5  
Old 03-06-2008
FWIW - here is a utility some folks here use for removing the top lines of files. It's in C.
Play with it first to make sure it works for you.
To compile with either gcc or cc:
Code:
[g]cc -o trunc trunc.c

Code:
/* truncfile.c

function: truncfile(char *filename, size_t lines)
	      reduce the number of lines in a text file keep the newer lines,
	      discarding the older lines.  Makes changes to files using memory
	      without tmpfiles.
Parameters:
	      [filename] file path 
	      [lines] to keep - valid range 0 <-> n.
	      If lines > number of existing lines no truncation occurs.
	      If lines = 0 all lines are removed.
Return value: 0 = success -1 == failure.		      
Hist:
	01-13-02 - jmc  truncfile(), reset_file_start()
	07-27-06 - BH  move into standalone utility trunc
	               usage ./trunc <filename> <lines to keep>
*/	      

#include <sys/types.h>
#include <unistd.h>
#include <sys/stat.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <errno.h>

#define FAILURE (-1)
#define SUCCESS 0
void usage(void);

/* number of bytes in a file */
size_t filesize(int fd)
{
	struct stat st;	
	st.st_size=0;
	return (fstat(fd, &st)== FAILURE)?0 :st.st_size;
}

/* remove top lines of file to result in size_t lines in new file */
char *reset_file_start(char *p, char *buf, size_t lines, size_t *filebytes)
{
	size_t count=0;
	size_t left=0;

	p=buf;
	for(p+=*filebytes; p > buf && count < lines; p--, left++)
		count+=(*p=='\n') ?1 :0;
	*filebytes=left;
	
	return p;
}

/*  truncate a file in memory  to have n lines, keep newest lines*/
int truncfile(const char *filename, const size_t lines)
{
	int retval=0;
	FILE *fp=fopen(filename, "r");

	if (fp !=NULL)
	{
		size_t filebytes=filesize(fileno(fp));
		char *buf=malloc(filebytes+1);

		if(buf==NULL ) /* no memory */
		{
			perror("Error");
			exit(1);
		}
		memset(buf, 0x0, filebytes + 1 );
		if(fread(buf, 1, filebytes, fp) == filebytes)
		{
			char *p=reset_file_start(p, buf, lines, &filebytes);

			fclose(fp);
	        if ((fp=fopen(filename,"w")) == NULL)
				retval=FAILURE;
				
			if (retval == SUCCESS && filebytes > 0)
			{
				if(fwrite(p, 1, filebytes, fp) != filebytes)			   
			    	retval=FAILURE;
			}
			
			if(retval == SUCCESS && fclose(fp) == EOF)
				retval=FAILURE;
		}
		else
			retval=FAILURE;
		free(buf);
	}
	else
	{
		usage();
		retval=FAILURE;
    }
	return retval;
}

/* BH create utility wrappers: main(), usage() 07-27-06 
------------------------------------------------------------------------
*/

/* help to stderr */
void usage(void)
{
	fprintf(stderr, "%s%s",
		"trunc: \ttruncate a file removing lines from top of file\n",
		"Usage: \ttrunc <file> <lines of text to keep>  lines = 0 -> n\n");
	exit(EXIT_FAILURE);     
}

/* requires two arguments - filename and lines. */
int main(int argc, char **argv)
{
	int retval=SUCCESS;

	errno=0;
	if(argc==3)
	{
		ssize_t lines=atoi(argv[2]);
		if(lines > -1)
		{
		    if(access(argv[1],W_OK)==SUCCESS)		
				retval=truncfile(argv[1], lines);
			else
				retval=FAILURE;	
		}
		else
			errno=EINVAL;
	}
	else
		usage();

	if(errno || retval==FAILURE)
	{
		perror("Error");
		retval=EXIT_FAILURE;
	}
	return retval;
}

# 6  
Old 03-06-2008
PS: I think driver wrote something like this and posted it on the forums, but I couldn't find it.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

File manipulation place 0 before the number using sed

I'm new with sed, and i am really confused with slashes, backslashes, parentheses, I've tried reading some beginner's guide, but still trouble fixing this problem, do you have any tips where or what to read to learn more about sed? can you also help me with my problem? Note: I was tasked to use... (4 Replies)
Discussion started by: akopocpoypoy
4 Replies

2. UNIX for Beginners Questions & Answers

How i can place in a log file an automatic chrome opener?

Okay, so i have this problem, i have a raspberry with rasbian lite (now i have openbox running on) and i do a few script for my client. But i have a question and a problem.... i got the logs file, that file need to open automatically the browser without having to do all the steps again at the next... (1 Reply)
Discussion started by: Oltrix
1 Replies

3. Shell Programming and Scripting

awk pattern matching in place of sed

I have written a script to parse data from some files on a Solaris 10 system and send the output to a CSV formatted file. The code snipped i am using to pull the data is as follows.... src_line=$(sed -n "/^search_pattern$/{=;}" $file) for i in $src_line do start_line1=$(( i + 9 )) nawk -v... (4 Replies)
Discussion started by: electricheadx
4 Replies

4. Shell Programming and Scripting

Print one's place for 1 to N times, ksh Perl whatever?

Hello all, I would like to create a for loop or whatever is quick that will print the one’s place of a number for 1-N times say for example a printed page formatting is 132 characters wide, I would like a single line 123456789012345678901234567890... ...012 That is 132 characters long. I... (11 Replies)
Discussion started by: KmJohnson
11 Replies

5. Shell Programming and Scripting

Place digit in front of the line searching pattern using sed command

hi All, i want to add the single digit front of the line in the report file and string compare with pattern file. patter file: pattern1.txt pattern num like 4 love 3 john 2 report file: report.txt i like very much but john is good boy i will love u so after execute... (9 Replies)
Discussion started by: krbala1985
9 Replies

6. Shell Programming and Scripting

Truncate Log files

Hi Gurus, I have several log files running in real time and needs to be truncated 50% or all but has to keep the logs piling up. Any ideas? For example: /var/adm/messages and others apps log files Thanks in advance! (7 Replies)
Discussion started by: anonymous1
7 Replies

7. Shell Programming and Scripting

sed edit in place -i issues

Hello, I am attempting to create a command that I can eventually put into a loop so I can edit 1file on many servers. I would like to edit the file in place with sed -i. If not I will take any suggestions on how to use a temp file. I need to remove a email address from the configuration file... (4 Replies)
Discussion started by: abacus
4 Replies

8. Shell Programming and Scripting

SED: Place char at starting and replace selected line

Hello Experts, I am working on a small file editing script. Since all experts here are very generous to give me the complete code, I would take up the problem in steps so that I ensure my opportunity to learn. AIM: The script has some commented and some uncommented lines. I need to : ... (2 Replies)
Discussion started by: hkansal
2 Replies

9. Shell Programming and Scripting

SED command help: Can we pass predefined variables in place of regex

Hi All, I have a doubt. Can we assign a regular expression for pattern searching to a variable in a script and then use that variable in place of a regular expression in sed command.I tried but got some syntax error!!Is it not possible.Because my requirement is that i have a generic script to get... (8 Replies)
Discussion started by: usha rao
8 Replies

10. Shell Programming and Scripting

display changing variable in one place on screen in ksh

Is it possible using just korn shell script to display a variable on the screen that is constantly changing in on place on the screen, to tell it in coordinates or something? In a loop, echo will print a new line each time, can I make it a static position? Thanks (7 Replies)
Discussion started by: raidzero
7 Replies
Login or Register to Ask a Question