cleaning log files


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers cleaning log files
# 1  
Old 10-09-2003
cleaning log files

Say I have a log file called [rptlog1] is there a way to clean out certain files in this log by [date ] or [catagorie] when I go to VI this file it tells me [ no more memory for the append to many lines in the field] and [ out of memory saving lines for undo]
I am guessing this means the file has become to large to append with VI so I was wondering if there was command with some argument to clean this out?
Thanks
Rockmiester
# 2  
Old 10-09-2003
I should tell you also I am running aix v4 and I guess I am talking
Truncating these files ? I should be doing this weekly and not sure how to do it.
Thanks
# 3  
Old 10-09-2003
Yes you have allowed the file to get too large. There are ways to view it like using "more" and page and tail/head... Also there is a command called split, that might work for you. I would try to copy it and then manipulate the copy to try and break it up.

To simply zero out the file type "> filename" this uses the cat redirect to input "null" to the file thus overwriting any data in the file.

However, most folks dont like to do that... I would suggest setup a cron to copy/move and compress log files so that they never get too large. The frequency depends on how fast they grow. I would start with a monthly cron at first.


cp log.orig log.bak.<date>
compress log.bak.<date>
> log.orig
# 4  
Old 10-09-2003
Quote:
Originally posted by Kelam_Magnus

cp log.orig log.bak.<date>
compress log.bak.<date>
> log.orig [/B]
This would not delete the file name would it ? just clean whats in it ? using >log.orig ?

Isn't there a way to use the [truncate] command also?
like [truncate log.orig] would this delete every thing but recent entries ?


Last edited by rocker40; 10-09-2003 at 01:26 PM..
# 5  
Old 10-09-2003
You could do any of the following or a combination -

This one grabs the last 5000 lines and copies to tmp file - then you copy it back to the original - it removes everything in the original and replaces it with only those 5000 lines.
# tail -5000 rptlog1 > rptlog.tmp
# cat rptlog.tmp > rptlog1

Same type of thing but you search for "Oct" - the assumption is that you grabbed enough lines (only you can tell for sure) AND that this file has the date posted in it for each entry. This would leave you with just the entries for October in the file.
# tail - 10000 rptlog1|egrep "Oct" > rptlog.tmp
# cat rptlog.tmp > rptlog1

You can use those commands plus the ones mentioned earlier by Kelam to do different searches, grab out different lines (starting from the start of the file with the head command)....
# 6  
Old 10-09-2003
[QUOTE]Originally posted by thehoghunter
[B]You could do any of the following or a combination -

This one grabs the last 5000 lines and copies to tmp file - then you copy it back to the original - it removes everything in the original and replaces it with only those 5000 lines.
# tail -5000 rptlog1 > rptlog.tmp
# cat rptlog.tmp > rptlog1

Same type of thing but you search for "Oct" - the assumption is that you grabbed enough lines (only you can tell for sure) AND that this file has the date posted in it for each entry. This would leave you with just the entries for October in the file.
# tail - 10000 rptlog1|egrep "Oct" > rptlog.tmp
# cat rptlog.tmp > rptlog1

These work thank you for taking the time to explain that.
# 7  
Old 10-15-2003
Can I make this a cron script?


# tail -5000 rptlog1 > rptlog.tmp
# cat rptlog.tmp > rptlog1
 
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

OCR text that needs cleaning

Hi, I have OCR'ed text that needs cleaning. Lines are delimited by parts of speech (POS), for example, each line will have either an adj. OR s. f. OR s. m. etc I need to uppercase all text before the POS but all text within parentheses to be lowercase Text after (and including) the POS... (6 Replies)
Discussion started by: safran
6 Replies

2. Shell Programming and Scripting

cleaning up files using find...

I am trying to cleanup a directory with around 4000 files, and using the below command to delete all .gz files older than 60 days, I am having the same issue of arguments being too long. is there a way i can use the same command to do what I intend to do. find /opt/et/logs/Archive/*.log.*.gz... (4 Replies)
Discussion started by: Shellslave
4 Replies

3. Shell Programming and Scripting

cleaning the file

Hi, I have a file with multiple rows. each row has 8 columns. Column 8 has entries separated by commas. I want to exclude all the rows in which column 8 has more than 3 commas. 1234#0/1 - ABC_1234 3 ATGCATGCATGC HHHIIIGIHVF 1 49:T>C,60:T>C,78:C>A,76:G>T,65:T>G Thanks, Diya (3 Replies)
Discussion started by: Diya123
3 Replies

4. Shell Programming and Scripting

File cleaning

HI , I am getting the source data as below. Source Data CDR_Data,,,,, F1,F2,F3,F4,F5,F6 5,5,6,7,8,7 6,6,g,,, 7,7,76,,, 8,8,gt,,, 9,9,df ,d,d,d ,,,,, (4 Replies)
Discussion started by: wangkc
4 Replies

5. UNIX for Dummies Questions & Answers

Cleaning core files

Hello *! Just a short question. Where on the system i can find core files. I have one SUN server (Solaris 8) and from time to time I must clean core files on it. But i am not sure where i can find those files. Thank you in advance. :) (5 Replies)
Discussion started by: ghost01
5 Replies

6. Shell Programming and Scripting

Cleaning the content of log files

Hi, I am facing problems while trying to clean a log file(Means making its file Size zero). I am not the owner of this file. From the command line, I can clean it by becoming a Sudo. (">logfilename"). I want to automate it using perl. But inside a script, sudo somehow doesnt seem to work. ... (1 Reply)
Discussion started by: jyotipg
1 Replies

7. UNIX for Dummies Questions & Answers

Database cleaning software

Hi everybody, I have been given a task to find the names of some products that can clean up databases by removing confidential information. The situation is that a client imports data from public sources (government websites, etc.) but that this data sometimes includes things like Social... (0 Replies)
Discussion started by: rhfrommn
0 Replies

8. UNIX for Dummies Questions & Answers

Cleaning text files

I wish to clean a text file of the following characters 1/2, 1/4, o (degrees) I cant display these characters. I have tried ALT+189 etc (my terminal emulator is set to ASCII). How do I display the above ? I am using HP UX 10. (5 Replies)
Discussion started by: ferretman
5 Replies

9. AIX

doing some spring cleaning....

USERS="me you jim joe sue" for user in ${USERS}; do rmuser -p $user usrdir=`cat /etc/passwd|grep $user|awk -F":" '{ print $6 }'` rm -fr `cat /etc/passwd|grep $user|awk -F":" '{ print $6 }'` echo Deleting: $user '\t' REMOVING: $usrdir done This is for AIX ONLY!!! but easily ported to... (0 Replies)
Discussion started by: Optimus_P
0 Replies
Login or Register to Ask a Question