Sponsored Content
Top Forums Shell Programming and Scripting deleting double entries in a log file Post 28322 by opusforum on Tuesday 17th of September 2002 03:02:45 AM
Old 09-17-2002
the answer

I made it Smilie

here is what worked for me:

perl -e 'print reverse <>' logfile|uniq -w 50|perl -e 'print reverse <>' >logfile.done

so first, the logfile is inverted (by lines) then the dupes are removed and finaly we do an invert again.

The inversion is needed in order to have the first of a duplicate line pair removed.

Thanks to your contributions folks. This pointed me into the right direction.

Klaus Smilie
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Deleting log file

Hi:- Will there be any difference in Solair 2.6 if I delete /var/cron/log file to free up some space. Thanks in advance (5 Replies)
Discussion started by: s_aamir
5 Replies

2. Shell Programming and Scripting

Deleting double entry in a file

Hi, I am having almost the same problem as junior member 'oupsforum' (refer to subjuct "deleting double entry in a log file"), only that I am using Sun Sorlaris Unix which the uniq command does not has the flag -w. So I am not able to ignore certain portion of the line when the uniq doing the... (3 Replies)
Discussion started by: Wing m. Cheng
3 Replies

3. Shell Programming and Scripting

deleting lines in a log file

Is there an easy way to delete the first so many lines in a log file? like I have a log file that has 10000 lines, i want to just get rid of the first 9000. (2 Replies)
Discussion started by: BG_JrAdmin
2 Replies

4. Shell Programming and Scripting

Deleting all characters from 350th character to 450th character from the log file

Hi All, I have a big log file i want to delete all characters (between 350th to 450th characters) starting at 350th character position to 450th character position. please advice or sample code. (6 Replies)
Discussion started by: rajeshorpu
6 Replies

5. Shell Programming and Scripting

Deleting double items in file

Hi, i need a script, which deletes doulbe items in a file. My file looks like: - - - xxx xxx G123 G234 G234 G234 o o ... First i want to sort the file an then i want to delete double items. Can anyone help me. I work under solaris10. (3 Replies)
Discussion started by: free2k
3 Replies

6. Shell Programming and Scripting

Delete log file entries based on the Date/Timestamp within log file

If a log file is in the following format 28-Jul-10 ::: Log message 28-Jul-10 ::: Log message 29-Jul-10 ::: Log message 30-Jul-10 ::: Log message 31-Jul-10 ::: Log message 31-Jul-10 ::: Log message 1-Aug-10 ::: Log message 1-Aug-10 ::: Log message 2-Aug-10 ::: Log message 2-Aug-10 :::... (3 Replies)
Discussion started by: vikram3.r
3 Replies

7. UNIX for Dummies Questions & Answers

Script For Deleting Contents of "Live" Log File

In our shop, we have a situation where a log file from our interface engine software has begun maxing out in file size (reaching the 32-bit "2147483647" limit). Currently, the only way to rectify this is to stop the interface and restart it, which generates a new log. Easy enough, but the... (6 Replies)
Discussion started by: rjhjr64
6 Replies

8. Shell Programming and Scripting

Deleting string within a file that finishes with .log

Hello, This is my first post. Nice forum! I have a file trls.results small exemple of content (actually the file can be very big): ./security/htaccess.htm ./security/ipcount.log ./adhoc/sql/datamod06.sql So there is 3 paths to 3 different files... I want to remove every string that has a... (9 Replies)
Discussion started by: Jacob106106
9 Replies

9. Shell Programming and Scripting

deleting lines in a log file

Hello Community, sorry iam from germany and my englisch is not so well. iam searching for less then 4 hours on the web for a solution of my problem. i have a proxy-log-file and want to delete lines wicht matches on two words. example of the line in the logfile: now i want to delet... (3 Replies)
Discussion started by: matze
3 Replies

10. Shell Programming and Scripting

Deleting entries older than 7 days from a file

Hi, I have a file which contains entries in this format. my-bin.000140 my-bin.000141 my-bin.000142 my-bin.000143 my-bin.000144 my-bin.000145 my-bin.000146 my-bin.000147 my-bin.000148 my-bin.000149 my-bin.000150 my-bin.000151 my-bin.000152 my-bin.000153 my-bin.000154... (2 Replies)
Discussion started by: arijitsaha
2 Replies
LOGSAVE(8)						      System Manager's Manual							LOGSAVE(8)

NAME
logsave - save the output of a command in a logfile SYNOPSIS
logsave [ -asv ] logfile cmd_prog [ ... ] DESCRIPTION
The logsave program will execute cmd_prog with the specified argument(s), and save a copy of its output to logfile. If the containing directory for logfile does not exist, logsave will accumulate the output in memory until it can be written out. A copy of the output will also be written to standard output. If cmd_prog is a single hyphen ('-'), then instead of executing a program, logsave will take its input from standard input and save it in logfile logsave is useful for saving the output of initial boot scripts until the /var partition is mounted, so the output can be written to /var/log. OPTIONS
-a This option will cause the output to be appended to logfile, instead of replacing its current contents. -s This option will cause logsave to skip writing to the log file text which is bracketed with a control-A (ASCII 001 or Start of Header) and control-B (ASCII 002 or Start of Text). This allows progress bar information to be visible to the user on the console, while not being written to the log file. -v This option will make logsave to be more verbose in its output to the user. AUTHOR
Theodore Ts'o (tytso@mit.edu) SEE ALSO
fsck(8) E2fsprogs version 1.41.14 December 2010 LOGSAVE(8)
All times are GMT -4. The time now is 09:28 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy