|
|
Sponsored Content
Top Forums
Shell Programming and Scripting
deleting double entries in a log file
Post 28322 by opusforum on Tuesday 17th of September 2002 03:02:45 AM
09-17-2002
the answer
I made it 
here is what worked for me:
perl -e 'print reverse <>' logfile|uniq -w 50|perl -e 'print reverse <>' >logfile.done
so first, the logfile is inverted (by lines) then the dupes are removed and finaly we do an invert again.
The inversion is needed in order to have the first of a duplicate line pair removed.
Thanks to your contributions folks. This pointed me into the right direction.
Klaus
here is what worked for me:
perl -e 'print reverse <>' logfile|uniq -w 50|perl -e 'print reverse <>' >logfile.done
so first, the logfile is inverted (by lines) then the dupes are removed and finaly we do an invert again.
The inversion is needed in order to have the first of a duplicate line pair removed.
Thanks to your contributions folks. This pointed me into the right direction.
Klaus
|
|
10 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
Hi:-
Will there be any difference in Solair 2.6 if I delete /var/cron/log file to free up some space.
Thanks in advance (5 Replies)
Discussion started by: s_aamir
5 Replies
2. Shell Programming and Scripting
Hi,
I am having almost the same problem as junior member 'oupsforum' (refer to subjuct "deleting double entry in a log file"), only that I am using Sun Sorlaris Unix which the uniq command does not has the flag -w. So I am not able to ignore certain portion of the line when the uniq doing the... (3 Replies)
Discussion started by: Wing m. Cheng
3 Replies
3. Shell Programming and Scripting
Is there an easy way to delete the first so many lines in a log file?
like I have a log file that has 10000 lines, i want to just get rid of the first 9000. (2 Replies)
Discussion started by: BG_JrAdmin
2 Replies
4. Shell Programming and Scripting
Hi All,
I have a big log file i want to delete all characters (between 350th to 450th characters) starting at 350th character position to 450th character position.
please advice or sample code. (6 Replies)
Discussion started by: rajeshorpu
6 Replies
5. Shell Programming and Scripting
Hi,
i need a script, which deletes doulbe items in a file.
My file looks like:
-
-
-
xxx
xxx
G123
G234
G234
G234
o
o
...
First i want to sort the file an then i want to delete double items.
Can anyone help me. I work under solaris10. (3 Replies)
Discussion started by: free2k
3 Replies
6. Shell Programming and Scripting
If a log file is in the following format
28-Jul-10 ::: Log message
28-Jul-10 ::: Log message
29-Jul-10 ::: Log message
30-Jul-10 ::: Log message
31-Jul-10 ::: Log message
31-Jul-10 ::: Log message
1-Aug-10 ::: Log message
1-Aug-10 ::: Log message
2-Aug-10 ::: Log message
2-Aug-10 :::... (3 Replies)
Discussion started by: vikram3.r
3 Replies
7. UNIX for Dummies Questions & Answers
In our shop, we have a situation where a log file from our interface engine software has begun maxing out in file size (reaching the 32-bit "2147483647" limit). Currently, the only way to rectify this is to stop the interface and restart it, which generates a new log. Easy enough, but the... (6 Replies)
Discussion started by: rjhjr64
6 Replies
8. Shell Programming and Scripting
Hello,
This is my first post. Nice forum!
I have a file trls.results
small exemple of content (actually the file can be very big):
./security/htaccess.htm ./security/ipcount.log ./adhoc/sql/datamod06.sql
So there is 3 paths to 3 different files... I want to remove every string that has a... (9 Replies)
Discussion started by: Jacob106106
9 Replies
9. Shell Programming and Scripting
Hello Community,
sorry iam from germany and my englisch is not so well.
iam searching for less then 4 hours on the web for a solution of my problem.
i have a proxy-log-file and want to delete lines wicht matches on two words.
example of the line in the logfile:
now i want to delet... (3 Replies)
Discussion started by: matze
3 Replies
10. Shell Programming and Scripting
Hi,
I have a file which contains entries in this format.
my-bin.000140
my-bin.000141
my-bin.000142
my-bin.000143
my-bin.000144
my-bin.000145
my-bin.000146
my-bin.000147
my-bin.000148
my-bin.000149
my-bin.000150
my-bin.000151
my-bin.000152
my-bin.000153
my-bin.000154... (2 Replies)
Discussion started by: arijitsaha
2 Replies
LEARN ABOUT OPENDARWIN
log1p
LOG(3) BSD Library Functions Manual LOG(3) NAME
log, log2, log10, log1p, -- logarithm functions SYNOPSIS
#include <math.h> double log(double x); double log2(double x); double log10(double x); double log1p(double x); DESCRIPTION
The log() function computes the value of the natural logarithm of argument x. The log2() function computes the value of the logarithm of argument x to base 2. The log10() function computes the value of the logarithm of argument x to base 10. The log1p() function computes the value of log(1+x) accurately even for very small values of x. SPECIAL VALUES
log(+-0) , log2(+-0) , and log10(+-0) return -infinity and raise the "divide-by-zero" floating-point exception. log(1) , log2(1) , and log10(1) return +0. log(x) , log2(x) , and log10(x) return a NaN and raise the "invalid" floating-point exception for x < 0. log(+infinity) , log2(+infinity) , and log10(+infinity) return +infinity. log1p(+-0) returns +-0. log1p(-1) returns -infinity and raises the "divide-by-zero" floating-point exception. log1p(x) returns a NaN and raises the "invalid" floating-point exception for x < -1. log1p(+infinity) returns +infinity. SEE ALSO
math(3), exp(3), exp2(3), expm1(3), pow(3), infnan(3) STANDARDS
The log() , log2() , log10() , and log1p() functions conform to ISO/IEC 9899:1999(E). 4th Berkeley Distribution July 2, 2003 4th Berkeley Distribution