|
|
Sponsored Content
Top Forums
Shell Programming and Scripting
deleting double entries in a log file
Post 28322 by opusforum on Tuesday 17th of September 2002 03:02:45 AM
09-17-2002
the answer
I made it 
here is what worked for me:
perl -e 'print reverse <>' logfile|uniq -w 50|perl -e 'print reverse <>' >logfile.done
so first, the logfile is inverted (by lines) then the dupes are removed and finaly we do an invert again.
The inversion is needed in order to have the first of a duplicate line pair removed.
Thanks to your contributions folks. This pointed me into the right direction.
Klaus
here is what worked for me:
perl -e 'print reverse <>' logfile|uniq -w 50|perl -e 'print reverse <>' >logfile.done
so first, the logfile is inverted (by lines) then the dupes are removed and finaly we do an invert again.
The inversion is needed in order to have the first of a duplicate line pair removed.
Thanks to your contributions folks. This pointed me into the right direction.
Klaus
|
|
10 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
Hi:-
Will there be any difference in Solair 2.6 if I delete /var/cron/log file to free up some space.
Thanks in advance (5 Replies)
Discussion started by: s_aamir
5 Replies
2. Shell Programming and Scripting
Hi,
I am having almost the same problem as junior member 'oupsforum' (refer to subjuct "deleting double entry in a log file"), only that I am using Sun Sorlaris Unix which the uniq command does not has the flag -w. So I am not able to ignore certain portion of the line when the uniq doing the... (3 Replies)
Discussion started by: Wing m. Cheng
3 Replies
3. Shell Programming and Scripting
Is there an easy way to delete the first so many lines in a log file?
like I have a log file that has 10000 lines, i want to just get rid of the first 9000. (2 Replies)
Discussion started by: BG_JrAdmin
2 Replies
4. Shell Programming and Scripting
Hi All,
I have a big log file i want to delete all characters (between 350th to 450th characters) starting at 350th character position to 450th character position.
please advice or sample code. (6 Replies)
Discussion started by: rajeshorpu
6 Replies
5. Shell Programming and Scripting
Hi,
i need a script, which deletes doulbe items in a file.
My file looks like:
-
-
-
xxx
xxx
G123
G234
G234
G234
o
o
...
First i want to sort the file an then i want to delete double items.
Can anyone help me. I work under solaris10. (3 Replies)
Discussion started by: free2k
3 Replies
6. Shell Programming and Scripting
If a log file is in the following format
28-Jul-10 ::: Log message
28-Jul-10 ::: Log message
29-Jul-10 ::: Log message
30-Jul-10 ::: Log message
31-Jul-10 ::: Log message
31-Jul-10 ::: Log message
1-Aug-10 ::: Log message
1-Aug-10 ::: Log message
2-Aug-10 ::: Log message
2-Aug-10 :::... (3 Replies)
Discussion started by: vikram3.r
3 Replies
7. UNIX for Dummies Questions & Answers
In our shop, we have a situation where a log file from our interface engine software has begun maxing out in file size (reaching the 32-bit "2147483647" limit). Currently, the only way to rectify this is to stop the interface and restart it, which generates a new log. Easy enough, but the... (6 Replies)
Discussion started by: rjhjr64
6 Replies
8. Shell Programming and Scripting
Hello,
This is my first post. Nice forum!
I have a file trls.results
small exemple of content (actually the file can be very big):
./security/htaccess.htm ./security/ipcount.log ./adhoc/sql/datamod06.sql
So there is 3 paths to 3 different files... I want to remove every string that has a... (9 Replies)
Discussion started by: Jacob106106
9 Replies
9. Shell Programming and Scripting
Hello Community,
sorry iam from germany and my englisch is not so well.
iam searching for less then 4 hours on the web for a solution of my problem.
i have a proxy-log-file and want to delete lines wicht matches on two words.
example of the line in the logfile:
now i want to delet... (3 Replies)
Discussion started by: matze
3 Replies
10. Shell Programming and Scripting
Hi,
I have a file which contains entries in this format.
my-bin.000140
my-bin.000141
my-bin.000142
my-bin.000143
my-bin.000144
my-bin.000145
my-bin.000146
my-bin.000147
my-bin.000148
my-bin.000149
my-bin.000150
my-bin.000151
my-bin.000152
my-bin.000153
my-bin.000154... (2 Replies)
Discussion started by: arijitsaha
2 Replies
LEARN ABOUT DEBIAN
ddrescuelog
DDRESCUELOG(1) User Commands DDRESCUELOG(1) NAME
ddrescuelog - tool for ddrescue logfiles SYNOPSIS
ddrescuelog [options] logfile DESCRIPTION
GNU ddrescuelog - Tool for ddrescue logfiles. Manipulates ddrescue logfiles, shows their contents, converts them to/from other formats, compares them, and tests rescue status. OPTIONS
-h, --help display this help and exit -V, --version output version information and exit -a, --change-types=<ot>,<nt> change the block types of a logfile -b, --block-size=<bytes> block size in bytes [default 512] -c, --create-logfile[=<tt>] create logfile from list of blocks [+-] -d, --delete-if-done delete the logfile if rescue is finished -D, --done-status return 0 if rescue is finished -f, --force overwrite existing output files -i, --input-position=<bytes> starting position of rescue domain [0] -l, --list-blocks=<types> print block numbers of given types (?*/-+) -m, --domain-logfile=<file> restrict domain to finished blocks in file -n, --invert-logfile invert block types (finished <-> others) -o, --output-position=<bytes> starting position in output file [ipos] -p, --compare-logfile=<file> compare block types in domain of both files -q, --quiet suppress all messages -s, --max-size=<bytes> maximum size of rescue domain to be processed -t, --show-status show a summary of logfile contents -v, --verbose be verbose (a 2nd -v gives more) -x, --xor-logfile=<file> XOR the finished blocks in file with logfile -y, --and-logfile=<file> AND the finished blocks in file with logfile -z, --or-logfile=<file> OR the finished blocks in file with logfile Numbers may be followed by a multiplier: b = blocks, k = kB = 10^3 = 1000, Ki = KiB = 2^10 = 1024, M = 10^6, Mi = 2^20, G = 10^9, Gi = 2^30, etc... REPORTING BUGS
Report bugs to bug-ddrescue@gnu.org Ddrescue home page: http://www.gnu.org/software/ddrescue/ddrescue.html General help using GNU software: http://www.gnu.org/gethelp COPYRIGHT
Copyright (C) 2012 Antonio Diaz Diaz. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. ddrescuelog 1.16 June 2012 DDRESCUELOG(1)