Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: deleting double entries in a log file
Top Forums Shell Programming and Scripting deleting double entries in a log file Post 28260 by auswipe on Monday 16th of September 2002 10:42:32 AM
Old 09-16-2002
Re: deleting double entries in a log file
Quote:
Originally posted by opusforum
I have a apache log file that has double entries (however not all lines appear twice).
In this situation, I like to use a Perl hash for doing the dirty work for me.

Something like this:

Code:
#!/usr/bin/perl

open(LOG, "myLogFile") || die "$!";

my %logHash;

while ($inputLine = <LOG>) {
  if (!exists($logHash{$inputLine})) {
    $logHash{$inputLine} = 1;
    print "$inputLine";
  };
};

That should remove the dupe entries. Just redirect the output to a new log.
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Deleting log file

Hi:- Will there be any difference in Solair 2.6 if I delete /var/cron/log file to free up some space. Thanks in advance (5 Replies)
Discussion started by: s_aamir
5 Replies

2. Shell Programming and Scripting

Deleting double entry in a file

Hi, I am having almost the same problem as junior member 'oupsforum' (refer to subjuct "deleting double entry in a log file"), only that I am using Sun Sorlaris Unix which the uniq command does not has the flag -w. So I am not able to ignore certain portion of the line when the uniq doing the... (3 Replies)
Discussion started by: Wing m. Cheng
3 Replies

3. Shell Programming and Scripting

deleting lines in a log file

Is there an easy way to delete the first so many lines in a log file? like I have a log file that has 10000 lines, i want to just get rid of the first 9000. (2 Replies)
Discussion started by: BG_JrAdmin
2 Replies

4. Shell Programming and Scripting

Deleting all characters from 350th character to 450th character from the log file

Hi All, I have a big log file i want to delete all characters (between 350th to 450th characters) starting at 350th character position to 450th character position. please advice or sample code. (6 Replies)
Discussion started by: rajeshorpu
6 Replies

5. Shell Programming and Scripting

Deleting double items in file

Hi, i need a script, which deletes doulbe items in a file. My file looks like: - - - xxx xxx G123 G234 G234 G234 o o ... First i want to sort the file an then i want to delete double items. Can anyone help me. I work under solaris10. (3 Replies)
Discussion started by: free2k
3 Replies

6. Shell Programming and Scripting

Delete log file entries based on the Date/Timestamp within log file

If a log file is in the following format 28-Jul-10 ::: Log message 28-Jul-10 ::: Log message 29-Jul-10 ::: Log message 30-Jul-10 ::: Log message 31-Jul-10 ::: Log message 31-Jul-10 ::: Log message 1-Aug-10 ::: Log message 1-Aug-10 ::: Log message 2-Aug-10 ::: Log message 2-Aug-10 :::... (3 Replies)
Discussion started by: vikram3.r
3 Replies

7. UNIX for Dummies Questions & Answers

Script For Deleting Contents of "Live" Log File

In our shop, we have a situation where a log file from our interface engine software has begun maxing out in file size (reaching the 32-bit "2147483647" limit). Currently, the only way to rectify this is to stop the interface and restart it, which generates a new log. Easy enough, but the... (6 Replies)
Discussion started by: rjhjr64
6 Replies

8. Shell Programming and Scripting

Deleting string within a file that finishes with .log

Hello, This is my first post. Nice forum! I have a file trls.results small exemple of content (actually the file can be very big): ./security/htaccess.htm ./security/ipcount.log ./adhoc/sql/datamod06.sql So there is 3 paths to 3 different files... I want to remove every string that has a... (9 Replies)
Discussion started by: Jacob106106
9 Replies

9. Shell Programming and Scripting

deleting lines in a log file

Hello Community, sorry iam from germany and my englisch is not so well. iam searching for less then 4 hours on the web for a solution of my problem. i have a proxy-log-file and want to delete lines wicht matches on two words. example of the line in the logfile: now i want to delet... (3 Replies)
Discussion started by: matze
3 Replies

10. Shell Programming and Scripting

Deleting entries older than 7 days from a file

Hi, I have a file which contains entries in this format. my-bin.000140 my-bin.000141 my-bin.000142 my-bin.000143 my-bin.000144 my-bin.000145 my-bin.000146 my-bin.000147 my-bin.000148 my-bin.000149 my-bin.000150 my-bin.000151 my-bin.000152 my-bin.000153 my-bin.000154... (2 Replies)
Discussion started by: arijitsaha
2 Replies

LEARN ABOUT SUNOS

shells

shells(4)							   File Formats 							 shells(4)

NAME

       shells - shell database

SYNOPSIS

       /etc/shells

DESCRIPTION

       The shells file contains a list of the shells on the system. Applications use this file to determine whether a shell is valid. See getuser-
       shell(3C). For each shell a single line should be present, consisting of the shell's path, relative to root.

       A hash mark (#) indicates the beginning of a comment; subsequent characters up to the end of the line are not interpreted by  the  routines
       which search the file. Blank lines are also ignored.

       The  following  default	shells are used by utilities: /bin/bash, /bin/csh, /bin/jsh, /bin/ksh, /bin/pfcsh, /bin/pfksh, /bin/pfsh, /bin/sh,
       /bin/tcsh,  /bin/zsh,  /sbin/jsh,  /sbin/sh,  /usr/bin/bash,  /usr/bin/csh,  /usr/bin/jsh,  /usr/bin/ksh,  /usr/bin/pfcsh,  /usr/bin/pfksh,
       /usr/bin/pfsh, and /usr/bin/sh, /usr/bin/tcsh, /usr/bin/zsh. Note that /etc/shells overrides the default list.

       Invalid shells in /etc/shells may cause unexpected behavior (such as being unable to log in by way of ftp(1)).

FILES

       /etc/shells	       lists shells on system

SEE ALSO

       vipw(1B), ftpd(1M), sendmail(1M), getusershell(3C), aliases(4)

SunOS 5.10							    4 Jun 2001								 shells(4)
All times are GMT -4. The time now is 02:11 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy