Sort file in perl Post: 23310

Sponsored Content

Top Forums Shell Programming and Scripting Sort file in perl Post 23310 by auswipe on Thursday 20th of June 2002 02:17:44 PM

06-20-2002

Registered User

D'oh!

I was re-reading your message and I see that you need to remove duplicates based upon the FIRST field of the colon seperated values.

That makes it a bit tricker but I'll see what I can do to help. The previous perl script is still good for complete lines of duplicate text.

EDIT: This code might help, however there might be some problems. I sort the removed duplicates on the first pass and then remove dupes based upon the first colon sperated value. This might be a problem for you in your application.

Give it a try and lemme know if it gets the job done.

Code:

#!/usr/bin/perl

# RemoveDupes.pl
# Auswipe 21 Feb 2002
# Auswipe sez: "Hey, no guarantees!"
# Usage:
#
#	RemoveDupes.pl -file someTextFile

use Getopt::Long;
GetOptions("file=s");

my %dataHash    = ();
my $currentLine = 0;

if ($opt_file) {
  open(INPUTFILE, "$opt_file") || die "Error: $!";

  while ($logEntry = <INPUTFILE> ) {
    chomp($logEntry);

    if (!exists($dataHash{$logEntry})) {
      $dataHash{$logEntry} = $currentLine;
    };

    $currentLine++;
  };
  
  close($opt_file);

} else {
  print STDOUT "You didn't select a file!\n";
};

my %secondHash = ();

foreach $logOutput (sort { $dataHash{$a} <=> $dataHash{$b} } (keys(%dataHash))) {
  my @columns = split(/:/, $logOutput);
  my $firstColumn = $columns[0];

  if (!exists($secondHash{$firstColumn})) {
    $secondHash{$firstColumn} = $logOutput;
  };
};

foreach $firstColumn (sort {$secondHash{$a} <=> $secondHash{$b} } (keys(%secondHash))) {
  print STDOUT "$secondHash{$firstColumn}\n";
};

Last edited by auswipe; 06-20-2002 at 03:30 PM..

auswipe

View Public Profile for auswipe

Find all posts by auswipe

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sort a file by date using perl

Hello, do any body help me to sort a file by date using perl? thanks in advance Esham

2. Shell Programming and Scripting

Perl find::file can I sort the out put

Perl file::find can I sort the out put I am using file::find in my script but how I wish to process each file found in date order. Can I sort this module? eg part of current script is.... use File::Find; # Recursively find all files and directories in $mqueue_directory find(\&wanted,...

3. Shell Programming and Scripting

Perl Sort on Text File

Hi, I have a file of names and I want perl to do a sort on this file. How can I sort this list of names using perl? I'm thinking of a command like: @sorted = sort { lc($a) cmp lc($b) } @not_sorted # alphabetical sort The only thing I'm sort of unsure of is, how would I get the name in my...

4. Shell Programming and Scripting

Perl function to sort a file based on key fields

Hi, I am new to PERL.I want to sort all the lines in a file based on 1,2 and 4th filelds. Can U suggest me a command/function in perl for this operation..

5. Shell Programming and Scripting

sort file with non ascii chars and cjk with perl

Hello, I am not a programmer, please be patient. Actually, I have started to look into Perl because it seems to be able to solve all the problems (or most of them) I happen meet using my computer. These problems are generally all text-manipulation-related. Although I started to study, I cannot...

6. Shell Programming and Scripting

Perl script to sort an Excel file

Hello! I need to sort a file that is partly in English partly in Bulgarian. The original file is an Excel file but I converted it to a tab-delimited text file. The encoding of the tab delimited file is UTF-8. To sort the text, the script should test every line of the text file to see if...

7. UNIX for Advanced & Expert Users

Script to sort the files and append the extension .sort to the sorted version of the file

Hello all - I am to this forum and fairly new in learning unix and finding some difficulty in preparing a small shell script. I am trying to make script to sort all the files given by user as input (either the exact full name of the file or say the files matching the criteria like all files...

8. Shell Programming and Scripting

Perl SFTP, to get, sort and process every file.

Hi All, I'm niks, and i'm a newbie here and newbie also in perl sorry, i'm just wondering how can i get the file from the other hostname using sftp? then after i get it i'm going to sort the file and process it one by one. sorry because i'm a newbie. Thanks, -niks

9. Shell Programming and Scripting

Sort flat file by 3rd column in perl

Hello Guys I want to sort a flat file by the third column (numeric ) and store it in some other name I/P 9924873|20111114|00000000000013013|130|13|10/15/2010 12:36:22|W860944|N|00 9924873|20111114|00000000000013009|130|09|10/15/2010 12:36:22|W860944|N|00...

10. Shell Programming and Scripting

Sort help: How to sort collected 'file list' by date stamp :

Hi Experts, I have a filelist collected from another server , now want to sort the output using date/time stamp filed. - Filed 6, 7,8 are showing the date/time/stamp. Here is the input: #---------------------------------------------------------------------- -rw------- 1 root ...

LEARN ABOUT HPUX

aclsort

aclsort(3C)															       aclsort(3C)

NAME

       aclsort() - sort an Access Control List (JFS File Systems only)

SYNOPSIS

DESCRIPTION

       The routine sorts JFS Access Control List (ACL) entries into the correct order to be accepted by the acl(2) system call.

       points  to a buffer containing ACL entries; if non-zero, indicates that the permissions should be recalculated; and specifies the number of
       ACL entries in the buffer.

       sorts the contents of the ACL buffer as follows:

	      Entries will be in order and

	      Entries of type and will be sorted in increasing order by numeric ID.

       The call will succeed if all of the following are true:

	      There is exactly one entry each of type and

	      There is at most one entry each of type and

	      Entries of type or may not contain duplicate entries.  A duplicate entry is one of the same type containing  the	same  numeric  id,
	      irrespective of permission bits.

	      If the argument is zero and there are no entries of type and no entries of type the permissions of the and entries must be the same.

	      If  there  are  no entries of type and no entries of type and the entry is specified, then the entry must also be specified, and the
	      permissions of the and entries must be the same.

RETURN VALUE

       Upon successful completion, the return value is 0.  If there are duplicate entries, the return value is the position of the first duplicate
       entry.  If there is more than one entry of type or they are treated as duplicate entries, and the return value is the position of the first
       duplicate entry.  For all other errors, the return value is -1.

NOTICES

       The buffer is sorted by type and ID before checking for any failures.  Therefore the buffer is always sorted, even if there is a failure.

       The position of a duplicate entry returned on failure is not the byte offset of the duplicate entry from its base; rather it refers to  the
       entry number of the duplicate entry within the sorted buffer.

       Checks  will  be performed in order of entry type.  If there are multiple failures, the failure returned will be the first encountered, for
       example, if the ACL buffer contains a duplicate entry and does not contain an entry, the return value will be the first duplicate entry.

       ACLs do not have to be sorted with prior to passing them to acl(2).

DEPENDENCIES

       is supported only on JFS file systems on the standard HP-UX operating system.

AUTHOR

       was developed by AT&T.

SEE ALSO

       acl(2), aclv(5).

																       aclsort(3C)

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sort a file by date using perl

Discussion started by: esham

2. Shell Programming and Scripting

Perl find::file can I sort the out put

Discussion started by: Andrek

3. Shell Programming and Scripting

Perl Sort on Text File

Discussion started by: eltinator

4. Shell Programming and Scripting

Perl function to sort a file based on key fields

Discussion started by: karthikd214