Sponsored Content
Full Discussion: Duplicate filename algorithm
Top Forums Shell Programming and Scripting Duplicate filename algorithm Post 302406220 by radoulov on Monday 22nd of March 2010 10:12:45 AM
Old 03-22-2010
Yes,
as already stated, the previous Perl solutions compare the content of the files.
Could you try this Perl code and compare its performance with your shell script?


Code:
perl -MFile::Find -e'
  $d = shift || die "$0 dir\n";
  find { 
    wanted => sub {
      -f and push @{$u{$_}}, $File::Find::name;
      }
    }, $d;
  @{$u{$_}} > 1 and printf "found %s in: \n\n%s\n\n", 
    $_, join $/, @{$u{$_}} for keys %u;    
  ' <dirname>

 

10 More Discussions You Might Find Interesting

1. Programming

Algorithm problem

Looking for an algorithm to compute the number of days between two given dates I came across a professor's C program located here: http://cr.yp.to/2001-275/struct1.c I was wondering if anyone could tell me where the value 678882 in the line int d = dateday - 678882; comes from and also the... (1 Reply)
Discussion started by: williamf
1 Replies

2. Shell Programming and Scripting

algorithm

PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP 21444 tomusr 213M 61M sleep 29 10 1:20:46 0.1% java/43 21249 root 93M 44M sleep 29 10 1:07:19 0.2% java/56 is there anyway i can use a command to get the total of the SIZE? 306M (Derive from... (5 Replies)
Discussion started by: filthymonk
5 Replies

3. UNIX for Dummies Questions & Answers

Report of duplicate files based on part of the filename

I have the files logged in the file system with names in the format of : filename_ordernumber_date_time eg: file_1_12012007_1101.txt file_2_12022007_1101.txt file_1_12032007_1101.txt I need to find out all the files that are logged multiple times with same order number. In the above eg, I... (1 Reply)
Discussion started by: sudheshnaiyer
1 Replies

4. Shell Programming and Scripting

gzcat into awk and then change FILENAME and process new FILENAME

I am trying to write a script that prompts users for date and time, then process the gzip file into awk. During the ksh part of the script another file is created and needs to be processed with a different set of pattern matches then I need to combine the two in the end. I'm stuck at the part... (6 Replies)
Discussion started by: timj123
6 Replies

5. Shell Programming and Scripting

Filename from splitting files to have the same filename of the original file with counter value

Hi all, I have a list of xml file. I need to split the files to a different files when see the <ko> tag. The list of filename are B20090908.1100-20090908.1200_CDMA=1,NO=2,SITE=3.xml B20090908.1200-20090908.1300_CDMA=1,NO=2,SITE=3.xml B20090908.1300-20090908.1400_CDMA=1,NO=2,SITE=3.xml ... (3 Replies)
Discussion started by: natalie23
3 Replies

6. Programming

Please help me to develop algorithm

Hi guys , in my study book from which I re-learn C is task to generate all possible characters combination from numbers entered by the user. I know this algorithm must use combinatorics to calculate all permutations. Problem is how to implement algortihm. // This program reads the four numbers... (0 Replies)
Discussion started by: solaris_user
0 Replies

7. Shell Programming and Scripting

Find duplicate based on 'n' fields and mark the duplicate as 'D'

Hi, In a file, I have to mark duplicate records as 'D' and the latest record alone as 'C'. In the below file, I have to identify if duplicate records are there or not based on Man_ID, Man_DT, Ship_ID and I have to mark the record with latest Ship_DT as "C" and other as "D" (I have to create... (7 Replies)
Discussion started by: machomaddy
7 Replies

8. UNIX for Dummies Questions & Answers

banker's algorithm.. help

i'm doing banker's algorithm.. got some error there but i cant fix it.. please help!! #!/bin/bash echo "enter no.of resources: " read n1 echo -n "enter the max no .of resources for each type: " for(( i=0; i <$n1; i++ )) do read ${t} done echo -n "enter no .of... (1 Reply)
Discussion started by: syah
1 Replies

9. UNIX for Dummies Questions & Answers

to extract all the part of the filename before a particular word in the filename

Hi All, Thanks in Advance I am working on a shell script. I need some assistance. My Requirement: 1) There are some set of files in a directory like given below OTP_UFSC_20120530000000_acc.csv OTP_UFSC_20120530000000_faf.csv OTP_UFSC_20120530000000_prom.csv... (0 Replies)
Discussion started by: aealexanderraj
0 Replies

10. Programming

to extract all the part of the filename before a particular word in the filename

Hi All, Thanks in Advance I am working on a shell script. I need some assistance. My code: if then set "subscriber" "promplan" "mapping" "dedicatedaccount" "faflistSub" "faflistAcc" "accumulator"\ "pam_account"; for i in 1 2 3 4 5 6 7 8;... (0 Replies)
Discussion started by: aealexanderraj
0 Replies
File::Find::Rule::Procedural(3) 			User Contributed Perl Documentation			   File::Find::Rule::Procedural(3)

NAME
File::Find::Rule::Procedural - File::Find::Rule's procedural interface SYNOPSIS
use File::Find::Rule; # find all .pm files, procedurally my @files = find(file => name => '*.pm', in => @INC); DESCRIPTION
In addition to the regular object-oriented interface, File::Find::Rule provides two subroutines for you to use. "find( @clauses )" "rule( @clauses )" "find" and "rule" can be used to invoke any methods available to the OO version. "rule" is a synonym for "find" Passing more than one value to a clause is done with an anonymous array: my $finder = find( name => [ '*.mp3', '*.ogg' ] ); "find" and "rule" both return a File::Find::Rule instance, unless one of the arguments is "in", in which case it returns a list of things that match the rule. my @files = find( name => [ '*.mp3', '*.ogg' ], in => $ENV{HOME} ); Please note that "in" will be the last clause evaluated, and so this code will search for mp3s regardless of size. my @files = find( name => '*.mp3', in => $ENV{HOME}, size => '<2k' ); ^ | Clause processing stopped here ------/ It is also possible to invert a single rule by prefixing it with "!" like so: # large files that aren't videos my @files = find( file => '!name' => [ '*.avi', '*.mov' ], size => '>20M', in => $ENV{HOME} ); AUTHOR
Richard Clamp <richardc@unixbeard.net> COPYRIGHT
Copyright (C) 2003 Richard Clamp. All Rights Reserved. This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself. SEE ALSO
File::Find::Rule perl v5.16.3 2011-09-19 File::Find::Rule::Procedural(3)
All times are GMT -4. The time now is 09:15 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy