Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Shellscript to sort duplicate files listed in a text file
Top Forums Shell Programming and Scripting Shellscript to sort duplicate files listed in a text file Post 302367219 by deaddevil on Monday 2nd of November 2009 01:07:02 AM
Old 11-02-2009
I now have a list of duplicate files, but the issue is I need to eliminate only the ones that are same not the ones that are different but still have the same name.

For eg
if the files are
david/project1/symbiosys.pdf
tom/project1/symbiosys.pdf

if both are workng on same project the pdf's may be similar, but I need to be sure, maybe by md5 checksum or something that can be found out,
but if the file size differs i need to save both of them, in 2 different folders to prevent them from overwriting.

Any suggestions or help in regards to shellscript needed.
 

10 More Discussions You Might Find Interesting

1. HP-UX

CVSWeb - Directories listed but files not listed

I am using CVSWeb on HPUnix. When i access it, all directories are listed but files are not listed. I am getting the error "NOTE: There are 51 files, but none matches the current tag. " in tomcat sevrer log i am getting the message "rlog warning: Missing revision or branch number after -r"... (0 Replies)
Discussion started by: ganesh
0 Replies

2. Shell Programming and Scripting

Copy files listed in a text file - whitespace problem.

Hi, Say I have this text file <copy.out> that contains a list of files/directories to be copied out to a different location. $ more copy.out dir1/file1 dir1/file2 dir1/file3 "dir1/white space" dir1/file4 If I do the following: $copy=`more copy.out` $echo $copy dir1/file1... (4 Replies)
Discussion started by: 60doses
4 Replies

3. Shell Programming and Scripting

Send a mail to IDs listed in a text file

I have a list of mail ids in text file and want a ksh script that reads this text file and sends a mail to all mail ids with same subject line and content. I am using UX-HP machine and KSH. Thanks for help in advance! (5 Replies)
Discussion started by: Sriranga
5 Replies

4. Shell Programming and Scripting

Sort and Remove Duplicate on file

How do we sort and remove duplicate on column 1,2 retaining the record with maximum date (in feild 3) for the file with following format. aaa|1234|2010-12-31 aaa|1234|2010-11-10 bbb|345|2011-01-01 ccc|346|2011-02-01 bbb|345|2011-03-10 aaa|1234|2010-01-01 Required Output ... (5 Replies)
Discussion started by: mabarif16
5 Replies

5. Shell Programming and Scripting

Copy files listed in text file to new directory

I am trying to write a script that will copy all file listed in a text file (100s of file names) to a new directory Assume script will run with main as current working directory and I know how many files/lines will be in List.txt Im trying to work up a test script using this model Contents of... (2 Replies)
Discussion started by: IAmTheGrass
2 Replies

6. Shell Programming and Scripting

How to grep a log file for words listed in separate text file?

Hello, I want to grep a log ("server.log") for words in a separate file ("white-list.txt") and generate a separate log file containing each line that uses a word from the "white-list.txt" file. Putting that in bullet points: Search through "server.log" for lines that contain any word... (15 Replies)
Discussion started by: nbsparks
15 Replies

7. Shell Programming and Scripting

Delete files listed in text file

Hi Team, Here's the scenario, I have a text file called "file_list.txt". Its content is as follows. 111.tmp 112.tmp 113.tmp 114.tmp These files will present in "workdir" directory. It has many files. But only the files present in file_list.txt has to be deleted from the workdir... (7 Replies)
Discussion started by: kmanivan82
7 Replies

8. Shell Programming and Scripting

Cat files listed in text file and redirect to new directory with same filename

I have a directory that is restricted and I cannot just copy the files need, but I can cat them and redirect them to a new directory. The files all have the date listed in them. If I perform a long listing and grep for the date (150620) I can redirect that output to a text file. Now I need to... (5 Replies)
Discussion started by: trigger467
5 Replies

9. Shell Programming and Scripting

Want to delete the junk files from a directory which are not listed in a TEXT file

Hello Everyone, I want to delete the image files from a directory, which are not listed in a TEXT file. The directory contains large number of image files (in millions) required / not required. I want to delete the image files which are "not required". I have generated a Text file having... (3 Replies)
Discussion started by: Praveen Pandit
3 Replies

10. UNIX for Beginners Questions & Answers

How do I custom sort the files in a directory using the filenames in a text file.?

Hi all, (5 Replies)
Discussion started by: KMusunuru
5 Replies

LEARN ABOUT DEBIAN

pdfconcat

PDFCONCAT(1)						      General Commands Manual						      PDFCONCAT(1)

NAME

       pdfconcat - program to concatenate several PDF files.

SYNOPSIS

       pdfconcat -o outfile.pdf input1.pdf [inputN.pdf ...]

DESCRIPTION

       This manual page documents briefly the pdfconcat command.

       pdfconcat  is  a small and fast command-line utility written in ANSI C that can concatenate (merge) several PDF files into a long PDF docu-
       ment. External libraries are not required, only ANSI C functions are used.  Several features of the output file are taken  from	the  first
       input  file  only. For example, outlines (also known as hierarchical bookmarks) in subsequent input files are ignored. pdfconcat compresses
       its input a little bit by removing whitespace and unused file parts.  This program has been tested on various huge PDFs downloaded from the
       Adobe  web  site,  plus	an  1200-pages	long mathematics manual typeset by LaTeX, emitted by pdflatex, dvipdfm and `gs -sDEVICE=pdfwrite',
       totalling 5981 pages in a single PDF file.

OPTIONS

       -o output.pdf
	      Place output in file output.pdf

SEE ALSO

       imgmerge(1), xpaint(1).

AUTHOR

       pdfconcat was written by Peter Szabo <pts@fazekas.hu>.

       This manual page was written by Josue Abarca <jmaslibre@debian.org.gt>, for the Debian project (and may be used by others).  Permission	is
       granted	to  copy, distribute and/or modify this document under the terms of the GNU General Public License, Version 2 or any later version
       published by the Free Software Foundation please see /usr/share/common-licenses/GPL-2 for the full text of the licence.

								 January 22, 2010						      PDFCONCAT(1)
All times are GMT -4. The time now is 08:06 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy