Sponsored Content
Top Forums Shell Programming and Scripting Shellscript to sort duplicate files listed in a text file Post 302365724 by deaddevil on Wednesday 28th of October 2009 12:45:42 AM
Old 10-28-2009
Shellscript to sort duplicate files listed in a text file

I have many pdf's scattered across 4 machines. There is 1 location where I have other Pdf's maintained. But the issues it the 4 machines may have duplicate pdf's among themselves, but I want just 1 copy of each so that they can be transfered to that 1 location.

What I have thought is:
1) I have designed a script that will scan each of the 4 machines, and print the list of pdf files in a text file named list.txt.
2)So now I have all the pdf's listed in the list.txt file.
3) I need a shellscript that will now check this list and sort duplicate files. So that I know where are they located and even have them grouped together.
The list.txt contains the path along with the file name. so I guess we have to check just the ending file name part before ".pdf".
Please help me do this.


The list.txt looks like below, which is already generated.

Code:
/home/santosh/z_literature/MIF_Oxime_ph4_JBC_May2007.pdf
/home/santosh/z_literature/J_immun_biochemOFmif.pdf
/home/santosh/z_literature/sak/san/06_JCTC_06_bome.pdf
/home/santosh/z_literature/sak/san/03_IEJMD_05_nkr1.pdf
/home/santosh/z_literature/sak/san/07_JCAMD_06_CoRIA.pdf
/home/santosh/z_literature/sak/san/DDP-IV-JMM2007.pdf

 

10 More Discussions You Might Find Interesting

1. HP-UX

CVSWeb - Directories listed but files not listed

I am using CVSWeb on HPUnix. When i access it, all directories are listed but files are not listed. I am getting the error "NOTE: There are 51 files, but none matches the current tag. " in tomcat sevrer log i am getting the message "rlog warning: Missing revision or branch number after -r"... (0 Replies)
Discussion started by: ganesh
0 Replies

2. Shell Programming and Scripting

Copy files listed in a text file - whitespace problem.

Hi, Say I have this text file <copy.out> that contains a list of files/directories to be copied out to a different location. $ more copy.out dir1/file1 dir1/file2 dir1/file3 "dir1/white space" dir1/file4 If I do the following: $copy=`more copy.out` $echo $copy dir1/file1... (4 Replies)
Discussion started by: 60doses
4 Replies

3. Shell Programming and Scripting

Send a mail to IDs listed in a text file

I have a list of mail ids in text file and want a ksh script that reads this text file and sends a mail to all mail ids with same subject line and content. I am using UX-HP machine and KSH. Thanks for help in advance! (5 Replies)
Discussion started by: Sriranga
5 Replies

4. Shell Programming and Scripting

Sort and Remove Duplicate on file

How do we sort and remove duplicate on column 1,2 retaining the record with maximum date (in feild 3) for the file with following format. aaa|1234|2010-12-31 aaa|1234|2010-11-10 bbb|345|2011-01-01 ccc|346|2011-02-01 bbb|345|2011-03-10 aaa|1234|2010-01-01 Required Output ... (5 Replies)
Discussion started by: mabarif16
5 Replies

5. Shell Programming and Scripting

Copy files listed in text file to new directory

I am trying to write a script that will copy all file listed in a text file (100s of file names) to a new directory Assume script will run with main as current working directory and I know how many files/lines will be in List.txt Im trying to work up a test script using this model Contents of... (2 Replies)
Discussion started by: IAmTheGrass
2 Replies

6. Shell Programming and Scripting

How to grep a log file for words listed in separate text file?

Hello, I want to grep a log ("server.log") for words in a separate file ("white-list.txt") and generate a separate log file containing each line that uses a word from the "white-list.txt" file. Putting that in bullet points: Search through "server.log" for lines that contain any word... (15 Replies)
Discussion started by: nbsparks
15 Replies

7. Shell Programming and Scripting

Delete files listed in text file

Hi Team, Here's the scenario, I have a text file called "file_list.txt". Its content is as follows. 111.tmp 112.tmp 113.tmp 114.tmp These files will present in "workdir" directory. It has many files. But only the files present in file_list.txt has to be deleted from the workdir... (7 Replies)
Discussion started by: kmanivan82
7 Replies

8. Shell Programming and Scripting

Cat files listed in text file and redirect to new directory with same filename

I have a directory that is restricted and I cannot just copy the files need, but I can cat them and redirect them to a new directory. The files all have the date listed in them. If I perform a long listing and grep for the date (150620) I can redirect that output to a text file. Now I need to... (5 Replies)
Discussion started by: trigger467
5 Replies

9. Shell Programming and Scripting

Want to delete the junk files from a directory which are not listed in a TEXT file

Hello Everyone, I want to delete the image files from a directory, which are not listed in a TEXT file. The directory contains large number of image files (in millions) required / not required. I want to delete the image files which are "not required". I have generated a Text file having... (3 Replies)
Discussion started by: Praveen Pandit
3 Replies

10. UNIX for Beginners Questions & Answers

How do I custom sort the files in a directory using the filenames in a text file.?

Hi all, (5 Replies)
Discussion started by: KMusunuru
5 Replies
pstopdf(1)						    BSD General Commands Manual 						pstopdf(1)

NAME
pstopdf -- convert PostScript input into a PDF document. SYNOPSIS
pstopdf [inputfile] [-o outname] [-l] [-p] [-i] DESCRIPTION
pstopdf is a tool to convert PostScript input data into a PDF document. The input data may come from a file or may be read from stdin. The PDF document is always written to a file. The name of the output PDF file is derived from the name of the input file or may be explicitly named using the -o option. Flags: -o outname The name of the output file to create. If an explicit file name is not supplied, the output file will be created in the current directory and named foo.pdf for an input file named foo.ps -i Reads from stdin rather than a named input file. If the output file is not explicitly named and the input data comes from stdin the named output file will be stdin.pdf -l Specifies that any messages generated during file conversion be written to a log file. For an output file named foo.pdf the generated log file is foo.pdf.log rather than generated to stdout. If there are no messages, the log file is not generated. -p Generates a simple progress message to stdout at the end of each page. Because conversion of complex or lengthy PostScript input can take time, it is sometimes useful to see that progress is being made. Progress messages are always written to stdout even when the -l (log file) option is specified. EXAMPLES
pstopdf inputfile.ps Creates a PDF file named inputfile.pdf from the PostScript data in the input file inputfile.ps pstopdf -i -o outputfilename Creates a PDF file named outputfilename from the PostScript data read from stdin. Apple Computer, Inc. June 2, 2019 Apple Computer, Inc.
All times are GMT -4. The time now is 10:02 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy