Recursive copy to a RANDOM name - Page 2

xerexes · #8 (**permalink**) 10-17-2008

Sorry, forgot to answer about the desired transformation....

either:

Fl_0112.jpg -> Fl_0112_000001.jpg
GB_0112.jpg -> GB_00112_000002.jpg

or

Fl_0112.jpg -> 00000001.jpg
GB_0112.jpg -> 00000002.jpg

Ideally it would be the former rather than the latter i.e. Fl_0112.jpg -> Fl_0112_000001.jpg where a number gets appended to the original filename

xerexes · #9 (**permalink**) 10-17-2008

Hello Perderabo,

the total file count with the command is 88127.

The typical folder/file structure is:

with spaces
./cdell/Documents and Settings/Roki/My Documents/Exported Pics/images/Fl_0112.jpg

or with square brackets
./epd/Pics/Chris/Temp Folder/665109_7_t[1].jpg

To be honest, the numbers don't need to be random at all. The only reason I suggested random number is that that way I would be sure not to have a duplicate filename if concatenated with an actual filename.

The goal would be to have all the 88127 files inside one folder so that I can easily remove duplicates based on md5 hash.

I hope I've explained myself clearly.

Many thanks again.

Roki.

Perderabo · #10 (**permalink**) 10-18-2008

There are a few problems here. A directory with 80,000 files will behave very slowly with most file systems. Directory accesses are almost always sequential. And those file names are asking for trouble. If you really want to do this, make a practice directory hierarchy will only a hundred files or so and try it out there.

Code:

$ find pix
pix
pix/a
pix/a/a.gif
pix/a.jpg
pix/b
pix/b/a.gif
pix/b/xyz 001 [1]  .jpg
pix/b.jpg
pix/c
pix/c/a.gif
pix/c.jpg
pix/d.jpg
pix/e.jpg
pix/f.jpg
pix/g.jpg
pix/h.jpg
pix/i.jpg
pix/j.jpg
pix/k.jpg
pix/l.jpg
pix/m.jpg
pix/n.jpg
pix/o.jpg
pix/p.jpg
pix/q.jpg
pix/r.jpg
pix/s.jpg
pix/t.jpg
pix/u.jpg
pix/v.jpg
pix/w.jpg
pix/x.jpg
pix/xyz 001 [1]  .jpg
pix/xyz 001 [2]  .jpg
pix/xyz 001 ]2[  .jpg
pix/y.jpg
pix/z.jpg
$
$
$
$
$ cat fixpix
#! /usr/local/bin/bash

SOURCE=pix
DEST=newpix
count=0

find $SOURCE -type f | while read pathname ; do
        ((count=count+1))
        dir=${pathname%/*}
        file=${pathname#$dir/}
        base=${file%.*}
        typ=${file##*.}
        newfile=$(printf "%s%09d.%s" "$base" $count "$typ")
        echo mv \""$pathname"\" \""$DEST/$newfile"\"
done

$
$
$
$
$
$ ./fixpix
mv "pix/a/a.gif" "newpix/a000000001.gif"
mv "pix/a.jpg" "newpix/a000000002.jpg"
mv "pix/b/a.gif" "newpix/a000000003.gif"
mv "pix/b/xyz 001 [1]  .jpg" "newpix/xyz 001 [1]  000000004.jpg"
mv "pix/b.jpg" "newpix/b000000005.jpg"
mv "pix/c/a.gif" "newpix/a000000006.gif"
mv "pix/c.jpg" "newpix/c000000007.jpg"
mv "pix/d.jpg" "newpix/d000000008.jpg"
mv "pix/e.jpg" "newpix/e000000009.jpg"
mv "pix/f.jpg" "newpix/f000000010.jpg"
mv "pix/g.jpg" "newpix/g000000011.jpg"
mv "pix/h.jpg" "newpix/h000000012.jpg"
mv "pix/i.jpg" "newpix/i000000013.jpg"
mv "pix/j.jpg" "newpix/j000000014.jpg"
mv "pix/k.jpg" "newpix/k000000015.jpg"
mv "pix/l.jpg" "newpix/l000000016.jpg"
mv "pix/m.jpg" "newpix/m000000017.jpg"
mv "pix/n.jpg" "newpix/n000000018.jpg"
mv "pix/o.jpg" "newpix/o000000019.jpg"
mv "pix/p.jpg" "newpix/p000000020.jpg"
mv "pix/q.jpg" "newpix/q000000021.jpg"
mv "pix/r.jpg" "newpix/r000000022.jpg"
mv "pix/s.jpg" "newpix/s000000023.jpg"
mv "pix/t.jpg" "newpix/t000000024.jpg"
mv "pix/u.jpg" "newpix/u000000025.jpg"
mv "pix/v.jpg" "newpix/v000000026.jpg"
mv "pix/w.jpg" "newpix/w000000027.jpg"
mv "pix/x.jpg" "newpix/x000000028.jpg"
mv "pix/xyz 001 [1]  .jpg" "newpix/xyz 001 [1]  000000029.jpg"
mv "pix/xyz 001 [2]  .jpg" "newpix/xyz 001 [2]  000000030.jpg"
mv "pix/xyz 001 ]2[  .jpg" "newpix/xyz 001 ]2[  000000031.jpg"
mv "pix/y.jpg" "newpix/y000000032.jpg"
mv "pix/z.jpg" "newpix/z000000033.jpg"
$

xerexes · #11 (**permalink**) 10-23-2008

Hello Perderabo,

Many thanks for your help.

I've magaged to find an easier solution to my issue.
There is a program called Fdupes which removed duplicate files. It didn't seem to mind that the file paths/names had difficult characters.
The following fdupes tutuial was useful: fdupes Tutorial Life at the CLI
I've double checked the results with variations on the following command: find . -type f -exec md5sum {} \; | cut -d " " -f1 | sort | uniq ...... and both results seem to agree.

Just wanted to thank you for the assistance.

Best,

Roki

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
Recursive FTP -- here at last.	Perderabo	Shell Programming and Scripting	52	03-25-2009 12:15 PM
recursive copy of the directory	deepthi.s	High Level Programming	7	08-23-2008 01:06 PM
recursive rcp	Nicol	Shell Programming and Scripting	6	11-06-2003 11:52 AM
recursive copy of hidden files	usfrog	UNIX for Advanced & Expert Users	2	02-19-2002 03:16 PM
Recursive FTP	aslamg	UNIX for Dummies Questions & Answers	1	03-08-2001 04:27 AM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode	Rate This Thread: