In directory /mnt/upload I have about 100 000 files (*.png) that have been created during the last six months. Now I need to move them to right folders. eg:
So the script first creates a directory with the correct name ( format +%Y%m%d) move all files in a day (but only files not directory) in to right directory.
any ideas ?
Last edited by jim mcnamara; 11-14-2014 at 04:52 PM..
Yes, not a too efficient one though... Use it at your own risk. All commands are supposed to be run in the /mnt/upload directory.
A proper backup of those png files (with preserved modification timestamps) is highly recommended before attempting to perform the mass mv operation.
A rather more direct way -- have find print a list of files like
...then read them into the shell to run with mv.
I could make it more efficient, but the real inefficiency is going to be that one dir with 100,000 files, which there is absolutely no way to speed up... Each relink() has to lock and update that one huge folder.
Last edited by Corona688; 11-14-2014 at 05:22 PM..
The big time cost is finding and removing them from the source dir. I suspect it might be fastest to move each file found in order, so the directory entry is the first one checked, or nearly so, each time a file is moved.
You might hard link them rather than move them, a sort of half move, no source directory rewriting but still a long search for file entries in there. Once they are all linked, you can delete all the old links in the source dir.
I'd use a find -ls and figure the destination from that in sed, not the time arguments.
Wow, needs some indentation and formatting for ease of maintenance! You, too, deserve pretty code. It's an investment in your future.
Moving all the files for one destination might have been an economy, using the mkdir -p line extended.
Note the need for a dummy last line to flush the last dir, and presetting the d_last to deal with the first line. Loops are neat except dealing with the ends.
Bash/ksh/awk/PERL have associative arrays that might be used in place of the sort with less loop ends stuff. All the file names would be stored under the date, and then at EOF you can go through the array key dates making dirs and moving files.
I have a script, which is checking if file exists and move it to another directory
if
then
mkdir -p ${LOCL_FILES_DIR}/cool_${Today}/monthly
mv report_manual_alloc_rpt_A_I_ASSIGNMENT.${Today}*.csv ${LOCL_FILES_DIR}/cool_${Today}/monthly
... (9 Replies)
I written unix script where a pdf file generates. But if the script is used by multiple people at a time it generates same pdf with two different owner names and creating a problem with permission while moving the file. is there a way where i can move the file filtering with the user? (4 Replies)
Hi folks,
In my application there is a job running which create a .dat file along with it zip file also at unix box location /opt/app/cvf/temp1
so in temp1 directory I have one .dat file and its zip file also.
Now since this job runs every day so if a job runs today there will be two files... (5 Replies)
Hi Team,
I have thousands of TIF files which are converted from PDF. Below is a sample of it.
LH9406_BLANCARAMOS_2012041812103210320001.tif
LH9406_BLANCARAMOS_2012041812103210320002.tif
LH9406_BLANCARAMOS_2012041812103210320003.tif
LH9411_ANGENIAHUTCHINSON_2012041812102510250001.tif... (9 Replies)
hi all,
here is the description to my problem.
input parameter: $date1
based on the date i need to select three files starting with audit.log* based on its modified date, a date before, a date after(if its exists). We need to compare the input date to modified date of the file. And then... (3 Replies)
I Need help for one requirement,
I want to move the latest/Older file in the folder to another file. File have the datetimestamp in postfix.
Example:
Source Directory : \a
destination Directory : \a\b
File1 : xy_MMDDYYYYHHMM.txt (xy_032120101456.txt)
File2: xy_MMDDYYYYHHMM.txt... (1 Reply)
Hello all.
I am new to this forum (and somewhat new to UNIX / LINUX - I started using ubuntu 1 year ago).:b:
I have the following problem that I have not been able to figure out how to take care of and I was wondering if anyone could help me out.:confused:
I have all of my music stored in... (7 Replies)
Good afternoon!
I need to move a bunch of files from an FTP site to my server.
However mget *, dele * will not suffice as I cannot guarantee that new files are not uploaded between commands.
Roughly, this is what I would like to do:
(using a .netrc)
ftp somehost
bin
prompt
cd... (1 Reply)