Sort/move files into folders and rename existing files.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Sort/move files into folders and rename existing files.
# 1  
Old 03-18-2020
Sort/move files into folders and rename existing files.

Hello everyone!
Im having a few problems with my file moving script.
Im trying to move files into folders based on the filenames and if a file already exists rename that file.

The files consists of three sets of different filenames. They will be moved/sorted into the folder structure below. The Source and Destination path will always be static.
Code:
OBJECTS/2018/1803/180313/DNG
OBJECTS/2018/1803/180313/TIFF
OBJECTS/2018/1803/180313/JPEG

Example filenames:

Code:
15512-1.dng
15512-1.tif
15512-1.jpg

180313-1.dng
180313-1.tif
180313-1.jpg

2002005-1.dng
2002005-1.tif
2002005-1.jpg


The code below kind of works but there must be a better way to loop through the files than to use two “While do”ť statements.
Another thing I'm having trouble with is if a file already exists, I would like to rename or append something to that existing file. I tried with --backup but all I get is illegal option, most likely due to me testing this on MacOS.

My intension is to run this on my Synology server as an automated task every evening.
When i try to execute the script on the server it complains about the Source path, maybe not correctly formatted?
find: `"/volume1/Archive/IMAGES/UPLOAD/TIFF/"': No such file or directory

Code:
#!/bin/bash
SRC1=/volume1/Archive/IMAGES/UPLOAD/JPEG/
SRC2=/volume1/Archive/IMAGES/UPLOAD/TIFF/
DST=/volume1/Archive/IMAGES/OBJECTS/
{
export LC_CTYPE=UTF-8
while IFS= read -r -d $'\\0' f1
do
    n=${f1##*/}
    [[ $n =~ ^(([0-9]{2})([0-9]{1,2})[0-9]{2,3})[^0-9] ]] || continue	 #5,6 & 7-digits (yymkk) (yymmkk) (yymmkkk)
    p=${BASH_REMATCH[1]}
    y=${BASH_REMATCH[2]}
    m=${BASH_REMATCH[3]}
    m='0'$m
    m=${m: -2}
	
	jpg=${DST}/'20'${y}/${y}${m}/${p}/JPEG
	[[ -d $jpg ]] || mkdir -p \"$jpg\" || continue
    

	cp -v -n \"$f1\" \"$jpg\"
	
	
done < <(find \"$SRC1\" -type f -iname '*.jpg' -print0)

# TIFF files
while IFS= read -r -d $'\\0' f2
do
    n=${f2##*/}
    [[ $n =~ ^(([0-9]{2})([0-9]{1,2})[0-9]{2,3})[^0-9] ]] || continue	 #5,6 & 7-digits (yymkk) (yymmkk) (yymmkkk)
    p=${BASH_REMATCH[1]}
    y=${BASH_REMATCH[2]}
    m=${BASH_REMATCH[3]}
    m='0'$m
    m=${m: -2}
	
	tif=${DST}/'20'${y}/${y}${m}/${p}/TIFF
	[[ -d $tif ]] || mkdir -p \"$tif\" || continue
    
	cp -v -n \"$f2\" \"$tif\"
	
	
done < <(find \"$SRC2\" -type f -iname '*.tif' -print0)


} 2>&1
exit 0


Last edited by oatfoolery; 03-18-2020 at 08:54 AM..
# 2  
Old 03-18-2020
Hint:
Code:
find /path/to/search \( -iname '*.jpg' -o -iname '*.tif' \)

This User Gave Thanks to balajesuri For This Post:
# 3  
Old 03-18-2020
Hmmm - I don't see /volume1/Archive/IMAGES/UPLOAD/DNG/ referenced anywhere in your script - that error msg can't come from that script. Does the DNG directory exist in the path?
This User Gave Thanks to RudiC For This Post:
# 4  
Old 03-18-2020
Quote:
Originally Posted by RudiC
Hmmm - I don't see /volume1/Archive/IMAGES/UPLOAD/DNG/ referenced anywhere in your script - that error msg can't come from that script. Does the DNG directory exist in the path?
Sorry, that was a typo. I used the DNG directory befor, but now only the TIFF and JPEG, still get the same error and they do exist.

--- Post updated at 14:24 ---

Quote:
Originally Posted by balajesuri
Hint:
Code:
find /path/to/search \( -iname '*.jpg' -o -iname '*.tif' \)

Thanks, i'm pretty new to this, not quite sure i get the hint.
so this would find all files with .jpg and .tif but only from one path. How can i search both SRC1 and SRC2.
(find \"$SRC1\" -type f -iname '*.tif' -o -iname '*.jpg' -print0)
# 5  
Old 03-18-2020
Drop the escaped double quotes in find's "starting point". find does allow multiple "starting points", BTW.
This User Gave Thanks to RudiC For This Post:
# 6  
Old 03-19-2020
Quote:
Originally Posted by RudiC
Drop the escaped double quotes in find's "starting point". find does allow multiple "starting points", BTW.
Great, thanks! now i don't get any errors, but also the script is not working. Nothing is happening, no error codes.
Obviously i must be missing something.

Code:
#!/bin/bash
SRC="/volume1/Archive/IMAGES/UPLOAD/TEST1"
DST="/volume1/Archive/IMAGES/UPLOAD/TEST2"
{
export LC_CTYPE=UTF-8

while IFS= read -r -d $'\\0' f1
do
    n=${f1##*/}
    [[ $n =~ ^(([0-9]{2})([0-9]{1,2})[0-9]{2,3})[^0-9] ]] || continue	 #5,6 & 7-digits (yymkk) (yymmkk) (yymmkkk)
    p=${BASH_REMATCH[1]}
    y=${BASH_REMATCH[2]}
    m=${BASH_REMATCH[3]}
    m='0'$m
    m=${m: -2}
	
	dng=${DST}/'20'${y}/${y}${m}/${p}/DNG
	[[ -d $dng ]] || mkdir -p \"$dng\" || continue
    
	mv -v -b \"$f1\" \"$dng\"

done < <(find $SRC -type f -iname '*.dng' -print0)


} 2>&1
exit 0

# 7  
Old 03-19-2020
Again: too many escapes. Drop one of the two back slashes in the read delimiter's -d definition. Same for ALL the escaped double quotes around file and directory names downstream.


Sure your sophisticated regex to analyse $n works reliably? I'd prefer to determine the number length upfront, and then dissect $n.
You might want to provide a default valuein case $DST is undefined, to avoid creating directories in root.
This User Gave Thanks to RudiC For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

SBATCH trinity for multiple files and rename/move the output files

Hey guys, I have wrote the following script to apply a module named "trinity" on my files. (it takes two input files and spit a trinity.fasta as output) #!/bin/bash -l #SBATCH -p node #SBATCH -A <projectID> #SBATCH -n 16 #SBATCH -t 7-00:00:00 #SBATCH --mem=128GB #SBATCH --mail-type=ALL... (1 Reply)
Discussion started by: @man
1 Replies

2. Shell Programming and Scripting

Move only folders and skipping files

How do I move all folders and its contents from a directory A to another directory B, skipping all files in Directory A ? ---------- Post updated at 12:53 PM ---------- Previous update was at 12:42 PM ---------- Ok. Got it. mv /A/*/ /B/ (1 Reply)
Discussion started by: DHeisenberg
1 Replies

3. Shell Programming and Scripting

Moving files to folders and rename

Hello! I am new to this. I have many files from b_ap00 to b_ap80, and I need to move them to folder 00 to 80 respectively, where b_ap00 is in folder 00, b_ap01 is in folder 01. On top of this, I need to rename the file once they are inside the folder to b_ot, and subsequently run it (ifort -o... (8 Replies)
Discussion started by: krustytherusty
8 Replies

4. Shell Programming and Scripting

How to move files to existing .tgz file?

Hi, I have already created the tar files. which consist of some log files and Audit.csv plz see the below code for that ================================================ tar -Pczf ARCH/${arc_date}.tgz $LOG/*.log $REYE/CEP/FiAdapter/Audit.${arc_date}.csv ... (3 Replies)
Discussion started by: pspriyanka
3 Replies

5. Shell Programming and Scripting

Move all files but not folders to a new folder

Hi, I have a sub directory with a number of files and folders. What i want is a subdirectory with just folders and not files for cleanliness sake. So I want to move the files into the new folder but keep the folders in the same place. Move all files (but not folders) to new folder. I am... (4 Replies)
Discussion started by: Hopper_no1
4 Replies

6. UNIX for Dummies Questions & Answers

Rename files to indexnumber in multiple folders

I have multiple subfolders with multiple jpg I want the group of files in each subfolder renamed to their index number (3 digits) in the folder, so: folder/a.jpg = folder/001.jpg folder/b.jpg = folder/002.jpg folder/c.jpg = folder/003.jpg folder2/1.jpg = folder2/001.jpg folder2/2.jpg =... (2 Replies)
Discussion started by: cmanniche
2 Replies

7. Shell Programming and Scripting

Move files to Folders

Hi Friends, Below is my requirement and i am not clear how to approach this issue in unix programming. I have a folder with 2500 files. The files are in below format. 1234_name1.txt 1234_name123.txt 4567_name1.txt 4567_name123.txt and i need a program which will read each file from this... (5 Replies)
Discussion started by: diva_thilak
5 Replies

8. UNIX for Dummies Questions & Answers

Move folders containing certain files

Hello, How can I move just the folders that contains files modified n days ago? Source tree: |-- SourceFolder | |-- Subfolder1 | | |-- file1.dat | | `-- file2.dat | |-- Subfolder2 | | |-- filea.dat | | `-- fileb.dat Destination tree: |-- ... (3 Replies)
Discussion started by: xavix
3 Replies

9. UNIX for Dummies Questions & Answers

Using Rename to move files

I am connecting to a remote server (Unix) and doing a ftp dowmload of files. The script (VB script) works fine except for not being able to move the downloaded files on the remote server to another folder. I need to move all files with an .asc extesnion from folder "tovecellio_edi" to folder... (1 Reply)
Discussion started by: snufse
1 Replies

10. Shell Programming and Scripting

Move files and rename ??

1. If I have a file-yyyymmdd.dat in a directory DATA1, then how do I move this file to directory DATA2 and the file name change to file-yyyymmdd.dat.currenttime I can manual do this $mv fileA-yyyymmdd.dat ./DATA2/fileA-yyyymmdd.dat.`date +%Y%m%d%H%M%S` but how do I move all of the files in... (1 Reply)
Discussion started by: sabercats
1 Replies
Login or Register to Ask a Question