Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Bash and Awk for creating directories and moving files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Bash and Awk for creating directories and moving files
# 1  
Old 07-24-2008
Bash and Awk for creating directories and moving files
I have a security system that FTPs the camera files to my machine, however I want to sort the pictures (taken every 30s) into directories by hour.

Every picture uses the following file format.
yymmddhhmmsstt.jpg (where tt is the milliseconds)

I am thinking the for loop is best

for file in *.jpg
do
?here?
done

I know its very vague, however the end result is I am guessing use awk (or better?) to create variables from the yy mm dd hh parts of the filenames, then creating directories based on those variables nested with a check to see if the directory exists, creating the directory if it doesnt.

i.e. (the way I imagine it working, I dont know how to actually construct the script to do this)

for file in 08072400123200.jpg
do
%y=08
%m=07
%d=24
%h=00
if not exist directory %y - mkdir %y
if not exist directory %y/%m - mkdir %y/%m
if not exist directory %y/%m/%d - mkdir %y/%m/%d
if not exist directory %y/%m/%d/%h - mkdir %y/%m/%d/%h
mv %y%m%d%h*.jpg %y/%m/%d/%h
done

Moving all the pictures taken within that hour into the respective subdirectories.

Any help would be much appreciated.
# 2  
Old 07-24-2008
using bash, you should start reading up here. It shows you how to get substrings.
# 3  
Old 07-24-2008
What I have so far, that I imagine should work (but doesnt) is (and isnt very elegant or sane)

Code:
#!/bin/bash

for file in *.jpg; do

set yy = `echo $file | awk '{split($0,a,""); print a[1]a[2]}'`
set mm = `echo $file | awk '{split($0,a,""); print a[3]a[4]}'`
set dd = `echo $file | awk '{split($0,a,""); print a[5]a[6]}'`
set hh = `echo $file | awk '{split($0,a,""); print a[7]a[8]}'`

if [ -d $yy ]; then
        if [ -d $yy/$mm ]; then
                if [ -d $yy/$mm/$dd ]; then
                        if [ -d $yy/$mm/$dd/$hh ]; then
                                mv $yy$mm$dd$hh*.jpg $yy/$mm/$dd/$hh/$file
                        elif
                                mkdir $yy/$mm/$dd/$hh
                                exit 1
                        fi
                elif
                        mkdir $yy/$mm/$dd
                        exit 1
                fi
        elif
                mkdir $yy/$mm
                exit 1
        fi
elif
        mkdir $yy
        exit 1
fi

done

# 4  
Old 07-24-2008
Even modifying for various shell stupidity doesnt help

./filter.sh: line 17: syntax error near unexpected token `fi'
./filter.sh: line 17: ` fi'

Code:
#!/bin/bash -x

for file in *.jpg; do

YY = "echo $file | awk '{split($0,a,""); print a[1]a[2]}'"
MM = "echo $file | awk '{split($0,a,""); print a[3]a[4]}'"
DD = "echo $file | awk '{split($0,a,""); print a[5]a[6]}'"
HH = "echo $file | awk '{split($0,a,""); print a[7]a[8]}'"

if [ -d $YY ]; then
        if [ -d $YY/$MM ]; then
                if [ -d $YY/$MM/$DD ]; then
                        if [ -d $YY/$MM/$DD/$HH ]; then
                                mv $YY$MM$DD$HH*.jpg $YY/$MM/$DD/$HH/$file
                        elif
                                mkdir $YY/$MM/$DD/$HH
                        fi
                elif
                        mkdir $YY/$MM/$DD
                fi
        elif
                mkdir $YY/$MM
        fi
elif
        mkdir $YY
fi

done

# 5  
Old 07-24-2008
We do not have to test for individual directory level 'cos if the bottom directory exists, the parent level should exist. Also, mkdir -p will make all the non-existing parent directories. Also, I introduce 'short circurt' && to ensure directory exist before I move the file.

We can also avoid all the repeating code using in extracting yy/mm/dd/hh by using 'set --' and sed. sed will change 2 digits with 2 digits + space so that it can put the result back to "set --" to set the positional variables accordingly

This is my contribution, it should work (even on sh)
Code:
for i in *.jpg
do
   # yy is $1, mm is $2, dd is $3, hh is $4
   set -- `echo $i | sed -e 's/\([0-9][0-9]\)/\1 /g'`
   dir="$1/$2/$3/$4"
   [ ! -d $dir ] && mkdir -p $dir && mv $i $dir
done

# 6  
Old 07-24-2008
Well, its working (for those out there that stumble across this site for a similar script)

Its not elegant, sane or other ... but it works.

If the better experienced here can clean it up and solve the elegant/sane issues then it would be great Smilie

Code:
#!/bin/bash -x

for file in *.jpg; do

YY=`echo $file | awk '{split($0,a,""); print a[1]a[2]}'`
MM=`echo $file | awk '{split($0,a,""); print a[3]a[4]}'`
DD=`echo $file | awk '{split($0,a,""); print a[5]a[6]}'`
HH=`echo $file | awk '{split($0,a,""); print a[7]a[8]}'`

if [ -d $YY ]
then
        if [ -d $YY/$MM ]
        then
                if [ -d $YY/$MM/$DD ]
                then
                        if [ -d $YY/$MM/$DD/$HH ]
                        then
                                mv $YY$MM$DD$HH*.jpg $YY/$MM/$DD/$HH
                        else
                                mkdir $YY/$MM/$DD/$HH
                        fi
                else
                        mkdir $YY/$MM/$DD
                fi
        else
                mkdir $YY/$MM
        fi
else
        mkdir $YY
fi

done

# 7  
Old 07-24-2008
Shorther:
Code:
for i in *.jpg;do d=${i:0:8};test -d $d || mkdir $d ;mv $i $d;done
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need BASH Script Help to Move Files While Creating Directories

I've got this script to loop through all folders and move files that are more than 2 years old. I'm using the install command because it creates the necessary directories on the destination path and then I remove the source. I'd like to change the script to use the mv command since it is much... (4 Replies)
Discussion started by: consultant
4 Replies

2. Shell Programming and Scripting

Archiving and moving files into directories, creating directories, etc.

how can i move "dataName".sql.gz into a folder called 'database' and then move "$fileName".tar.gz * .htaccess into a folder called 'www' with the entire gzipped file being "$fileName".tar.gz? Is this doable or overly complex. so mydemo--2015-03-23-1500.tar.gz > database -... (5 Replies)
Discussion started by: wyclef
5 Replies

3. Shell Programming and Scripting

Need Help Moving Long List Of Files Into Directories

I am very new to BASH and I am having difficulties moving a long list of image files into similarly named directories. I've been trying to come with a script all night and no luck. Here is what my list of files looks like: DSC_0059_01.jpg DSC_0059_02.jpg DSC_0059_03.jpg DSC_0059_04.jpg... (5 Replies)
Discussion started by: jowens1138
5 Replies

4. Shell Programming and Scripting

moving files to different directories

im trying to move media and other files which are in a specified directory to another directory and create another one if it does not exits(where the files will go),them also create a directory will the remaining files with different extensions will go.my first problem is that my script is not... (8 Replies)
Discussion started by: elginmulizwa
8 Replies

5. Shell Programming and Scripting

Moving files from several directories into parent

I am fairly new to bash(but am proficient in C++), and have only completed a few simple scripts. This is my first script that I actually need to do a serious task. All of my audiobooks are stored in traditional MP3 format: Music/Artist/Album/*.mp3 (which in this case is... (0 Replies)
Discussion started by: gamendorf
0 Replies

6. Shell Programming and Scripting

moving files between directories !!

hi i have a list of directory in a text file with all directories name in a column.(this is not exactly a file but i need to do a grep and awk on a file to find that list) i have the source folders like abchome/abc/xxyz/nl_xxabc/mm// v01 ... (4 Replies)
Discussion started by: debu000
4 Replies

7. UNIX for Dummies Questions & Answers

Moving files out of multiple directories and renaming them in numerical order

Hi, I have 500 directories each with multiple data files inside them. The names are sort of random. For example, one directory has files named e_1.dat, e_5.dat, e_8.dat, etc. I need to move the files to a single directory and rename them all in numerical order, from 1.dat to 1000(or some... (1 Reply)
Discussion started by: renthead720
1 Replies

8. Shell Programming and Scripting

moving files and creating a link to it

Hi All, I am in a tricky situation where I have to move my files to a different mount point and create a link in place of the file which will point to the moved location. to explain you in details:- say I have two mount points /dir/mount1/ /dir/mount2/ I have my application... (5 Replies)
Discussion started by: rpraharaj84
5 Replies

9. UNIX for Dummies Questions & Answers

Moving files between directories using SFTP

I want to connect to an SFTP server, GET some files, then move those files to a different directory on the SFTP server so I don't try to GET them next time. But there doesn't seem to be a way to move files between directories on the remote server from SFTP. I missing something obvious? And if... (6 Replies)
Discussion started by: cjhancock
6 Replies

10. Shell Programming and Scripting

help needed with creating challenging bash script with creating directories

Hi, Can someone help me with creating a bash shell script. I need to create a script that gets a positive number n as an argument. The script must create n directories in the current directory with names like map_1, map_2 etcetera. Each directory must be contained within its predecessor. So... (7 Replies)
Discussion started by: I-1
7 Replies
Login or Register to Ask a Question