Hi Friends,
I'm trying to create a script that allows me to recursively resize, crop (holding the center of the image) and optimize images jpg, jpeg, png for a specific folder and subfolder with the ability to exclude certain folder and its subdirectory.
Again, I should to do with this script:
1) exclude images under a certain KB size
2) exclude images previous a certain date
3) exclude images in specific folders or sub-folders
4) resize images to precise height (400px for example)
5) crop images (gravity center) transforming into the square image with each side by 400px
6) optimize the image compression ratio
Code:
for f in `find . -name "*.jpg"`
do
convert $f -resize x400\> $f.resized.jpg
done
then crop (gravity center) and optimize the result images
Code:
for f in `find . -name "*.jpg"`
do
convert $f -gravity center -crop 400x400+0+0 -quality 50% $f.resized.jpg
done
But still I don't know how exclude file for size or folder and eventually how to merge the whole in a single script
I'm not a scripting expert, so I ask you if you can help me.
I'm trying to create a script that allows me to recursively resize, crop (holding the center of the image) and optimize images jpg, jpeg, png for a specific folder and subfolder with the ability to exclude certain folder and its subdirectory.
Again, I should to do with this script:
1) exclude images under a certain KB size
2) exclude images previous a certain date
3) exclude images in specific folders or sub-folders
4) resize images to precise height (400px for example)
5) crop images (gravity center) transforming into the square image with each side by 400px
6) optimize the image compression ratio
Code:
for f in `find . -name "*.jpg"`
do
convert $f -resize x400\> $f.resized.jpg
done
then crop (gravity center) and optimize the result images
Code:
for f in `find . -name "*.jpg"`
do
convert $f -gravity center -crop 400x400+0+0 -quality 50% $f.resized.jpg
done
But still I don't know how exclude file for size or folder and eventually how to merge the whole in a single script
I'm not a scripting expert, so I ask you if you can help me.
The trick in programming is to use a step-by-step approach. Start dissecting the problem into smaller sub-problems and these into ever smaller sub-sub-problems until at the lowest level there is no problem any more. You haven't told us anything about your environment, so i assume you can use Korn shell and the standard utilities. So let us start:
First problem: how to treat a single file? You already answered that and we make a function to call for that purpose. Functions are like building blocks and since we will treat any file the same way it should be done by executing the same code over and over. The question is: what does that function need to know - we must provide this as parameter(s) to the function - and what must it return? The usual return code is 0 for success and any other value for various ways of failure.
So the first "script" we write will contain only this function and a call to that function with a single test file. We just want to see if everything goes right before we proceed:
Code:
#! /usr/bin/ksh
pConvertPicture ()
{
typeset fIn="$1" # here lands our parameter, the pathname of a file "/some/where/file"
typeset fTmp="/tmp/${fIn##*/}" # we extract the filename from the pathname: "/some/where/file" -> "/tmp/file"
convert "$fIn" -resize x400\> "$fTmp"
convert "$fTmp" -gravity center -crop 400x400+0+0 -quality 50% "${fIn}.resized.jpg"
rm "$fTmp"
return 0
}
pConvertPicture "/some/example/picture"
exit 0
As you see the function will give back always 0, regardless of the "convert"-program being successful or not. We don't want that but a reasonable return code, telling us if it succeeded and, if not, where it went wrong. Let us fix that before we proceed:
Code:
#! /usr/bin/ksh
pConvertPicture ()
{
typeset fIn="$1" # here lands our parameter, the pathname of a file "/some/where/file"
typeset fTmp="/tmp/${fIn##*/}" # we extract the filename from the pathname: "/some/where/file" -> "/tmp/file"
if ! convert "$fIn" -resize x400\> "$fTmp" ; then
print -u2 - "Error resizing ${fIn}, return code was $?"
return 1
fi
if ! convert "$fTmp" -gravity center -crop 400x400+0+0 -quality 50% "${fIn}.resized.jpg" ; then
print -u2 - "Error cropping ${fIn}, return code was $?"
return 2
fi
rm "$fTmp"
return 0
}
# --------- main program starts here ------
typeset -i iRetVal=0
# here we process our test file:
pConvertPicture "/some/example/picture" ; iRetVal=$?
if [ $iRetVal -eq 1 ] ; then # here we analyse the various return codes of pConvertPicture()
print -u2 - "pConvertPicture() failed to resize, exiting
exit 1
elif [ $iRetVal -eq 2 ] ; then
print -u2 - "pConvertPicture() failed to crop, exiting
exit 1
elif [ $iRetVal -gt 2 ] ; then
print -u2 - "miscellaneous error, exiting
exit 1
fi
exit 0
Notice that we remove the file in /tmp only if everything goes well, otherwise we leave it there because it may help to debug the problem. You won't maybe need all the diagnostic messages in the end but at the start is good to be generous with them - you can always remove them later. In general expect every program to fail at every single possible step - this in itself is not bad. As long as you are able to find out where it failed and why you can do something about it, so build into it everything that will help you analyse where it failed, why and in which way.
Start by trying this script under various conditions: change the file name with which the function is called to some non-existing file and see if the script fails accordingly (it should, otherwise some more logic is needed), try with a non-picture file (the convert utility should fail and the function accordingly too), etc.. Only if you are satisfied with the outcome, proceed:
Now, the next problem: we need to generate a list of files to feed into our function because the single test file business is growing old rapidly. So let us do that - with find, as you did, but in a slightly different way: first, we want full pathnames instead of relative ones, so we need to provide our script a starting directory as parameter. Let us set the problem of how to pass parameters aside for the moment and let us use a single path which we define in the script for now. I suggest that you copy one or two files along with several directories and subdirectories to this path, something like this:
Now we will implement the logic to traverse this tree: Notice that we need to pass only filenames to our function so we need to filter out only the files and only the ones ending in "jpg":
Satisfied? I am! So let us incorporate this into our script:
Code:
#! /usr/bin/ksh
pConvertPicture ()
{
typeset fIn="$1" # here lands our parameter, the pathname of a file "/some/where/file"
typeset fTmp="/tmp/${fIn##*/}" # we extract the filename from the pathname: "/some/where/file" -> "/tmp/file"
if ! convert "$fIn" -resize x400\> "$fTmp" ; then
print -u2 - "Error resizing ${fIn}, return code was $?"
return 1
fi
if ! convert "$fTmp" -gravity center -crop 400x400+0+0 -quality 50% "${fIn}.resized.jpg" ; then
print -u2 - "Error cropping ${fIn}, return code was $?"
return 2
fi
rm "$fTmp"
return 0
}
# --------- main program starts here ------
typeset -i iRetVal=0
typeset fToProcess=""
typeset fStartDir="/path/to/start" # this is fixed, but only for now
# we create a list of files and run these through our function:
for fToProcess in $( find "$fStartDir" -type f -name "*jpg" -print ) ; do
print - "processing now: $fToProcess"
pConvertPicture "$fToProcess" ; iRetVal=$?
if [ $iRetVal -eq 1 ] ; then # here we analyse the various return codes of pConvertPicture()
print -u2 - "pConvertPicture() failed to resize, exiting"
exit 1
elif [ $iRetVal -eq 2 ] ; then
print -u2 - "pConvertPicture() failed to crop, exiting"
exit 1
elif [ $iRetVal -gt 2 ] ; then # this is just a "catch-all" if the functions gets extended
print -u2 - "miscellaneous error, exiting"
exit 1
fi
done
exit 0
Next problem: we want to exclude some directories: again, let us agree on some fixed list, postponing the problem of how to pass it to the script to later. Always one step at a time. Removing certain subdirectories from the search is done like this (modifying the last find-example):
You see this takes away everything in /path/to/start/subdir2 from the result set. Alas, for several subdirectories to exclude this needs a further little twist (it makes little sense to first remove the subdirectory and the parent directory altogether, but it shows the mechanism and is syntactically correct).
Now we build that into our script, here is just the main portion, the rest is unchanged. Notice that we do not know in advance how many subdirectories to exclude, so we create an array for this type of information:
Code:
[snip]
# --------- main program starts here ------
typeset -i iRetVal=0
typeset fToProcess=""
typeset fStartDir="/path/to/start" # this is fixed, but only for now
typeset afExcludeDir[1]="/path/to/start/subdir1/subdir1_1" # you may want to put a file there to see if its really excluded
typeset afExcludeDir[2]="/path/to/start/subdir2/subdir2_1" # this should exclude file3.jpg but not file2.jpg
typeset chExclusion=""
typeset -i iCnt=0 # our array counter to loop through it
# First we construct the exclude-list from the array of dirs to exclude if there are
# any array elements at all: first a "\(", then "-path" and the first array element.
# for any further array element we append "-o -path", followed by that element.
# After the last element we append "\)" and "-prune -o".
# If there are no array elements we skip this step altogether.
if [ ${#afExcludeDir[*]} -gt 0 ] ; then # if there are any elements in the array
chExclusion="\\( -path \"${afExcludeDir[1]}\" "
(( iCnt = 2 ))
while [ $iCnt -le ${#afExcludeDir[*]} ] ; do
chExclusion="$chExclusion -o -path \"${afExcludeDir[$iCnt]}\""
(( iCnt += 1 ))
done
chExclusion="$chExclusion \\) -prune -o"
fi
# we create a list of files excluding some directories and run these through our function:
for fToProcess in $( find "$fStartDir" $chExclusion -type f -name "*jpg" -print ) ; do
print - "processing now: $fToProcess"
pConvertPicture "$fToProcess" ; iRetVal=$?
if [ $iRetVal -eq 1 ] ; then # here we analyse the various return codes of pConvertPicture()
print -u2 - "pConvertPicture() failed to resize, exiting"
exit 1
elif [ $iRetVal -eq 2 ] ; then
print -u2 - "pConvertPicture() failed to crop, exiting"
exit 1
elif [ $iRetVal -gt 2 ] ; then # this is just a "catch-all" if the functions gets extended
print -u2 - "miscellaneous error, exiting"
exit 1
fi
done
exit 0
Now, if this works satisfyingly we have a last problem to solve: how to pass parameters to the script in an organised way. Luckily, there is an utiity which does exactly help us with that: getopts. It can decode options that get no arguments and flags that do and still leaves the rest of the commandline so that the script can process it. I suggest you go over the man page of getopts carefully because i am going to skp over some of your requirements once i have shown the process to implement them:
To use getopts we need first to design our interface: which options should the script understand and how should they be formed?
We need:
- a path to start with (exactly one such path, so it needs not to be an option but a commandline argument)
- none, one or several directories to exclude. These will be options with one argument (a path). Since we are eXcluding we can use an "-x" to introduce each of them. We will also check these directories to see if they exist, are accessible, etc.. Feel free (in fact you are very welcome) to expand on the checks if you find a possible error condition i oversaw.
Now, i don't know how many files you have and how time-consuming the conversion is but probably you want the possibility for a "dry run"? How about
- an (optional) option "-S" (simulate) which doesn't really execute the the conversion but only prints the commands that would take place?
Now, here we are: the script will take one commandline parameter ("/path/to/start" in the above example). Again, the function is left out as it isn't changed:
Code:
[snip]
# --------- main program starts here ------
typeset -i iRetVal=0
typeset fToProcess=""
typeset fStartDir=""
typeset afExcludeDir[1]="/path/to/start/subdir1/subdir1_1" # you may want to put a file there to see if its really excluded
typeset afExcludeDir[2]="/path/to/start/subdir2/subdir2_1" # this should exclude file3.jpg but not file2.jpg
typeset chExclusion=""
typeset -i iCnt=0 # our array counter to loop through it
if [ -d "$1" ] ; then # if the first commandline argument is a directory
fStartDir="$1" # then make it the starting dir
shift
else # otherwise complain and exit
print -u2 "Error: specify a starting point for the operation."
exit 1
fi
if [ ${#afExcludeDir[*]} -gt 0 ] ; then # if there are any elements in the array
chExclusion="\\( -path \"${afExcludeDir[1]}\" "
(( iCnt = 2 ))
while [ $iCnt -le ${#afExcludeDir[*]} ] ; do
chExclusion="$chExclusion -o -path \"${afExcludeDir[$iCnt]}\""
(( iCnt += 1 ))
done
chExclusion="$chExclusion \\) -prune -o"
fi
for fToProcess in $( find "$fStartDir" $chExclusion -type f -name "*jpg" -print ) ; do
print - "processing now: $fToProcess"
pConvertPicture "$fToProcess" ; iRetVal=$?
if [ $iRetVal -eq 1 ] ; then # here we analyse the various return codes of pConvertPicture()
print -u2 - "pConvertPicture() failed to resize, exiting"
exit 1
elif [ $iRetVal -eq 2 ] ; then
print -u2 - "pConvertPicture() failed to crop, exiting"
exit 1
elif [ $iRetVal -gt 2 ] ; then # this is just a "catch-all" if the functions gets extended
print -u2 - "miscellaneous error, exiting"
exit 1
fi
done
exit 0
Now for the getopts: we need to implement 2 options: -S takes no arguments, "-x" takes one. "-x" can be specified multiple times. getopts will be called as often as there are options to process and every time it is called it cuts off one option (plus argument if any) from the commandline. We start with "-S":
Code:
#! /usr/bin/ksh
pConvertPicture ()
{
typeset fIn="$1"
typeset fTmp="/tmp/${fIn##*/}"
if ! $SIMULATE convert "$fIn" -resize x400\> "$fTmp" ; then
print -u2 - "Error resizing ${fIn}, return code was $?"
return 1
fi
if ! $SIMULATE convert "$fTmp" -gravity center -crop 400x400+0+0 -quality 50% "${fIn}.resized.jpg" ; then
print -u2 - "Error cropping ${fIn}, return code was $?"
return 2
fi
$SIMULATE rm -f "$fTmp"
return 0
}
# --------- main program starts here ------
typeset -i iRetVal=0
typeset fToProcess=""
typeset fStartDir=""
typeset afExcludeDir[1]="/path/to/start/subdir1/subdir1_1" # you may want to put a file there to see if its really excluded
typeset afExcludeDir[2]="/path/to/start/subdir2/subdir2_1" # this should exclude file3.jpg but not file2.jpg
typeset chExclusion=""
typeset -i iCnt=0 # our array counter to loop through it
typeset chOpt="" # our buffer for commandline processing
typeset SIMULATE=""
while getopts ":S" chOpt ; do # commandline
case $chOpt in
S) # simulation flag
SIMULATE=' print - '
;;
"?")
if [ "$chOpt" == "?" -a "$OPTARG" == "?" ] ; then
print -u2 "a help option is not implemented yet"
else
print -u2 "unknown option -${OPTARG}"
fi
exit 1
;;
esac
done
shift $(( OPTIND -1 )) # get everything already processed off the command line
if [ -d "$1" ] ; then # if the (now) first commandline argument is a directory
fStartDir="$1" # then make it the starting dir
shift
else # otherwise complain and exit
print -u2 "Error: specify a starting point for the operation."
exit 1
fi
if [ ${#afExcludeDir[*]} -gt 0 ] ; then # if there are any elements in the array
chExclusion="\\( -path \"${afExcludeDir[1]}\" "
(( iCnt = 2 ))
while [ $iCnt -le ${#afExcludeDir[*]} ] ; do
chExclusion="$chExclusion -o -path \"${afExcludeDir[$iCnt]}\""
(( iCnt += 1 ))
done
chExclusion="$chExclusion \\) -prune -o"
fi
for fToProcess in $( find "$fStartDir" $chExclusion -type f -name "*jpg" -print ) ; do
print - "processing now: $fToProcess"
pConvertPicture "$fToProcess" ; iRetVal=$?
if [ $iRetVal -eq 1 ] ; then # here we analyse the various return codes of pConvertPicture()
print -u2 - "pConvertPicture() failed to resize, exiting"
exit 1
elif [ $iRetVal -eq 2 ] ; then
print -u2 - "pConvertPicture() failed to crop, exiting"
exit 1
elif [ $iRetVal -gt 2 ] ; then # this is just a "catch-all" if the functions gets extended
print -u2 - "miscellaneous error, exiting"
exit 1
fi
done
exit 0
Next, we implement "-x": we need to construct the array afExcludeDir[] with each new "-x" we encounter. This involves some array arithmentic:
Code:
#! /usr/bin/ksh
pConvertPicture ()
{
typeset fIn="$1"
typeset fTmp="/tmp/${fIn##*/}"
if ! $SIMULATE convert "$fIn" -resize x400\> "$fTmp" ; then
print -u2 - "Error resizing ${fIn}, return code was $?"
return 1
fi
if ! $SIMULATE convert "$fTmp" -gravity center -crop 400x400+0+0 -quality 50% "${fIn}.resized.jpg" ; then
print -u2 - "Error cropping ${fIn}, return code was $?"
return 2
fi
$SIMULATE rm -f "$fTmp"
return 0
}
# --------- main program starts here ------
typeset -i iRetVal=0
typeset fToProcess=""
typeset fStartDir=""
# typeset afExcludeDir[]
typeset chExclusion=""
typeset -i iCnt=0 # our array counter to loop through it
typeset chOpt="" # our buffer for commandline processing
typeset SIMULATE=""
while getopts ":Sx:" chOpt ; do # commandline
case $chOpt in
S) # simulation flag
SIMULATE=' print - '
;;
x) # dir to exclude
if [ -d "$OPTARG" ] ; then
typeset afExcludeDir[$((${#afExcludeDir[@]}+1))]="$OPTARG"
else
"print -u2 - "Error: exclusion directory $OPTARG does not exist"
fi
;;
"?")
if [ "$chOpt" == "?" -a "$OPTARG" == "?" ] ; then
print -u2 "a help option is not implemented yet"
else
print -u2 "unknown option -${OPTARG}"
fi
exit 1
;;
esac
done
shift $(( OPTIND -1 )) # get everything already processed off the command line
if [ -d "$1" ] ; then # if the (now) first commandline argument is a directory
fStartDir="$1" # then make it the starting dir
shift
else # otherwise complain and exit
print -u2 "Error: specify a starting point for the operation."
exit 1
fi
if [ ${#afExcludeDir[*]} -gt 0 ] ; then # if there are any elements in the array
chExclusion="\\( -path \"${afExcludeDir[1]}\" "
(( iCnt = 2 ))
while [ $iCnt -le ${#afExcludeDir[*]} ] ; do
chExclusion="$chExclusion -o -path \"${afExcludeDir[$iCnt]}\""
(( iCnt += 1 ))
done
chExclusion="$chExclusion \\) -prune -o"
fi
for fToProcess in $( find "$fStartDir" $chExclusion -type f -name "*jpg" -print ) ; do
print - "processing now: $fToProcess"
pConvertPicture "$fToProcess" ; iRetVal=$?
if [ $iRetVal -eq 1 ] ; then # here we analyse the various return codes of pConvertPicture()
print -u2 - "pConvertPicture() failed to resize, exiting"
exit 1
elif [ $iRetVal -eq 2 ] ; then
print -u2 - "pConvertPicture() failed to crop, exiting"
exit 1
elif [ $iRetVal -gt 2 ] ; then # this is just a "catch-all" if the functions gets extended
print -u2 - "miscellaneous error, exiting"
exit 1
fi
done
exit 0
We are done and you should be able to implement the rest of your requirements easily now. For instance: a minimum size for files - goes to the process function itself, just return if the file is too small. You may also want to skip over already processed files (tip: put a condition into the processing function to look for a file named like the target file but with ".resized.jpg" at the end).
Some finishing touches you should implement, though: an instructive help message (see the program code for pointers where to put it), some more checks if files are really in JPG format (the extension actually says nothing, i could rename any file to "file.jpg") for which i recommend the file utility.
I hope this helps.
bakunin
Last edited by bakunin; 11-11-2018 at 03:41 AM..
These 3 Users Gave Thanks to bakunin For This Post:
WAW! A real lesson in code development!
I would never have thought of such a complete and authoritative answer!
I'm really very grateful! I'll show your answer to some friends I think they'll be amazed!
Regarding what you suggest, it is believed that the utility "file" could be very useful.
Also find the way to not process files already modified by the script.
Unfortunately I do not know if I will succeed in the enterprise, but I try, but give me "some time" to process everything (I'm still under shok! :-))...
Regarding the environment in use you're absolutely right, I forgot to specify it, I'm in a Debian Jessie (Bourne shell), can the script work?
I'll try!
If I can complete it I would like to publish it (not for personal glory, but only to help other people like me who are looking for something like this. You must to know that my search started for a Joomla plugin that had such features, but I did not find it, and I thought it could help GNU Linux!) on GitLab, what do you think about? ;-)
Thanks again!
Davide
------ Post updated at 10:43 AM ------
...relatively "file" utility I read that it would be preferable for safety reasons "identify": h t t p s : / / unix.stackexchange.com/questions/189364/script-to-determine-if-files-are-images
Location: Asia Pacific, Cyberspace, in the Dark Dystopia
Posts: 19,118
Thanks Given: 2,351
Thanked 3,359 Times in 1,878 Posts
Quote:
Originally Posted by bakunin
.
.....
Some finishing touches you should implement, though: an instructive help message (see the program code for pointers where to put it), some more checks if files are really in JPG format (the extension actually says nothing, i could rename any file to "file.jpg") for which i recommend the file utility.
I hope this helps.
bakunin
Wow. What a great reply and implementation tutorial.
Hi All,
I have written a new script to check for DB space and size of dump log file before it can be imported into a Oracle DB.
I'm relatively new to shell scripting.
Please help me optimize this script further. (0 Replies)
Hello,
I'm wondering if there is a quicker way of doing this.
Here is my mv script.
d=/conversion/program/out
cd $d
ls $d > /home/tempuser/$$tmp
while read line ; do
a=`echo $line|cut -c1-5|sed "s/_//g"`
b=`echo $line|cut -c16-21`
if ;then mkdir... (13 Replies)
Hi,
I need a shell script to determine if a no. is either even, greater than 4, less than 8
SHELL : ksh
OS : RHEL 6
this is the if block of the script
mod=`expr $num % 2`
if || ||
then
echo "No. is either even or greater than 4 or less than 8"
fi
this code works... (2 Replies)
Dear Forum experts
I have the below script which I made to run under bash shell, it runs perfectly for low records number, let us say like 100000. when I put all records (3,000,000), it's takes hours
can you please suggest anything to optimize or to run in different way :-|
{OFS="|";... (6 Replies)
Hi All,
There is a script (test.sh) which is taking more CPU usage. I am attaching the script in this thread.
Could anybody please help me out to optimize the script in a better way.
Thanks,
Gobinath (6 Replies)
I am having a problem getting this to work right. The script needs to search through directories and subdirectories. If a jpg is found then create a folder in that directory, so on and so forth. Here is what I have so far but it doesn't work right. Help please
#!/bin/bash
for d in `find ./... (1 Reply)
System: Ubuntu Intrepid Ibex
I'm running webcamd as a sort of "security" program, but I need a script that will archive my webcam.jpg files.
So, take the following file:
/home/slag/www/webcam.jpg
Rename it--preferably with a time stamp.
Place it in say:
/home/slag/www/history/
... (4 Replies)
I've a script to do some snapshots but the time it does so is very different...
once i got a snapshot under 1 sec, on the other hand it took 3 sec, but nothing else changed, i didnt even move the cursor or something.
I put the script on a ramdisk and its faster, but still swing from under 1... (1 Reply)
Hi All ,
I am just a new bie in Unix/Linux .
With help of tips from 'here and there' , I just created a simple script to
1. declare one array and some global variables
2. read the schema names from user (user input) and want2proceed flag
3. if user want to proceed , keep reading user... (8 Replies)
Hi,
I have this following script below. Its searching a log file for 2 string and if found then write the strings to success.txt and If not found write strings to failed.txt . if one found and not other...then write found to success.txt and not found to failed.txt.
I want to optimize this... (3 Replies)
11-10-2018
