Script behaves different when run from cron vs. manually
Hey all,
Just wanted to get some input on a script I am using to import files into a MySQL database.
The process is pretty simple: my main server exports these files and FTPs them. I have a script that FTPs them to the machine running that runs this script. The FTP script runs without issue every time, bith from the cron or manually.
The problem occurs when the cron runs the script below. Please be kind , these are my first scripts.
The loop runs fine the first time , but when it loops back again it dies at the point where mysqlimport is called again. The first file , in this case the website_tabProperty.txt works fine. It dies at website_tabBuilding.txt.
My cron is set to run this script as root.
I have been running this script manually "./" for a long time and works every time.
I figure its a problem with the loop and the cron but I cant come up with anything.
Thanks for your help.
Bill
Code:
#!/bin/bash
echo "Starting new batch import `date`"
echo "Starting new batch import `date`" >> /root/import.log
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#Imporatnt: The last file and table in these arrays MUST be the data that is targeted for
#the append operation.
#define the files names for import and existance verification. These are renamed to the names below
file_name[1]=/root/website_tabProperty.txt
file_name[2]=/root/website_tabBuilding.txt
file_name[3]=/root/website_tabApartment.txt
file_name[4]=/root/website_tabMeters.txt
file_name[5]=/root/website_tabReadings.txt
#define the file names for importation. The file names must match the table names
target_file_name[1]=/root/tabProperty.txt
target_file_name[2]=/root/tabBuilding.txt
target_file_name[3]=/root/tabApartment.txt
target_file_name[4]=/root/tabMeters.txt
target_file_name[5]=/root/tabReadings.txt
#define the target tables to flush
target_table[1]=tabProperty
target_table[2]=tabBuilding
target_table[3]=tabApartment
target_table[4]=tabMeters
target_table[5]=tabReadings
#this number nust be one higher than the max index in the arrays.
MAX_LOOP=6
#Set this value to the index we want to append - will force it to skip the flush
APPEND_INDEX=5
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#set the file name suffix for the renaming routine
suffix=`date +%m_%d_%Y`
COUNTER=1
while [ $COUNTER -lt $MAX_LOOP ]; do
echo "---------------------------------------"
echo "---------------------------------------" >> /root/import.log
echo "Processing index - "$COUNTER
echo "Processing index - "$COUNTER >> /root/import.log
DIR=${file_name[COUNTER]}
if [ -f $DIR ];then
echo "Original "${file_name[COUNTER]}" exists"
echo "Original "${file_name[COUNTER]}" exists" >> /root/import.log
echo "Renaming "${file_name[COUNTER]}
echo "Renaming "${file_name[COUNTER]} >> /root/import.log
cp ${file_name[COUNTER]} ${target_file_name[COUNTER]}
echo "Removing "${file_name[COUNTER]}
echo "Removing "${file_name[COUNTER]} >> /root/import.log
rm ${file_name[COUNTER]}
#Only flush the table if it is not the last one in the index
if [ $COUNTER != $APPEND_INDEX ]; then
echo "Flushing "${target_table[COUNTER]}
echo "Flushing "${target_table[COUNTER]} >> /root/import.log
mysql -u root -p8895asdf7665553 -e "use wellspring; delete from "${target_table[COUNTER]}";"
sleep 2
else
echo "Not Flushing "${target_table[COUNTER]}
echo "Not Flushing "${target_table[COUNTER]} >> /root/import.log
fi
echo "Importing "${target_file_name[COUNTER]}
echo "Importing "${target_file_name[COUNTER]} >> /root/import.log
mysqlimport -u root -p8895asdf7665553 --local wellspring ${target_file_name[COUNTER]}
sleep 2
echo "Renaming "${target_file_name[COUNTER]}
echo "Renaming "${target_file_name[COUNTER]} >> /root/import.log
cp ${target_file_name[COUNTER]} ${target_file_name[COUNTER]}.$suffix
sleep 1
echo "Removing "${target_file_name[COUNTER]}
echo "Removing "${target_file_name[COUNTER]} >> /root/import.log
rm ${target_file_name[COUNTER]}
sleep 1
else
echo "Original "${file_name[COUNTER]}" does not exist"
echo "Original "${file_name[COUNTER]}" does not exist" >> /root/import.log
fi
let COUNTER=COUNTER+1
done
echo "---------------------------------------"
echo "---------------------------------------" >> /root/import.log
echo "Finished batch import `date`"
echo "Finished batch import `date`" >> /root/import.log
echo "*********************************************************************************"
echo "*********************************************************************************" >> /root/import.log
exit
There are tons of the same questions for cronjobs here on the forum so that you would find plenty with the search function provided by the forum. Also check this:
There are tons of the same questions for cronjobs here on the forum so that you would find plenty with the search function provided by the forum. Also check this:
While I do appreciate the link and the suggestion to use search ---which I did -- there was not a great amount of specific information about how the environment influences scripts specifically. I was hoping that someone might have experience in this area and could point me in the direction of using more vanilla (ie - runs no matter what) vs what I have done above.
By the way, I wish I had a nickel for every hour I played Zaxxon on my C64 as a kid.
Hello Lads,
I deployed a script on my mac to start and stop EC2 instances on AWS console. The script when started manually on the terminal does the expected stop and start. Problem is when i try to schedule it on a cron, it fails to recognize the AWS Keys which i set up as ENV variable by... (2 Replies)
Hi,
I "borrowed" a script I found online, to start a SAP router application on a Solaris 11 (SPARC) server.
The script runs fine when calling it manually, but when I schedule it to run from cron, it doesn't. I don't see any warning or failure messages anywhere, just nothing happens.
... (11 Replies)
Hello gurus,
I am making what I think is a simple db2 call from within a shell script but I am having difficulty producing the desired
report when I run the script shown below from a shell script in cron. For example, my script and the crontab file setup
is shown below:
#!/bin/ksh
db2... (3 Replies)
Hi, I've got a Bash backup script I'm trying to run on a directory via a cron job nightly. If I ssh in and run the script manually it works flawlessly. If I set up the cron to run evertything is totally messed up I don't even know where to begin.
Basically the path structure is
... (6 Replies)
I get a different output when i manually run the .sh script and when it is run by a cron job. Please help me ..
TMP1="/lhome/bbuser/script/wslog/sar.t1"
TMP2="/lhome/bbuser/script/wslog/sar.t2"
TMP3="/lhome/bbuser/script/wslog/sar.t3"
OUTPUT="/lhome/bbuser/script/wslog/sar.out"... (8 Replies)
Hello Every one,
I have a shell script which is running fine manually, but its giving me hard time when running tru cron job. :wall:.
Am using #!/usr/bin/ksh
>echo $SHELL
/usr/bin/ksh
Cron Job is as below, it execues but dosent do what i want it to do.
47 15 * * *... (1 Reply)
Hi have a script which transferers from Microsoft server to Linux box.
The scripts(ksh) is on Linux box.
If I run script from terminal, it transfers files to directory. Where as If
I run script from CRON. It does not.
Here is the log of both:
Terminal execution log:... (2 Replies)
I am trying to run a sript on cron to SFTP data to a company. Private and public keys are set up.
When I run this manully it works fine, however it was failing when run on cron. I have narrowed down the problem - it fails at the code that says if the error code is 0 then continue
. . . I... (2 Replies)
I want to create an automated script which is called by another maually executed script.
The condition is that the no one should be able to manually execute the automated script.
The automated script can be on the same machine or it can be on a remote machine.
Can any one suggest a check in the... (1 Reply)
Hi all,
I have a main script (called OracleCleanup) that runs some sql queries. that runs off a wrapper script which contains the sources for the login information and and JOB_HOME (the script is below). When I schedule this job in the cron the log says that it cannot open my list file, which... (4 Replies)