UNIX for Dummies Questions & Answers

I am new at the korn shell and have inherited a script i am trying to make work the errors im getting are as follows:

ERRORS
+ pa_batching.ksh
pa_batching.ksh: factset: not found
pa_batching.ksh[9]: automation.: not found
pa_batching.ksh[22]: CT: not found
pa_batching.ksh[24]: syntax error at line 52 : `for' unmatched

The script itself is as below , it is a batch processing script

ANY IDEAS AS TO WHERE I SHOULD LOOK?

SCRIPT

factset $ more pa_batching.ksh
#! /bin/ksh
#
# Filename: factset_transfer
#
# Author: Steven bowskill
#
# Description: This file is the main script for the PA BATCHING FactSet
automation.
#
#
#-------------------------------------------------------------------------------

#set -vx
exit_handler()
{
printLog "$1: $2"
exit $1
}

#FUNCTION TO SEARCH FOR THE CORRECT NAME OF THE FILE AND PLACE THEM IN THE CORRE
CT DIRECOTRY

function getfilesandcopy
{

##set up some vars
paftpPramFile='pa_batch_put.cfg'
paremote_node=lon-ks-util01
paremote_user=ftpwilshire
paremote_password=wilshire
paftpcommand=PUT
pafileExt=
paftptransmethod=binary
##export them now

export paremote_node
export paremote_user
export paremote_password
export paftpcommand
export pafileExt
export paftptransmethod
export paftpPramFile

#create the file of the xls file in then this file will also con
tain the accounts

awk -F']' '/]/ {print $2}' $1 > $palocal_check/factserepor
ts.txt

# now go thru this file lookgin for the account
for acct in `cat $palocal_check/factsereports.txt`
do
echo "Start to search **********************************
****************************"


#get the factset file
filetoDownload=`echo $acct | awk -F',' '{print $1}'
`

#get the xls file to download
XlsfiletoDownload=`echo $acct | awk -F',' '{print $
1}'`

filetoDownload=`echo $acct | awk -F',' '{print $2}'
`
filetoDownload=`echo $f
iletoDownload | awk -F'_' '{print $1"_"$2"_"$3"_"$4}'`



printLog "searching for ${filetoDownload}"

cut -d "/" -f2-10 $palocal_check/dirs.txt > $palocal_check/dirsfound.txt

cat $palocal_check/dirsfound
d.txt

##get the correct name of the file from the dirs.txt fi
le
## NewFileToCopy=`awk -F, '/
'$filetoDownload'/ {print $3}' $palocal_check/dirs.txt`

##echo $NewFileToCopy

DirectoryFound=`cat $palocal_check/dirsfound.txt`



#did we get a match
if [ ! -z ${DirectoryFound} ] ;
then

while read dirs
do
#now ch
eck the file exists here
printLo
g xls file $XlsfiletoDownload
if [ -f ${XlsfiletoDownload} ] ; then


#first lets r
ename the factset file


# cp ${palocal_check}/$XlsfiletoDownload ${palocal_check}/${filetoDownload}
.xls

##now need to ftp the file from sun42 to
the g drive

##set up the ftp vars

padataname=$XlsfiletoDownload
paremote_dir=$dirs
palocal_dir=$palocal_check

#export the vars
export padataname
export paremote_dir
export palocal_dir
# now ftp the file
printLog "Ftp ${filetoDownload} to ${dir
s} this is the factset file ${XlsfiletoDownload} "
ftpfile

#copy the xls file
cp ${palocal_check}/$XlsfiletoDownload ${p
alocal_check}/${filetoDownload}.xls
#now delete the xls file from factset this will
tell us what was not found
rm ${pa
local_check}/$XlsfiletoDownload


fi


done <
$palocal_check/dirsfound.txt

else
continue
fi
echo "Finish search **********************************
****************************"
echo
echo

done

}


# FUNCTION TO UNZIP THE FILES FTP'D FROM THE FACTSET SERVER
#-------------------------------------------------------------------------------
function unzipfile
{

unzip -ql $palocal_check/$1

}
#-------------------------------------------------------------------------------


#FUNCTION TO FTP THE ZIP FILE, WE CAN NOT USE LIST AS FACTSET DOES NOT ALLOW IT
# SO WE HAVE TO GET THE FILES ONE BY ONE
#-------------------------------------------------------------------------------

function ftpzipfiles
{

#set up the zip ext as we are always going to use this
pafileExt='.ZIP'
export pafileExt
paftptransmethod=bin
export paftptransmethod

lnCount=1

for filename in `cat ${palocal_check}/1global_daily.tag_zip`
do


let lnCount=lnCount+1

# if [ ${lnCount} -eq 26 ] ; then

if [ ${lnCount} -lt 5 ] ; then

filenametodownload=`echo $filename | awk -F'.' '{print $1}'`

padataname=${filenametodownload}
export padataname
#ftp the file from the factset server
ftpfile
#unzip the file
cd $palocal_check
unzip -qo $palocal_check/$filename



fi

done

}




#FUNCTION TO PRINT THE MESSAGES TO THE LOG FILE
#-------------------------------------------------------------------------------

printLog()
{
print "`date '+%H:%M:%S'`> $1"
return
}

#-------------------------------------------------------------------------------





# Set up basic environment

#delete the log file we want a clean one
rm /opt/lonsched/log/pabatching.log

#set the log file up
logfile=/opt/lonsched/log/pabatching.log
[ -z ${logfile} ] && logfile=/dev/tty
export logfile

exec 1>> ${logfile}
exec 2>&1

printLog "Setting environment parameters .." date

. /opt/lonsched/scripts/factset/pabatching_params.ksh
let retstat=$?
if [ ${retstat} -ne 0 ] ; then
exit_handler 101 "Failed to set up environment - unknown mode setting"
fi



# Derive the data file name from the tag file name and make it
# available to runFTP.
#

#lcDate=`date +"%d%b%Y" | tr "[a-z]" "[A-Z]"`
lcDate="04OCT2004"

#set up the vars and cal the ftp function
paftpPramFile='pa_batch_get.cfg'
paftpcommand=GET
padataname=1global_daily_$lcDate
pafileExt=.tag_zip
paftptransmethod=ascii

#export the vars
export paftpcommand
export padataname
export pafileExt
export paftptransmethod

#call the ftp function
ftpfile

##get the second tag_zip file
padataname=2global_daily_$lcDate
export padataname
pafileExt=.tag_zip
export pafileExt
#call the ftp function
#ftpfile


##more more the lst file into the first tag_zip this wil give us the whole list
of zip file to ftp over

more ${palocal_check}/2global_daily.tag_zip >> ${palocal_check}/1global_
daily.tag_zip

##change the windows file to UNIX
copyWinToUnix



#We now need to bring down all the zip files listed within the /1global_daily.t
ag_zip file
ftpzipfiles

##This is where we do the searching and placing of the files from the dirs.txt .
...

##first set up the vars
printLog "PROCESSING THE FIRST FILE "
padataname=1global_daily
export padataname
pafileExt=.tag
export pafileExt
getfilesandcopy ${palocal_check}/${padataname}.tag

ANY IDEAS AS TO WHERE I SHOULD LOOK?

Just a quick look see througz your script...

1. Make sure that all of your comments are preceded with a # symbol.

2. You are calling a function or script called factset in this line factset $ more pa_batching.ksh, but this function or script is not defined in this script and it appears that the file (if its outside this script) was never sourced.

3. The 'CT' not found error looks to be caused by a line wrap. If this file was saved somehow with line wrapping then you will need to un-garble it.

4. The 'for' unmatched probably is related to the formatting as at a quick glance you have closed both the 'for' loop annd the 'while' loop. Next steps would be to check your quoting throughout the script.

Also, in your post, please put your code in between some code tags.. [code] code here[\/code]

I am trying to run a script and am getting the following
error pa_batching.ksh[24]: syntax error at line 25 : `{' unmatched
factset

The script I am running is as follows

#! /bin/ksh
#
# Filename: factset_transfer
#
# Author: Steven bowskill
#
# Description: This file is the main script for the PA BATCHING FactSet automation

#
#
#-------------------------------------------------------------------------------

#set -vx
exit_handler()
{
printLog "$1: $2"
exit $1
}

#FUNCTION TO SEARCH FOR THE CORRECT NAME OF THE FILE AND PLACE THEM IN THE CORRE
#CT DIRECTORY

function getfilesandcopy
{

##set up some vars
paftpPramFile='pa_batch_put.cfg'
paremote_node=lon-ks-util01
paremote_user=ftpwilshire
paremote_password=wilshire
paftpcommand=PUT
pafileExt=
paftptransmethod=binary
##export them now

export paremote_node
export paremote_user
export paremote_password
export paftpcommand
export pafileExt
export paftptransmethod
export paftpPramFile

#create the file of the xls file in then this file will also con
tain the accounts

awk -F']' '/]/ {print $2}' $1 > $palocal_check/factserepor
ts.txt

# now go thru this file lookgin for the account
for acct in `cat $palocal_check/factsereports.txt`
do
echo "Start to search **********************************
****************************"
done

#get the factset file
filetoDownload=`echo $acct | awk -F',' '{print $1}'
`

#get the xls file to download
XlsfiletoDownload=`echo $acct | awk -F',' '{print $
1}'`

filetoDownload=`echo $acct | awk -F',' '{print $2}'
`
filetoDownload=`echo $f
iletoDownload | awk -F'_' '{print $1"_"$2"_"$3"_"$4}'`



printLog "searching for ${filetoDownload}"

cut -d "/" -f2-10 $palocal_check/dirs.txt > $palocal_check/dirsfound.txt

cat $palocal_check/dirsfound
d.txt

##get the correct name of the file from the dirs.txt fi
le
## NewFileToCopy=`awk -F, '/
'$filetoDownload'/ {print $3}' $palocal_check/dirs.txt`

##echo $NewFileToCopy

DirectoryFound=`cat $palocal_check/dirsfound.txt`



#did we get a match
if [ ! -z ${DirectoryFound} ] ;
then

while read dirs
do
#now ch
#eck the file exists here
printLo
g xls file $XlsfiletoDownload
if [ -f ${XlsfiletoDownload} ] ; then


#first lets r
#ename the factset file


# cp ${palocal_check}/$XlsfiletoDownload ${palocal_check}/${filetoDownload}
.xls

##now need to ftp the file from sun42 to
the g drive

##set up the ftp vars

padataname=$XlsfiletoDownload
paremote_dir=$dirs
palocal_dir=$palocal_check

#export the vars
export padataname
export paremote_dir
export palocal_dir
# now ftp the file
printLog "Ftp ${filetoDownload} to ${dir
s} this is the factset file ${XlsfiletoDownload} "
ftpfile

#copy the xls file
cp ${palocal_check}/$XlsfiletoDownload ${p
alocal_check}/${filetoDownload}.xls
#now delete the xls file from factset this will
tell us what was not found
rm ${pa
local_check}/$XlsfiletoDownload


fi


done

I have merged the threads. Please don't start another thread on this subject. Post follow up question here using "reply". Thanks.

Here is a formatted version of your script. I made a
half-hearted attempt to look at your script. I noticed
that you still have code segments spanning multiple
lines - you cant do that. You also had an "if" statement
that was not closed. I fixed that. The function "function getfilesandcopy" does not have a closing brace,
furthermore, its never actually called! A fatal design flaw
to say the least! Lastly, you had a ";" after your "if"
test at line 75. I removed that. Now that there arent
any more syntax errors (that the shell complained about)
I would suggest you fix your design flaws!!!

Code:

#!/bin/ksh
#
# Filename: factset_transfer
#
# Author: Steven bowskill
#
# Description: This file is the main script for the PA BATCHING FactSet automation
#
# 
#-------------------------------------------------------------------------------
#set -vx
exit_handler()
{
  printLog "$1: $2"
  exit $1
}

#FUNCTION TO SEARCH FOR THE CORRECT NAME OF THE FILE AND PLACE THEM IN THE CORRECT
#DIRECTORY 

function getfilesandcopy
{

##set up some vars
paftpPramFile='pa_batch_put.cfg'
paremote_node=lon-ks-util01
paremote_user=ftpwilshire
paremote_password=wilshire
paftpcommand=PUT
pafileExt=
paftptransmethod=binary
##export them now

export paremote_node
export paremote_user
export paremote_password
export paftpcommand
export pafileExt
export paftptransmethod
export paftpPramFile

#create the file of the xls file in then this file will also contain the accounts

awk -F']' '/]/ {print $2}' $1 > $palocal_check/factsereports.txt

# now go thru this file lookgin for the account 
for acct in `cat $palocal_check/factsereports.txt`
do
  echo "Start to search **************************************************************"
done 

#get the factset file 
filetoDownload=`echo $acct | awk -F',' '{print $1}'`

#get the xls file to download
XlsfiletoDownload=`echo $acct | awk -F',' '{print $1}'`

filetoDownload=`echo $acct | awk -F',' '{print $2}'`
filetoDownload=`echo $filetoDownload | awk -F'_' '{print $1"_"$2"_"$3"_"$4}'`

printLog "searching for ${filetoDownload}" 

cut -d "/" -f2-10 $palocal_check/dirs.txt > $palocal_check/dirsfound.txt 

cat $palocal_check/dirsfoundd.txt

##get the correct name of the file from the dirs.txt file 
##NewFileToCopy=`awk -F, '/' $filetoDownload'/ {print $3}' $palocal_check/dirs.txt`
##echo $NewFileToCopy

DirectoryFound=`cat $palocal_check/dirsfound.txt`

#did we get a match 
if [ ! -z ${DirectoryFound} ] ; 
then
  while read dirs
  do
    #now ch
    #eck the file exists here 
    printLog xls file $XlsfiletoDownload
  if [ -f ${XlsfiletoDownload} ]
   then
     #first lets rename the factset file 
     # cp ${palocal_check}/$XlsfiletoDownload ${palocal_check}/${filetoDownload}.xls 
     ##now need to ftp the file from sun42 tothe g drive set up the ftp vars
     padataname=$XlsfiletoDownload
     paremote_dir=$dirs
     palocal_dir=$palocal_check
     #export the vars
     export padataname
     export paremote_dir
     export palocal_dir
     # now ftp the file
     printLog "Ftp ${filetoDownload} to ${dirs} this is the factset file ${XlsfiletoDownload} " 
     ftpfile

     #copy the xls file
     cp ${palocal_check}/$XlsfiletoDownload ${palocal_check}/${filetoDownload}.xls 
     
     #now delete the xls file from factset this will tell us what was not found 
     rm ${palocal_check}/$XlsfiletoDownload
  fi 
 done
fi
}

My script seems to be working fine but the out put seems to be hanging and only processes the first set of files an error also comes up to say awk :can't open /opt/lonsched/data/factset/pabatch/check/1global_daily.tag

ERROR MESSAGE FROM LOG FILE:

16:11:21> PROCESSING THE FIRST FILE
awk: can't open /opt/lonsched/data/factset/pabatch/check/1global_daily.tag
16:11:21> searching for ___
Daily/test/Ceurope/EuropeFactset/YTD/
Daily/test/Ceurope/EuropeFactset/
Daily/test/Ceurope/EuropeFactset/YTD/
Daily/test/Ceurope/EuropeFactset/

The script itself is as follows:

SCRIPT

Set up basic environment

#delete the log file we want a clean one
rm /opt/lonsched/log/pabatching.log

#set the log file up
logfile=/opt/lonsched/log/pabatching.log
[ -z ${logfile} ] && logfile=/dev/tty
export logfile

exec 1>> ${logfile}
exec 2>&1

printLog "Setting environment parameters .." date

. /opt/lonsched/scripts/factset/pabatching_params.ksh
let retstat=$?
if [ ${retstat} -ne 0 ] ; then
exit_handler 101 "Failed to set up environment - unknown mode setting"
fi



# Derive the data file name from the tag file name and make it
# available to runFTP.
#

#lcDate=`date +"%d%b%Y" | tr "[a-z]" "[A-Z]"`
lcDate="05OCT2004"

#set up the vars and cal the ftp function
paftpPramFile='pa_batch_get.cfg'
paftpcommand=GET
padataname=1global_daily_$lcDate
pafileExt=.tag_zip
paftptransmethod=ascii

#export the vars
export paftpcommand
export padataname
export pafileExt
export paftptransmethod

#call the ftp function
ftpfile

##get the second tag_zip file
padataname=2global_daily_$lcDate
export padataname
pafileExt=.tag_zip
export pafileExt
#call the ftp function
ftpfile


##more more the lst file into the first tag_zip this wil give us the whole list
#of zip file to ftp over

more ${palocal_check}/2global_daily.tag_zip >> ${palocal_check}/1global_daily.tag_zip


##change the windows file to UNIX
copyWinToUnix



#We now need to bring down all the zip files listed within the /1global_daily.t
#ag_zip file
ftpzipfiles

##This is where we do the searching and placing of the files from the dirs.txt .
#...

##first set up the vars
printLog "PROCESSING THE FIRST FILE "
padataname=1global_daily
export padataname
pafileExt=.tag
export pafileExt
getfilesandcopy ${palocal_check}/${padataname}.tag

##do the second file
printLog "PROCESSING THE SECOND FILE "
padataname=2global_daily
export padataname
pafileExt=.tag
export pafileExt
getfilesandcopy ${palocal_check}/${padataname}.tag

printLog "Finished Processing .." date

ITS HANGING AT PROCESSING FIRST FILE AND CANNOT PROCEED ANY IDEAS AS TO WHERE I SHOULD LOOK

Compare the syntax in the formatted post to this latest post. Correct the issues that I identified in my other posts. Most importantly, try debugging these scripts yourself. That is the only way you will learn.


To debug,

add this line to the top of your script, under the #!/bin/ksh (or similar) line.

set -x