Script to pick the files from source folder

Vijay81 · #1 02-03-2012

Hi,

I wrote a script to pick files based on some logic. when I execute my script, I get the below message ( I bolded the lines in the script where this error message shows up), it says that files are not available and it is a valid scenario if the files are not available and I don't know how to get rid of this message.. I am guessing this is a warning not an error as my script continues after this message..can someone help me on how to get rid of this message?

Message:

ls: FERAL_DATA_*_20121213.FERAL_DAT.TXT: No such file or directory
ls: FERAL_CONTROL_20121213.FERAL_CTRL.TXT: No such file or directory

below is my script..as I am new to unix scripting, I have posted the script to get suggestions from experts.. like I may missed something.. or could have done it in a better way.. please help me!

PHP Code:


			
# pickdate_paramfile1.sh
###script to pick the files based on the date of run and generate a parameter file #####
#!/bin/sh
. /opt/informatica/infa_shared/SCRIPTS/env_file.ksh
date_str=`date "+%Y%m%d%H%M%S"`
ENVID=$1
logfile=$LOGS/${ENVID}_pickupdate_${date_str}.log
echo Script $0 started at `date` >${logfile}
#### Check if one parameter is received#######
if [ $# -ne 1 ]; then
echo "input parameter for the script is missing" >>${logfile} 2>&1
exit 1
fi
######Connect to DB to fetch the date of run#####
DATE_OF_RUN_DB=`sqlplus -s <<sqlEOF
$ORACLE_USERNAME/$ORACLE_PASSWORD@$ORACLE_DB
set feedback off
set heading off
set linesize 5000
set pagesize 0
set echo off
whenever sqlerror exit failure
whenever oserror exit failure
SELECT TO_CHAR(MAX(LAST_RUN_DATE) +1,'YYYYMMDD') FROM ENV_LOAD_HISTORY WHERE ENV_ID='$ENVID' AND STATUS_OF_RUN ='SUCCESS';
sqlEOF`
if [ $? -ne 0 ]; then
echo "error connecting to database" >>${logfile}
exit 1
fi
#######If DATE_OF_RUN_DB is empty throw message and abend the script
if  [ -z "$DATE_OF_RUN_DB" ]
then
echo "No date of run value fetched from DB" >>${logfile} 2>&1
exit 1
fi
echo "Date of run fetched from DB is - $DATE_OF_RUN_DB" >>${logfile}
####Get the files based on the date received from the DB#####
cd $SOURCE_FOLDER
Data_Files=`ls ${ENVID}_DATA_*_${DATE_OF_RUN_DB}.${ENVID}_DAT.TXT`
control_file=`ls ${ENVID}_CONTROL_${DATE_OF_RUN_DB}.${ENVID}_CTRL.TXT`
#####if control file is not available for that date then look for the files with the least date value in the landing folder and decide the date of run#####
if  [ -z "${control_file}" ]
then
echo "Control File is missing for date $DATE_OF_RUN_DB" >>${logfile} 2>&1
cd $SOURCE_FOLDER
Actual_Date=`ls ${ENVID}_CONTROL_*.${ENVID}_CTRL.TXT| sort -t"_" -k3.8 | head -1 | cut -d"_" -f3| cut -d"." -f1`
####pick files based on the date value
Data_Files_New=`ls $SOURCE_FOLDER/${ENVID}_DATA_*_${Actual_Date}.${ENVID}_DAT.TXT`
control_file_New=`ls $SOURCE_FOLDER/${ENVID}_CONTROL_${Actual_Date}.${ENVID}_CTRL.TXT`
##### if no files for that date then throw error message and notify via email & abend the script
if  [ -z "${Data_Files_New}" -o -z "${control_file_New}"  ]
then
echo "Files are missing for date $Actual_Date" >>${logfile} 2>&1
#### include email notification script#####
exit 1
fi
DATE_OF_RUN=${Actual_Date}
CONTROL_FILE_FINAL=${control_file_New}
echo "Files are $Data_Files_New $control_file_New" >>${logfile}
cp ${Data_Files_New} ${control_file_New} $WORK
if [ $? -ne 0 ]; then
echo "Files are not copied" >>${logfile}
exit 1
else echo "Files are copied to work directory" >>${logfile}
fi
else
DATE_OF_RUN=${DATE_OF_RUN_DB}
CONTROL_FILE_FINAL=${control_file}
echo "Files are $Data_Files $control_file" >>${logfile}
cp ${Data_Files} ${control_file} $WORK
if [ $? -ne 0 ]; then
echo "Files are not copied" >>${logfile}
exit 1
else
echo "Files are copied to work directory" >>${logfile}
fi
fi
echo "date of run = $DATE_OF_RUN" >>${logfile} 2>&1
echo "Control file name is $CONTROL_FILE_FINAL" >>${logfile} 2>&1
#######If DATE_OF_RUN is empty throw message and abend the script
if  [ -z "${DATE_OF_RUN}" ]
then
echo "No date of run value fetched unable to proceed" >>${logfile} 2>&1
exit 1
fi
######Create a workflow parameter file#####
parmfile=$PARM_FILES/${ENVID}_parm.parm
parmfile1=$PARM_FILES/${ENVID}_PARM_I.TXT
##### Check if <EOF> tag is available in control file####
if [ `tail -1 "$CONTROL_FILE_FINAL"` = "<EOF>" ]
then echo "<EOF> tag is available in the $CONTROL_FILE_FINAL" >>${logfile} 2>&1
#####copy contents from control file to parameter file####
touch ${parmfile}
cd $SOURCE_FOLDER
cp $CONTROL_FILE_FINAL $parmfile1
sed '$d' $parmfile1 > $parmfile
rm $parmfile1
echo "DATE_OF_RUN=$DATE_OF_RUN" >>${parmfile}
echo "ENVID=$ENVID" >>${parmfile}
echo "parameter file created is $parmfile"  >>${logfile}
else  echo "<EOF> tag missing in $CONTROL_FILE_FINAL" >>${logfile} 2>&1
exit 1
fi

ctsgnb · #2 02-05-2012

Did you set SOURCE_FOLDER variable in /opt/informatica/infa_shared/SCRIPTS/env_file.ksh ?

Could you please give us the output of :

Code:

env

Vijay81 · #3 02-06-2012

hi CTSGNB,

thanks for respnding.

below are the details you asked for:

env:

PHP Code:


			
_=/bin/env
G_BROKEN_FILENAMES=1
HISTSIZE=1000
HOME=/home/wertty
INPUTRC=/etc/inputrc
LANG=en_US.UTF-8
LESSOPEN=|/usr/bin/lesspipe.sh %s
LOGNAME=wertty
LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz=00;31:*.arj=00;31:*.taz=00;31:*.lzh=00;31:*.zip=00;31:*.z=00;31:*.Z=00;31:*.gz=00;31:*.bz2=00;31:*.bz=00;31:*.tz=00;31:*.rpm=00;31:*.cpio=00;31:*.jpg=00;35:*.gif=00;35:*.bmp=00;35:*.xbm=00;35:*.xpm=00;35:*.png=00;35:*.tif=00;35:
MAIL=/var/spool/mail/wertty
PATH=/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin
PWD=/opt/informatica/infa_shared/SrcFiles/logs
SHELL=/bin/ksh
SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass
SSH_TTY=/dev/pts/0
TERM=xterm
USER=wertty
_AST_FEATURES=UNIVERSE - ucb

cat /opt/informatica/infa_shared/SCRIPTS/env_file.ksh

PHP Code:


			
#!/usr/bin/ksh
# Define environment variables for running Informatica workflow
#--------------------------------------------------------------
export SOURCE_FOLDER=/opt/informatica/infa_shared/SrcFiles
export TARGET_FOLDER=/opt/informatica/infa_shared/TgtFiles
export PARM_FILES=/opt/informatica/infa_shared/PARMS
export CACHE=/opt/informatica/infa_shared/Cache
export LOGS=/opt/informatica/infa_shared/SrcFiles/logs
export SESSION_LOGS=/opt/informatica/infa_shared/SessLogs
export WORKFLOW_LOGS=/opt/informatica/infa_shared/WorkflowLogs
export ARCHIVE=/opt/informatica/infa_shared/SrcFiles/arch
export SCRIPT_LOGS=/opt/informatica/infa_shared/SrcFiles/script_logs
export TEMP=/opt/informatica/infa_shared/Temp
export SCRIPTS=/opt/informatica/infa_shared/SCRIPTS
export WORK=/opt/informatica/infa_shared/SrcFiles/work
#----------------------------------------------------------------
# Database credentials
#----------------------------------------------------------------
export ORACLE_HOME=
export ORACLE_DB=ERDW_DEV
export ORACLE_USERNAME=resdt
export ORACLE_PASSWORD=opiutn3

My database is ORACLE.

I am on LINUX.

Data_Files and control files variables can have no values.. it is not a must that they should have values.. means it a valid scenario not to have files in the directory.But at the same time, I want to get rid of the message that says
ls: FERAL_DATA_*_20121213.FERAL_DAT.TXT: No such file or directory
ls: FERAL_CONTROL_20121213.FERAL_CTRL.TXT: No such file or directory
thanks!

ctsgnb · #4 02-06-2012

Do this :

Code:

cd /opt/informatica/infa_shared/SrcFiles
ls FERAL_DATA_*_20121213.FERAL_DAT.TXT

Vijay81 · #5 02-06-2012

I get the same error message:

$ cd /opt/informatica/infa_shared/SrcFiles
$ ls FERAL_DATA_*_20121213.FERAL_DAT.TXT
ls: FERAL_DATA_*_20121213.FERAL_DAT.TXT: No such file or directory

I am not sure if I got your question about the command line... as it is a .sh script, I execute it like- ./scriptname.sh parameter

thanks!

Corona688 · #6 02-06-2012

It means what it says -- there's no files matching that in the current directory. What files do you have, or where?

methyl · #7 02-06-2012

Quote:

I execute it like- ./scriptname.sh parameter

And the "parameter" is ... ?

Thread Tools	Search this Thread
Show Printable Version Email this Page Subscribe to this Thread	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
Script for moving files from one folder to other	lalitkumar	Shell Programming and Scripting	8	12-15-2008 06:32 AM
Script for moving files from one folder to other	lalitkumar	Shell Programming and Scripting	2	11-26-2008 07:27 AM
script for Finding files in a folder and copying to another folder	satish2712	Shell Programming and Scripting	5	09-11-2008 04:07 PM
Script to pick out files from a file which are older than 15 days	viv1	Shell Programming and Scripting	1	04-02-2008 09:42 AM
Suppres error message when moving files from empty source folder	Steven	Shell Programming and Scripting	2	11-19-2001 12:25 PM