I am writing a script to automate FTP files from AIX to Mainframe. ( Binary mode, passive connection)
FTP the currently available files(poll a directory to find the list) and put it to mainframe in one connection instance
Verify if all the files are copied to Mainframe successfully
If the FTP command fails due to some issue, try to resend the files and if the issue persist after 5 attempts to resend, inform particular user by email.
If the retry succeeds(send only those files that was not sent., this is in case of partially succeeded transfer; say 5 files , 2 transferred and command failed due to some issue and other 3 not sent... in that case send only those 3 files).
Now again verification of file(s) transferred to mainframe
Below is the script that I have written so far
Issue/Doubt/Question/Help required:
Not able to do FTP with out giving password. Please let me know how to use .netrc here. Also please suggest any good alternative approach
I don't have any idea about verifying the transferred file to Mainframe
Requesting for help / guidance;
Pseudo code/ outline:
Code:
#! /usr/bin/ksh
########################################
#-- Defining Variable --#
########################################
DATE=`date +%Y%m%d`
POLL_DIR=/home/abcd/processing
TMP_DIR=/home/abcd/processing/tmpdir
LOG_DIR=/home/abcd/processing/log
KSH_REC=$POLL_DIR/`basename $0`"_rec" ##-- Recovery file for the script
# This will have current state of the script and will be used at while
# restarting the script after failure(before failing the job updated the rec file)
HOST=`hostname`
LOCK_FILE=`basename $0`_$DATE.lock
STOPPER_FILE=`basename $0`_$DATE.stop
MASTER_LOG_FILE=$LOG_DIR/ftp_to_mvs_$DATE.log
PROCESSED_FILES=$TMP_DIR/processed_files_$DATE.dat
SOURCE_SERVER=`hostname`
REMOTE_SERVER='remoteserver.abcd.com'
REMOTE_PORT=12345 ##-- Dummy, not sharing the details here in open forum.
REMOTE_USER=userid
REMOTE_USER_PASS=userpasswd
POLL_COUNTER=0
########################################
#-- Defining Functions --#
########################################
##--------------------------------------
#-- (1) function FTP to MVS
##--------------------------------------
f_FTP_MVS()
{
echo 'user $REMOTE_USER $REMOTE_USER_PASS' > $TMP_BATCH_FTP
echo 'type binary' >> $TMP_BATCH_FTP
echo 'passive' >> $TMP_BATCH_FTP
cat $TMP_BATCH_FILE >> $TMP_BATCH_FTP
echo 'bye' >> $TMP_BATCH_FTP
ftp -sinv $REMOTE_SERVER $REMOTE_PORT << `cat $TMP_BATCH_FTP`
}
##------------------------------------------------------
#-- (2) function to verify files transferred to MVS
##------------------------------------------------------
f_VERIFY_FTP_MVS()
{
##Use nlist or something to verify the transfer
}
if [[ ! -f $
[[ -f $PROCESSED_FILES ]] && rm -f $PROCESSED_FILES; touch $PROCESSED_FILES || touch $PROCESSED_FILES; ## Remove and touch if it exist
## on first run; if not touch it. This is one time activity
while [ 1 ] ### Future enhancement --> Include a parameter for Date and compare with system time,
# script should execute only for current day 00:00:01 hrs to 23:59:59 hrs else graceful exit;
# include STOPPER FILEs for convenient stoping by support team
do
DATE_TS=`date +%Y%m%d%H%M%S`
** TMP_LOG_FILE=$LOG_DIR/ld_rdy_$DATE_TS.log #--- Remove this file when processed
** TMP_FILE_ld_rdy=$TMP_DIR/ld_rdy_$DATE_TS.dat #--- Remove this file when processed
** TMP_BATCH_FILE=$TMP_DIR/ftp_batch_$DATE_TS.dat #--- Remove this file when processed
** TMP_BATCH_FTP=$TMP_DIR/ld_rdy_$DATE_TS.ftp
FTP_RETRY_COUNTER=0
if [ `ls -lrt $POLL_DIR/*.ld_rdy | awk 'END {print NR}'` -gt 0 ] ##** Change the condition with 1. LS and 2. grep "not found"
then
### Creating the list of currently available load ready file(s) to process(which is to be FTP'ed to MVS
ls -lrt $POLL_DIR/*.ld_rdy | awk '{print $NF}' > $TMP_FILE_ld_rdy
awk '{print "put", $0}' $TMP_FILE_ld_rdy | sed 's/.ld_rdy//g' > $TMP_BATCH_FILE
###########################
### FTP To MVS Part ###
###########################
echo "POLL COUNTER\t:\t"$POLL_COUNTER > $TMP_LOG_FILE
echo "DATE TIME \t:\t"$DATE_TS >> $TMP_LOG_FILE
echo "Trying to FTP below file(s) to Mainframe: "$TARGET_SERVER >> $TMP_LOG_FILE
sed 's/.ld_rdy//g' $TMP_FILE_ld_rdy >> $TMP_LOG_FILE
f_FTP_MVS
### Check for success or failure of FTP command
if [[ $? -ne 0 ]]
then
### Statements following failed FTPs command
### Step 1: Update the log for failure and
echo "\nFailure in transfering file to targer server" >> $TMP_LOG_FILE
FTP_RETRY_COUNTER=$(($FTP_RETRY_COUNTER+1))
sleep 10;
### Step 2: Check if FTP connection can be established
## if not send an email to Support(if Counter is more than 2) stating there is FTP connection issue.
## ----- Create a Function and call it here.
### Step 3: If connection can be established then try re-sending the unprocessed file(S). -- Write a Function for this
## i. Create Processed and unprocessed file list
## ii. Try re-sending the unprocessed file
## iii. If succeeds., update the log and come out of this loop and start Polling
## iv. If fails, Update FTP_RETRY_COUNTER to ++1
## and Call the STEP 2 - Connection check Function if
## FTP_RETRY_COUNTER is less than or equal to 2
## v. If FTP failed again, send a note to support to check on the FTP connection
### Step 4: Verify the files transferred to MVS
echo "exit 1 ||
echo "POLL COUNTER\t:\t"$POLL_COUNTER > $TMP_LOG_FILE
echo "DATE TIME \t:\t"$DATE_TS >> $TMP_LOG_FILE
echo "File(s) not available in the polling directory at this moment.\nPOLLING DIR\t:\t"$POLL_DIR"\n" >> $TMP_LOG_FILE
else
echo "Below files were successfully transferred to target server"; `cat TMP_FILE_ld_rdy | sed 's/.ld_rdy//g'cd`
fi
else
echo "POLL COUNTER\t:\t"$POLL_COUNTER > $TMP_LOG_FILE
echo "DATE TIME \t:\t"$DATE_TS >> $TMP_LOG_FILE
echo "File(s) not available in the polling directory at this moment.\nPOLLING DIR\t:\t"$POLL_DIR"\n" >> $TMP_LOG_FILE
fi
done
Last edited by rbatte1; 12-21-2017 at 09:27 AM..
Reason: Converted text lists to formatted lists with LIST=1 tags, corrected spelling and split long lines to make it more readable
Hi
when i am trying to ftp to a newly cretaed mainframe GDG for the first time,i am getting the following error...
Verbose mode on.
200 Representation type is Ebcdic NonPrint
200 Port request OK.
550-SVC99 Return code=4 S99INFO=0 S99ERROR=38668 HEX=970C S99ERSN code X'00004379').
550... (1 Reply)
hi
suppose i have a file named xyz(-1) and i have to transfer(ftp) it on a Mainframe from unix,how should i do it as whenever i try to do so it says use MVS naming conventions (1 Reply)
suppose i have a file named xyz(-1) and i have to transfer(ftp) it on a Mainframe from unix,how should i do it as whenever i try to do so it says use MVS naming conventions (1 Reply)
Hi All,
I am new to this site, I have a requirement where in i have to FTP a file from mainframe to Unix box.
The catch here is there are few Spanish characters like N with tilde(~) and a with ` etc., all other characters are coming fine but those mentioned above are not coming in a proper... (1 Reply)
Hi Experts...
Greetings for the day..!
I just want to FTP the files to mainframe system..
my code is not working..and also i need to put the files in a particular directory in a specific naming format...
ftp -i -n ${HOST_NAME} << END_FTP
user ${USER_NAME} ${PASSWORD}
put ${FILE_NAME}... (3 Replies)
Hi,
I'm not sure if this has been solved in this forum already but please do help me out if possible. Basically, I've already setup a passwordless SSH connection between 2 AIX IDs (say ID-1 and ID-2) with a Mainframe server ID (say MVSID). I'm able to successfully do an SSH from the AIX server to... (1 Reply)
Hi All,
The issue is that, we have a dataset in mianframe whose record length is 153 characters. And a batch job ftpies it to the unix server(SunOS) as a test file. But the ftpied file in unix does not have a record length of 153 chars. Each record of 153 chars gets splited into two line of... (8 Replies)
Hi,
I have a task of setting up connectivity between Oracle 10g (AIX) and Mainframe (1 library). Went through couple of documents, forums, blogs etc. MY understanding is ODBC Generic Connectivity is free from Oracle side.
Question: (may be dumb to you)
1. Has anybody done this and would... (3 Replies)
This discussion thread is an extension to what was discussed in Shell scripting section.
Please refer the post for the requirement:
Requirement Post - Click Here
The whole thread - Click Here
I would like to know how I can use NDM to transfer file from AIX to Mainframe and to verify the... (3 Replies)