Shell Programming and Scripting

At work we have a script that pulls a file from a list of remote sites every 10 min. Currently it only does this ftp one sites at a time and this process takes up to 8 minutes to complete. I want to have this list of 24 sites and be able to run the ftp from up to 2 sites on the list at a time. I don't want to do it in chunks of 2 but a free flowing limit of 2. I am just at a mental block right now and looking for a push to help me figure this out. This is a ksh script on an AIX machine. So an example of what is currently being done is below. This will run the cust1 and then when finished go and do cust2. I would like to run cust1 and then while that is running start cust2 and then wait until either 1 or 2 is finished and then start cust3......and so on.

#!/usr/bin/ksh

SAMPLE ()
{
cd $HOME_DIR
rm $SITE >/dev/null 2>/dev/null
ftp -i -n -v $IP >>$FTP_DIR/ftp_log 2>>$FTP_DIR/ftp_log <<HERE
user $CUST $PASSWD
cd $SITE_DIR
bin
get ${SITE}
bye
HERE
}

#CUST1
SITE=home1
IP=192.168.1.1
CUST=Guy1
PASSWD=12345
SAMPLE

#CUST2
SITE=home2
IP=192.168.1.2
CUST=Guy2
PASSWD=12345
SAMPLE

#CUST3
SITE=home3
IP=192.168.1.3
CUST=Guy3
PASSWD=12345
SAMPLE


I was thinking that I could make a counter that +/- when a ftp is started and finished and a loop that would check this and keep going unti lmy list has been completed. I think the part that is confusing me the most is how to get a ksh script to run this SAMPLE function 2 times and once completed start the next on the list. Any ideas or suggestions to help me figure this out?

Here is a code I did that regulates the flow of programs running to 4

You should be able to modify as you need

Code:

while read LINE                          # go through the output file from the view one line at a time
do

     export SRC_FILE_ID=$(echo $LINE | cut -f 1 -d \|)
     export SRC_FILE_TYPE=$(echo $LINE | cut -f 2 -d \|)
     export INBOUND_DIR=$(echo $LINE | cut -f 3 -d \|)
     export FILENAME=$(echo $LINE | cut -f 4 -d \|)
     export XTB_TB_NAME=$(echo $LINE | cut -f 5 -d \|)
     export REGEXP=$(echo $LINE | cut -f 6 -d \|)



     echo_f ""
     echo_f 'SRC_FILE_ID:      ' $SRC_FILE_ID
     echo_f 'SRC_FILE_TYPE:    ' $SRC_FILE_TYPE
     echo_f 'INBOUND_DIR:      ' $INBOUND_DIR
     echo_f 'FILENAME:         ' $FILENAME
     echo_f 'XTB_TB_NAME:      ' $XTB_TB_NAME
     echo_f 'REGEXP:           ' "$REGEXP"


    if [[ ! -r $INBOUND_DIR/$FILENAME ]]  # if the file does not exist, go to next one
    then
       echo_f $INBOUND_DIR/$FILENAME not found at $(date)
       echo_f skipping this processing
       continue;
    fi

   case $(echo $SRC_FILE_TYPE | cut -c -1) in
         M)    export PACKAGE="PKG_DW_CLM_LOAD";;
         P)    export PACKAGE="PKG_DW_CLMRX_LOAD";;
   esac


    # getting here, we want to submit the job to run, up to the limit of 4 jobs running at one time
    # See if there are 4 jobs running right now
      while [[ $( jobs | wc -l ) -ge 4 ]]
      do
        echo_f going to sleep now at $(date)
        sleep 30
      done

      /avidyn/scripts/avd_runit_002.ksh 1>/dev/null 2>&1 &   # parameters are passed by the export statements

done < /avidyn/tmp/avd_runit_001_jobs_to_run.txt