Shell Script to poll files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Shell Script to poll files
# 1  
Old 02-23-2012
Shell Script to poll files

Hello,

I need to write a script for my prod servers, here is the requirement.

Code:
1. Shell script should poll the directory  everyday at 3pm for 2 files a.csv and a.dat. It should poll the  directory for the files for only 15 mins after that it should error out  saying that no files received.
2. After receving the first file ( the order of the file doesnt  matter) it should wait for max 5 mins. if second file not recieved then  throw an error saying that the other file is not received.
3. Only after both the files are received then the shell script  should copy the a.csv file to another directory and wait for 1 min
4.  After 1 MIN copy a.dat to another directory and after successful copy  archive the files with a current timestamp to another directory



if you need any further details then let me know.

-Siddhesh
# 2  
Old 02-23-2012
what you tried so far?
# 3  
Old 02-23-2012
Hello,

Sorry for the late response.This is what i have tried so far.

Code:
#!/bin/bash

#########################################################################################################
# Shellscript   :       codecheck_trans.sh -Status Code Check
# Version       :       V1.0
# Owner         :       Siddhesh Khavnekar <siddhesh.khavnekar@mobixell.com>
# Date          :       2012-02-21
# Category      :       Files Management
#########################################################################################################

#Defining Variable
#------------------------
DATE=`date +%Y%m%d%H%M%S`
SRC_DIR=/home/xmp/archive/logmgmt/result/billing
DEST_DIR=/storage3/archive/logmgmt/result/billing
HOST=`hostname`
FILE_CSV=`find $SRC_DIR -type f abc.exe| wc -l`
FILE_DAT=`find $SRC_DIR -type f xyz.bat| wc -l`

if [ $FILE_CSV -eq 1 ]; then
echo " CSV file found on $HOST !!! "
else
echo " CSV file not found "

if [ $FILE_BAT -eq 1 ]; then
echo " BAT file found on $HOST !!! "
else
echo " BAT file not found "

exit 0

This will search for file and echo. Not sure how do i loop it for search for 5 times and send a echo message if the files are not found.

Regards,
Siddhesh

---------- Post updated at 12:39 PM ---------- Previous update was at 11:50 AM ----------

Hello Image itkamaraj,

Please advice. its a bit urgent this for me. Thanks in advance.

-Siddhesh
# 4  
Old 02-23-2012
Hi, use while for the loop and use sleep for the pauses.
# 5  
Old 02-23-2012
something like this..

but not the complete one. Not tested

Code:
#!/usr/bin/bash
i=0
while [ "$i" -lt "900" ]                                                                                                                                         
do
 echo "$i"
 i=$[$i+1]
 if [ "$i" -eq "900" ]
 then
 #check the $csv and $dat here also...
  break
 else
  [ -f a.csv ] && csv="Found"
  [ -f a.dat ] && dat="Found"
  if [[ "$csv" -eq "Found" -a "$dat" -eq "Found" ]]
  then
   #Do your copy work
   break;
  else
   sleep 1
  fi
 fi
done

# 6  
Old 02-23-2012
Quote:
Originally Posted by Siddheshk
Hello,

Sorry for the late response.This is what i have tried so far.

Code:
#!/bin/bash
 
#########################################################################################################
# Shellscript   :       codecheck_trans.sh -Status Code Check
# Version       :       V1.0
# Owner         :       Siddhesh Khavnekar <siddhesh.khavnekar@mobixell.com>
# Date          :       2012-02-21
# Category      :       Files Management
#########################################################################################################
 
#Defining Variable
#------------------------
DATE=`date +%Y%m%d%H%M%S`
SRC_DIR=/home/xmp/archive/logmgmt/result/billing
DEST_DIR=/storage3/archive/logmgmt/result/billing
HOST=`hostname`
FILE_CSV=`find $SRC_DIR -type f abc.exe| wc -l`
FILE_DAT=`find $SRC_DIR -type f xyz.bat| wc -l`
 
if [ $FILE_CSV -eq 1 ]; then
echo " CSV file found on $HOST !!! "
else
echo " CSV file not found "
 
if [ $FILE_BAT -eq 1 ]; then
echo " BAT file found on $HOST !!! "
else
echo " BAT file not found "
 
exit 0

This will search for file and echo. Not sure how do i loop it for search for 5 times and send a echo message if the files are not found.

Regards,
Siddhesh

---------- Post updated at 12:39 PM ---------- Previous update was at 11:50 AM ----------

Hello itkamaraj,

Please advice. its a bit urgent this for me. Thanks in advance.

-Siddhesh
You can put the script in Crontab for scheduling it run at 3 AM or PM

As for the code, try this: This might help.

Code:
#!/bin/bash
 
#########################################################################################################
# Shellscript   :       codecheck_trans.sh -Status Code Check
# Version       :       V1.0
# Owner         :       Siddhesh Khavnekar <siddhesh.khavnekar@mobixell.com>
# Date          :       2012-02-21
# Category      :       Files Management
#########################################################################################################
 
#Defining Variable
#------------------------
DATE=`date +%Y%m%d%H%M%S`
SRC_DIR=/home/xmp/archive/logmgmt/result/billing
DEST_DIR=/storage3/archive/logmgmt/result/billing
HOST=`hostname`
FILE_CSV="${SRC_DIR}/abc.exe"
FILE_DAT="${SRC_DIR}/xyz.bat"
NSLEEP=900   # 15 * 60 = 900 seconds
PSLEEP=300   # 5 * 60 = 300 seconds
SLGAP=60      # 60 Seconds Sleep. You can modify it but make sure u keep the factor of 900 and 300. 
 
NCTR=$((NSLEEP/SLGAP))    # 900/60 = 15
PCTR=$((PSLEEP/SLGAP))    # 300/5 = 5
 
FOUND_CSV=0
FOUND_DAT=0
 
CTR=0
while [ 1 ]; do
    CTR=$((CTR+1))
    if [[ ! -f ${FILE_CSV} !! ! -f ${FILE_DAT} ]]; then ## Either of them do not exist
      echo " Files not found on $HOST"
      if [[ ${CTR} -lt ${NCTR} ]]; then
          sleep $SLGAP
          continue
      else
          echo "Error: Files not found after $NSLEEP Seconds."
          exit 1
    else
          echo "File Found"
          [[ -f ${FILE_CSV} ]] && FOUND_CSV=1
          [[ -f ${FILE_DAT} ]] && FOUND_DAT=1
          break
    fi
done
 
 
SRCH_FILE=''
 
[[ ${FOUND_CSV} -eq 0 ]] && SRCH_FILE=${FILE_DAT}
[[ ${FOUND_DAT} -eq 0 ]] && SRCH_FILE=${FILE_CSV}
 
CTR=0
 
 
if [[ ! -z ${SRCH_FILE} ]]; then
    while [ 1 ]; do
 
      CTR=$((CTR+1))
      if [[ ${CTR} -lt ${PCTR} ]]; then
        if [[ ! -f  ${SRCH_FILE} ]]; then
            echo "${SRCH_FILE} not found"
            sleep $SLGAP
            continue
        fi
        echo "ERROR: ${SRCH_FILE} not found"
        exit 1
      fi
  done
else
   echo "Both files are found"
   ## {{{ DO YOUR STUFFS BELOW WITH FOUND FILE }}} 
fi
 
exit 0

# 7  
Old 02-23-2012
Thanks. Will test the same and get back to you.

-Siddhesh
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Find and removing the old files and zipping the files using shell script

Hi, I am trying to removing the old files which were older than 10 days and same g zipping the files using the shell script. script was return as follows. find /jboss7_homes/JBOSS7/SKYLIV??/SKYLIV??_CRM/jboss-eap-7.0/standalone/log -mtime +10 -type f | xargs rm -f find /cer_skyliv??/log... (6 Replies)
Discussion started by: venkat918
6 Replies

2. Shell Programming and Scripting

Shell script for field wise record count for different Files .csv files

Hi, Very good wishes to all! Please help to provide the shell script for generating the record counts in filed wise from the .csv file My question: Source file: Field1 Field2 Field3 abc 12f sLm 1234 hjd 12d Hyd 34 Chn My target file should generate the .csv file with the... (14 Replies)
Discussion started by: Kirands
14 Replies

3. Shell Programming and Scripting

How to create or convert to pdf files from csv files using shell script?

Hi, Can anyone help me how to convert a .csv file to a .pdf file using shell script Thanks (2 Replies)
Discussion started by: ssk250
2 Replies

4. Shell Programming and Scripting

Read files in shell script code and run a C program on those files

HI, I am trying to implement a simple shell script program that does not make use of ls or find commands as they are quite expensive on very large sets of files. So, I am trying to generate the file list myself. What I am trying to do is this: 1. Generate a file name using shell script, for... (2 Replies)
Discussion started by: shoaibjameel123
2 Replies

5. Shell Programming and Scripting

Shell script to poll a directory and stop upon an event

Need shell script to: 1/keep polling a directory "receive_dir" irrespective of having files or no files in it. 2/move the files over to another directory "send_dir". 3/the script should only stop polling upon a file "stopfile" get moved to "receive_dir". Thanks !! My script: until do... (0 Replies)
Discussion started by: iaav
0 Replies

6. Shell Programming and Scripting

need a shell script to extract the files from source file and check whether those files existonserve

Hi, I am new to shell scripting.Please help me on this.I am using solaris 10 OS and shell i am using is # echo $0 -sh My requirement is i have source file say makefile.I need to extract files with extensions (.c |.cxx |.h |.hxx |.sc) from the makefile.after doing so i need to check whether... (13 Replies)
Discussion started by: muraliinfy04
13 Replies

7. Shell Programming and Scripting

Script to Poll Directory and Copy files

Hi all, I'm looking for a script to poll a specified directory and copy new files to another location. The script should only copy new files so, I based on mtime I guess? Can anyone point me in the right direction of a script which could do this? My scripting skills aren't too bad, but... (1 Reply)
Discussion started by: JayC89
1 Replies

8. Shell Programming and Scripting

how to poll for new files?

Hi , i have a requirement in which i have to ftp files to unix from windows and vice versa. I have to encrypt files in windows which will then be decrypted in unix and vice versa. Now the process needs to be automated ..therefore when windows server or unix server recieves the files a shell... (5 Replies)
Discussion started by: lifzgud
5 Replies

9. UNIX for Dummies Questions & Answers

SFTP script - poll every min to check file complete before transfering

Hello, Before I do a GET remote file, I need to ensure the remote file is a complete file i.e. whatever process is saving the file to the remote folder should complete the transfer before I go GET it through my script. So I'm thinking I need to poll the remote file every minute or so to... (4 Replies)
Discussion started by: srineel
4 Replies
Login or Register to Ask a Question