Script to perform some actions on multiple files


 
Thread Tools Search this Thread
Operating Systems AIX Script to perform some actions on multiple files
# 1  
Old 06-10-2006
Question Script to perform some actions on multiple files

I have this Korn script that I wrote (with some help) that is run by cron. I basically watches a file system for a specific filename to be uploaded (via FTP), checks to make sure that the file is no longer being uploaded (by checking the files size), then runs a series of other scripts. The script is in a never ending loop that is "throttled" so that it when it is finding matching files its loop is shorter, but once it stops finding mathing files (in the monitored file system) it slow down until it finds another matching file.

Everything works great except that now I need to have the script monitor the same filesystem for two filenames. Actually they are the same filename but sometimes they are in ALL CAPS and sometimes they are not (for example I might get an upload.0002 or an UPLOAD.0002). Here is the script:

Code:
#!/bin/ksh
#
# Function : is_file_arrived file
# Arg(s)   : file = file to verify
# Output   : None
# Status   : 0 = yes file arrived, 1 = no
# Env.     : IFA_WAIT : interval (secs) for file size check (def=5)
#
 
is_file_arrived() {
   [ -z "$1" ] && return 1
   local file=$1
   local arrived=1
   local size1 size2
   if [ -f "$file" -a -z "$(fuser $file 2> /dev/null)" ] ; then
      size1=$(ls -l $file 2>/dev/null | awk '{print $5}')
      sleep ${IFA_WAIT:-15}
      size2=$(ls -l $file 2>/dev/null | awk '{print $5}')
      [ ${size1:-1} -eq ${size2:-2} ] && arrived=0
   fi
   return $arrived
}
 
 
processFile ()
{
   local fileName=$1
   local fileExtension=$2
   local fileNewName="/datafiles/str${fileExtension}.asc"
   local filePrintPath="/print"
   local fileTmpPath="/tmp"
   local fileODIName="str${fileExtension}.pos"
   mv -Eignore $fileName $fileNewName      # Add status check
   prepup $fileNewName $fileExtension
   mv -Eignore  $filePrintPath/$fileODIName $fileTmpPath/$fileODIName
   save2tmp $fileExtension
   call_siu $fileExtension
}
 
# Main Processing
 
nsec=1
#hile :
while [[ "$(date +%H%M)" -lt 2329 ]]
do
   for fileName in /datafiles/UPLOAD.[0-9][0-9][0-9][0-9]
   do
      fileExtension=${fileName#*.}
      is_file_arrived "$fileName" && nsec=1 && processFile $fileName $fileExtens
ion
   done
   sleep $nsec
   case $nsec in
      1)   nsec=15;;
      15)  nsec=45;;
      45)  nsec=90;;
      90)  nsec=300;;
      300) nsec=600;;
      600) nsec=900;;
      *)   nsec=1800;;
   esac
done

...how can I add additional filenames to the expression:

Code:
for fileName in /datafiles/UPLOAD.[0-9][0-9][0-9][0-9]

# 2  
Old 06-12-2006
Quote:
Originally Posted by heprox
...how can I add additional filenames to the expression:

Code:
for fileName in /datafiles/UPLOAD.[0-9][0-9][0-9][0-9]

Very simple. Depends on what you want:
- Every file in /datafiles:
Code:
for fileName in /datafiles/*; do
   whatever_you_want
done

- Specific files inside /datafiles:
Code:
for fileName in /datafiles/file1 /datafiles/file2 /datafiles/file3; do
   whatever_you_want
done

If you only want to ignore case, use:

/datafiles/[Uu][Pp][Ll][Oo][Aa][Dd].[0-9][0-9][0-9][0-9]


Regards.
# 3  
Old 06-12-2006
The syntax for:

Code:
/datafiles/[Uu][Pp][Ll][Oo][Aa][Dd].[0-9][0-9][0-9][0-9]

...was exactly what I was looking for, thanks. Worked like a charm.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk script to perform an action similar to vlookup between two csv files in UNIX

Hi, I am new to awk/unix and am trying to put together an awk script to perform an action similar to vlookup between the two csv files. Here are the contents of the two files: File 1: Date,ParentID,Number,Area,Volume,Dimensions 2014-01-01,ABC,247,83430.33,857.84,8110.76... (9 Replies)
Discussion started by: Prit Siv
9 Replies

2. Shell Programming and Scripting

Shell script to perform appending of .sql files

Hi, We are having multiple .sql files generated programatically which is not containing.. create or replace -- at the start and / -- at the end We need to append those .sql files with the above 2 statements at their respective positions. We will be really thankful to get responses... (6 Replies)
Discussion started by: Rami Reddy
6 Replies

3. Shell Programming and Scripting

perform actions at specific locations in data frame

Hi everyone, I got a data frame like the one below and and would like to do the following: Ignore the first 3 rows and check in all following rows the second position. If the value is >500, subtract 100. Example DF: ABC 22 DE 12 BCD 223 GH 12 EFG 2104 DH ... (4 Replies)
Discussion started by: TuAd
4 Replies

4. Shell Programming and Scripting

perform 3 awk commands to multiple files in multiple directories

Hi, I have a directory /home/datasets/ which contains a bunch (720) of subdirectories called hour_1/ hour_2/ etc..etc.. in each of these there is a single text file called (hour_1.txt in hour_1/ , hour_2.txt for hour_2/ etc..etc..) and i would like to do some text processing in them. Each of... (20 Replies)
Discussion started by: amarn
20 Replies

5. Shell Programming and Scripting

shell script to take input from a text file and perform check on each servers and copy files

HI all, I want to script where all the server names will be in a text file like server1 server2 server3 . and the script should take servernames from a text file and perform copy of files if the files are not present on those servers.after which it should take next servername till the end of... (0 Replies)
Discussion started by: joseph.dmello
0 Replies

6. UNIX for Advanced & Expert Users

awk - remove block of text, multiple actions for 'if', inline edit

I'm having a couple of issues. I'm trying to edit a nagios config and remove a host definition if a certain "host_name" is found. My thought is I would find host definition block containing the host_name I'm looking for and output the line numbers for the first and last lines. Using set, I will... (9 Replies)
Discussion started by: mglenney
9 Replies

7. Shell Programming and Scripting

Sed or awk script to remove text / or perform calculations from large CSV files

I have a large CSV files (e.g. 2 million records) and am hoping to do one of two things. I have been trying to use awk and sed but am a newbie and can't figure out how to get it to work. Any help you could offer would be greatly appreciated - I'm stuck trying to remove the colon and wildcards in... (6 Replies)
Discussion started by: metronomadic
6 Replies

8. Shell Programming and Scripting

Perform a set of actions for a specific file type

Hello, I have a problem that I'm having quite a bit of trouble with. I am trying to create a script that performs a specific sequence of actions for a file of a specific type. This is an abbreviated version of my basic script: #!/bin/sh #coulombic calculations... (2 Replies)
Discussion started by: oehtus
2 Replies

9. Shell Programming and Scripting

Need help on Mulitple files mutliple actions

Hi all, I have mistkanely gzipped twice an entire folder and sub folders, and also renamed the files during that process. I am trying to undo this, and I need help to create the batch to work on it. All folders are under my images directory, I have a output.txt file that holds all the... (1 Reply)
Discussion started by: saariko
1 Replies

10. UNIX for Dummies Questions & Answers

Perform a command to multiple files

How do I perform a command to multiple files? For example, I want to look at all files in a directory and print the ones that do not contain a certain string. How do I go about doing this? (4 Replies)
Discussion started by: mcgrawa
4 Replies
Login or Register to Ask a Question