A script to scan a directory for XML files,


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting A script to scan a directory for XML files,
# 1  
Old 12-17-2009
A script to scan a directory for XML files,

Hi,

I am fairly new to unix/linux scripting (about 1 week) and have written a script to scan a directory for xml files, if found call and oracle procedure passing in the file name and then move the file once processed to an archive area.

Now everything seems to be working except when there are no xml files in the directory.

I use the following :-

Code:
XMLFILES=`ls -rt $DIR_PATH/*.xml`

and use that in a for loop further down in the script

Code:
for f in $XMLFILES

I have two questions really... am I using the correct approach, I would ask collegues at work... but we don't have any other unix/linux type people at the moment that I can talk to. Secondly is there a way to check for the existence or maybe have an exception handler around the XMLFILES= line that I could use to exit the script?

Thanks in advance for the help
# 2  
Old 12-17-2009
Does the list of files have to be time-orderer? If not, you can use
Code:
find $DIR_PATH -type f -name '*.xml' -print | while read f

Otherwise, you can check if the variable is empty by using the -z test operator, eg:
Code:
if [ ! -z "$XMLFILES" ]; then
# processing loop here
fi

Side note: it's advisable to use $() instead of ``, as it's more readable and can be nested if needed.
# 3  
Old 12-17-2009
Quote:
Originally Posted by apacheuk
Code:
for f in $XMLFILES

Code:
shopt -s nullglob
for file in *.xml
do
 # ....
done

Code:
for file in *.xml
do
  if [ -f "$file" ];then
     # ........
  fi
done

# 4  
Old 12-18-2009
Yeah it does have to be in time order, thanks for the replies

Code:
if [ ! -z "$XMLFILES" ]; then
# processing loop here
fi



---------- Post updated 18th Dec 2009 at 10:50 AM ---------- Previous update was 17th Dec 2009 at 11:48 AM ----------

OK, so I've finished my script and it runs successfully Smilie

I'll be honest this is the first unix/linux script I've coded before, would it be possible for you kind people to give it the once over and give me some feedback??? It's very simplistic and I could probably have found a script on the net somewhere to do the same thing, but I find the best way to learn is too role your sleeves up and get stuck in

It's purpose is when called is to process all the xml files in a directory and then move them elsewhere based on the result of an Oracle stored procedure.

Thanks

Code:
#!/bin/sh
# setup some constants
#      DIR_PATH is the path to the xml files to be processed
#      PROCESSED_PATH is the path to xml files that have been successfully processed
#      ERRORED_PATH is the path to xml files that failed processing
DIR_PATH='/home/f450242/xml'
PROCESSED_PATH='/home/f450242/xml/processed'
ERRORED_PATH='/home/f450242/xml/errors'
# run timestamp
DT=`date`
echo " ***** BEGIN PROCESS ***** "
# first make sure our paths are all accessible
if [ ! -d $DIR_PATH ]
then
  echo "$DT:> $DIR_PATH cannot be found"
  exit
fi
# check to see if there are any xml files to process
# no point continuing if there arent any!
if [ test ! -e $DIR_PATH/*.xml ]
then
  echo "$DT:> no xml files found"
  exit
fi
if [ ! -d $PROCESSED_PATH ]
then
  echo "$DT:> $PROCESSED_PATH cannot be found"
  exit
fi
if [ ! -d $ERRORED_PATH ]
then
  echo "$DT:> $ERRORED_PATH cannot be found"
  exit
fi
# check to make sure we can connect to SQLPLUS
# exit and report if we can't
SQLUP=`sqlplus -s xxxxxx@dev/xxxxxxx<<EOF
       exit
       EOF`
# $? variable is used to hold the exit status of the previous command
# 0 success, anything else is an error
if [ $? -ne 0 ]
then
  echo "$DT:> SQLPLUS not available"
  exit
fi
# check for the existence of our lock_file
# if found it means the process is still running
# so we need to exit
if [ -f $DIR_PATH/lock_file ]
then
  echo "$DT:> lock_file found, exiting"
  exit
fi
# about to start our process so create our lock_file
touch lock_file
# store the list of files to process
XML_FILES=`ls -rt $DIR_PATH/*.xml`
# loop through each file and call our Oracle routine to process the file
for f in $XML_FILES
do
  SUCCESS=`sqlplus -s xxxxxx@dev/xxxxxxx << EOF
           set serveroutput on
           declare
             vout varchar2(10);
           begin
             test_proc(1,vout);
             dbms_output.put_line(vout);
           end;
           /
           commit;
           exit;
           EOF`
  CHECK=`echo $SUCCESS | awk '{print $1}'`
  if [ $CHECK = "TRUE" ]
  then
    mv $f $PROCESSED_PATH
  else
    mv $f $ERRORED_PATH
  fi
done
rm lock_file
echo " ***** END PROCESS ***** "

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Create automated scan of specific directory using bash

I am trying to use bash to automate the scan of a specific directory using clamav. Having this in place is a network requirement. The below is an attempt to: 1. count the extensions (.txt, .jpeg) in a directory and write them to a virus-scan.log (section in bold) 2. scan each folder in the... (6 Replies)
Discussion started by: cmccabe
6 Replies

2. Shell Programming and Scripting

Moving Files one directory to another directory shell script

Hi, Could you please assist how to move the gz files which are older than the 90 days from one folder to another folder ,before that it need to check the file system named "nfs" if size is less than 90 or not. If size is above 90 then it shouldn't perform file move and exit the script throwing... (4 Replies)
Discussion started by: venkat918
4 Replies

3. Shell Programming and Scripting

Copying xml files to a chosen directory

I want to determine if there's any xml files exist & if so copy each xml to that directory. Is my code correct for doing that? I can't test my script yet. Somebody please explain it to me please? if ]; then #print "No Status type XML files received from server in $DIRECTORY" else for... (2 Replies)
Discussion started by: emc^24sho
2 Replies

4. UNIX for Dummies Questions & Answers

Need help how to create a file (xml) list all files from directory

I have more than 10K songs in two directories on a hard drive. I would like to create a file list all of files name then change to .xml extension to upload to iPhone so I have a Karaoke list on my iPhone. I need your help to create a file by using command in Linux. Files names: 0001 More... (4 Replies)
Discussion started by: ggcc
4 Replies

5. Shell Programming and Scripting

Script which removes files from the first directory if there is a file in the second directory

Script must removes files from the first directory if there is a file with same name in the second directory Script passed to the two directories, it lies with them in one directory: sh script_name dir1 dir2 This is my version, but it does not work :wall: set - $2/* for i do set -... (6 Replies)
Discussion started by: SLAMUL
6 Replies

6. Shell Programming and Scripting

Generating an xml having information related to files in the directory

Hi all, Have to generate an xml having information related to files in the directory Suppose i have file file1.xml (datafile) file2.xml (datafile) file3.xml (metafile) Now i need to generate an xml in the format >> <?xml version="1.0" encoding="UTF-8"?> <AuditFile Version="2.0">... (8 Replies)
Discussion started by: abhinav192
8 Replies

7. Shell Programming and Scripting

Scan directory and sub directories

I am really stuck on a issue and have not been able to find a solution. With the code below I need to not only scan the current directory which as you can see below is... /lcl/sit/apps/Tivoli/ But I also want it to scan all sub directories it finds under Tivoli as well for the same thing... (2 Replies)
Discussion started by: LRoberts
2 Replies

8. Shell Programming and Scripting

scan directory

The script should _scan a specific directory _If a file name is like one provided, then run the command to send the file via CFT The name should be picked from a list. The current list is : ... (11 Replies)
Discussion started by: fireit
11 Replies

9. Shell Programming and Scripting

Script to Scan proclog files

i need to create a shell script, which will go into a directory , and scan the files in it for defined errors, there will be around 10 files in the directory. (3 Replies)
Discussion started by: deeprajn95
3 Replies

10. Shell Programming and Scripting

Perl script to scan through files

Dear perl gurus, I plan to create a script that will scan through a logfile line by line. And if ever a certain line meets the below conditions, it will alert me via email. --> a) Position 10 to 13 = "ABCD" b) And also if the amount specified in position 620-640 is less than the amount in... (1 Reply)
Discussion started by: gholdbhurg
1 Replies
Login or Register to Ask a Question