Running batches of files at a time from a script


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Running batches of files at a time from a script
# 1  
Old 05-06-2010
Running batches of files at a time from a script

Hi

I have a script that performs a process on a file.

I want to know how to include a function to run a batch of files?

Here is my script

Code:
 
#!/bin/bash
#---------------------------------------------------------------------------------------------------------------------- 
#This script allows the user to import a dsx file 
#
# Usage:
#./import.sh -s -d -r -o -f
#
#-----------------------------------------------------------------------------------------------------------------------
dbname="$dbname"
dsn="$dsn"
dsxfile="$dsxfile"
oprange=""
norule=""
verbose=""
reverb=""
module=`basename $0`
DSXLOG_MODULE=$module; export DSXLOG_MODULE
dsxdir="/var/local/dsx/import"
dsximp="/usr/local/bin/dsximp.exe"
dsxpids="/var/run/"
importdir="/var/local/bin/Automated_Testing/Auto_Scripts"

I wrote something like this:
Code:
function batch
{
for dsxfile in $dsxdir/*
do
$dsximp $norule $oprange --dsn=$dsn --dbname=$dbname --user=datasafe --password=datasafe --dsxfile=$dsxdir/$dsxfile
done

Code:
}
function displayHelp()
{
echo ""
echo " Usage: ./import.sh [ -s -d -r -o -f ]"
echo ""
echo " -s, --dsn identified by the Data Source Name "
echo " -d, --dbname Identified by the Database Name "
echo " -r, --norule Do not excecute dsxrule " 
echo " -o, --oprange Use optimize range checking " 
echo " -f, --dsxfile Identified by the file in dsx format "
echo " -b, --batch Runs the import for a batch of files "
echo " -h, --help Identified by the help menu "
echo ""

}
if [ -e ${dsxpids}$module.pid ] 
then
dsxlog $reverb --warning --module="$module" " pid = $$ : Is already running"
true
exit;
else
dsxlog $reverb --info --module="$module" "started pid = $$" 
while getopts " s: d: r o f: b e h " option
do
case $option in

f ) dsxfile="$OPTARG";;
d ) dbname="$OPTARG";;
s ) dsn="$OPTARG";;
r ) norule="-r";;
o ) oprange="-O";;
e ) verbose="--verbose";
reverb="--echo";;
b ) batch;;
h ) displayHelp;
dsxlog $reverb --info --module="$module" "displaying --helpmenu";
dsxlog $reverb --info --module="$module" "ended, pid = $$";
exit;;
? | * )displayHelp;
dsxlog $reverb --info --module="$module" "invalid option entered --see helpmenu ";
dsxlog $reverb --info --module="$module" "ended, pid = $$";
exit;;

esac;
done
if [ $# -eq 0 ]; then
displayHelp
dsxlog $reverb --info --module="$module" "no options have been entered --see helpmenu"
dsxlog $reverb --info --module="$module" "ended, pid = $$" 
exit 1
fi
 
$dsximp $norule $oprange --dsn=$dsn --dbname=$dbname --user=datasafe --password=datasafe --dsxfile=$dsxdir/$dsxfile
dsxlog $reverb --info --module="$module" "ended, pid = $$" 
fi

# 2  
Old 05-06-2010
are u getting any errors??
# 3  
Old 05-07-2010
Not errors, but it isn't performing the function I would like it to do.

It doesn't read the files....
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash script to take cPanel backup in batches

I'm trying to resolve the below scenario by writing bash script. On my managed linux server I have about 30 accounts and there is not enough space to generate full 30 accounts backup in one go and transfer it via SFTP to my Backup Synology Server. What I'm thinking of doing is breaking the... (3 Replies)
Discussion started by: humble_learner
3 Replies

2. Shell Programming and Scripting

Killing the process if running for long time in script

I am running a script which will read the data from fail line by line and call the Java program by providing the arguments from the each line. The Java code is working fast for few records and for some records its getting hanged not providing response for morethan one hour. Currently am... (4 Replies)
Discussion started by: dineshaila
4 Replies

3. Shell Programming and Scripting

Example of running script with time limits?

Hi, I want to write a script that does some sort of health check on the database. It will query the database for information, some query takes long and some are quick. For example, inside the script I will do something as below: #!/bin/ksh run_query_01 & run_query_02 &... (1 Reply)
Discussion started by: newbie_01
1 Replies

4. Shell Programming and Scripting

Setting time for running of the script

Dear all, I wonder if it is possible that we can run the script from time to time..I meant, it should repeat the sourcing of the script by itself? In my case, I need to source this script manually from time to time, like once in every 10 minutes. emily, (2 Replies)
Discussion started by: emily
2 Replies

5. Shell Programming and Scripting

How to know the exact running time of script!

Hi All, newbie here, I'm just wondering how can i know the exact running time of my script? Please advise, THanks, (1 Reply)
Discussion started by: nikki1200
1 Replies

6. Shell Programming and Scripting

Help in running a script after a particular time

Unix Gurus, I have a requirement where the shell script needs to do specific tasks after certain period of time. Daily we receive few files in a particular folder. The script does the file renaming, pass parameters to run some web services and pushes to remote FTP location. But my... (3 Replies)
Discussion started by: shankar1dada
3 Replies

7. Emergency UNIX and Linux Support

Help to optimize script running time

Dear Forum experts I have the below script which I made to run under bash shell, it runs perfectly for low records number, let us say like 100000. when I put all records (3,000,000), it's takes hours can you please suggest anything to optimize or to run in different way :-| {OFS="|";... (6 Replies)
Discussion started by: yahyaaa
6 Replies

8. UNIX and Linux Applications

Processing batches of files at a time

Hi I would like to run my script to process as many as 50 files at a time. Currently my script is being called like so: ./import.sh -f filename so I want to call my script in this way and it must execute every file in the directory /var/local/dsx/import (1 Reply)
Discussion started by: ladyAnne
1 Replies

9. Shell Programming and Scripting

display time required to complete running script

hi is there any way i can display a countdown time needed to run a script? like load a counter at the beginning of the script with the estimated time and display the counter decrementing till it finishes running the script? (3 Replies)
Discussion started by: npatwardhan
3 Replies

10. Shell Programming and Scripting

Need to know abt script that invokes batches and get d log files if batches failed

hi , I need to know commands to be used in the script to invoke batches in order from other scripts and then run those batches,and how to take those logs of those batches which fails........If anyone give me a better idea to complete this entire task in a single script... (5 Replies)
Discussion started by: gopimeklord
5 Replies
Login or Register to Ask a Question