Sponsored Content
Full Discussion: looping through files
Top Forums Shell Programming and Scripting looping through files Post 302451999 by vpv0002 on Wednesday 8th of September 2010 09:39:38 PM
Old 09-08-2010
looping through files

I am writing a ksh which has to load 7 files(.dat files) from input directory into oracle tables using sql loader. The process has to take each file at a time and once if it is loaded succesfully using sql loader into oracle tables then the process has to pick next file and load it into oracle tables. Here we do not know when we will have the files in the input directory. So my script has to check for a file, if exists then load it if it does not exists then has to wait for a period of 15 mins and then start looking for a file and so on. Below is the code which i have written. Any help is greatly appreciated.
Code:
 
while [ 1 ]
do

## check for .dat files in inputdir
FILE_EXISTS="0"
FILE_EXISTS=`ls ${INPUT_DIR}/*.txt | wc -l`
echo "value in File_exist is $FILE_EXISTS"
if [ $FILE_EXISTS -eq 0 ]; then
   continue;
fi

##  if files exists then start processing them
for file in `ls ${INPUT_DIR}/*.txt`
do
   execute_sql_loader $file
   RC=?
   echo "sqlldr generated output $RC"
      if [ $RC -ne 0 ]; then
         continue;
      fi
done

sleep 10;
done


Last edited by vpv0002; 09-08-2010 at 10:48 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

looping files

Hi, I have a file a.lst which lists all files. as a.dat b.dat c.dat I want to process these files mentioned in the list file in a loop. Say I want to display only the first line of all the files a.dat , b.dat, c.dat. How can I go about it? Please help. (5 Replies)
Discussion started by: dharmesht
5 Replies

2. Shell Programming and Scripting

Help looping through files, please...

Okay... I've solved one problem. Here's the next. I'm writing a script file that needs to go through a directory and list all files in that directory. I'm using TCL/TK. I figured out how to go through the directory and how to loop through it, but I ran into a little problem. ... (2 Replies)
Discussion started by: kapolani
2 Replies

3. Shell Programming and Scripting

Looping on a list of files...

This isn't working for multiple files. It works for one file though. exists1=$(ls | grep gspp*) for FILES in $exists1 do echo "Loading $exists1" ... (23 Replies)
Discussion started by: lazerfoursix
23 Replies

4. Shell Programming and Scripting

Looping through files...

I posted this in the Solaris forum, but I don't think it's platform specific, so I'm posting it here. Here is the situation. We are a company that has been using a professional publishing system, the software is called "ProType". It runs on Solaris 2.4, however it is no longer supported and we... (6 Replies)
Discussion started by: Fred Goldman
6 Replies

5. Shell Programming and Scripting

Looping through 2 files simultaneously

Hi all, I'm having a problem with a script which should ultimately provide a filename by reading a value from file1 and file2 then join together. I'm planning to use a loop/ loops to get the values out of both files and create a single string unfortunately the code currently treats the second... (7 Replies)
Discussion started by: chris01010
7 Replies

6. Shell Programming and Scripting

looping through files with different extensions

Hi all, I am trying to make a for loop invoking files with different extensions (*.ugrd and *.vgrd) and I cant just make it work. Cant figure out how to load the files so as to use them in subsequent commands like the ones in this pseudo code. the files are arranged such that in one date for... (8 Replies)
Discussion started by: ida1215
8 Replies

7. Shell Programming and Scripting

Conditional Looping In Files

I have a req. where i need to read data from multiple files and take counts of row which satisfy the condition. e.g.: FILE1: Col1 Col2 Col3 12 ab cd 15 de fg 25 gh tm FILE2: Col1 Col2 Col3 21 ab1 cd1 13 de1 fg1 25 gh1 tm1 --- --- FILE-N... i need to find the count of rows... (6 Replies)
Discussion started by: kunal007
6 Replies

8. Shell Programming and Scripting

Not looping or creating files

So my script is supposed to repeat for every server in my file, but as of now it is getting stuck on my awk commands # Read file cred.txt (with one IP per line), connect to servers (one at a time), and download directory listing i=1 param=$(sed -n "{$1}p" $parm_dir/cdm_param.txt) #Get the last... (6 Replies)
Discussion started by: MJCreations
6 Replies

9. Shell Programming and Scripting

Looping through Files

Hi all , I am new on this forum . I have to face a particoular implementation issue and I need some help . Requirement : I need to read a particoular file (an xml file) and after reading it I need to call an Oracle Stored Procedure passing the content of the file as paramenter , in order... (3 Replies)
Discussion started by: Kolas79
3 Replies

10. Shell Programming and Scripting

Looping through files in pairs

Hi all, Please guide. It has to do with parsing the input file names. I have a fairly large number of files, I want to do some operations on them in a pairwise fashion (every file has a pair). The names are in the following pattern, with the pairs of files named with _1 and _2 , the... (4 Replies)
Discussion started by: newbie83
4 Replies
BP_BULK_LOAD_GFF(1p)					User Contributed Perl Documentation				      BP_BULK_LOAD_GFF(1p)

NAME
bulk_load_gff.pl - Bulk-load a Bio::DB::GFF database from GFF files. SYNOPSIS
% bulk_load_gff.pl -d testdb dna1.fa dna2.fa features1.gff features2.gff ... DESCRIPTION
This script loads a Bio::DB::GFF database with the features contained in a list of GFF files and/or FASTA sequence files. You must use the exact variant of GFF described in Bio::DB::GFF. Various command-line options allow you to control which database to load and whether to allow an existing database to be overwritten. This script differs from bp_load_gff.pl in that it is hard-coded to use MySQL and cannot perform incremental loads. See bp_load_gff.pl for an incremental loader that works with all databases supported by Bio::DB::GFF, and bp_fast_load_gff.pl for a MySQL loader that supports fast incremental loads. NOTES If the filename is given as "-" then the input is taken from standard input. Compressed files (.gz, .Z, .bz2) are automatically uncompressed. FASTA format files are distinguished from GFF files by their filename extensions. Files ending in .fa, .fasta, .fast, .seq, .dna and their uppercase variants are treated as FASTA files. Everything else is treated as a GFF file. If you wish to load -fasta files from STDIN, then use the -f command-line swith with an argument of '-', as in gunzip my_data.fa.gz | bp_fast_load_gff.pl -d test -f - The nature of the bulk load requires that the database be on the local machine and that the indicated user have the "file" privilege to load the tables and have enough room in /usr/tmp (or whatever is specified by the $TMPDIR environment variable), to hold the tables transiently. Local data may now be uploaded to a remote server via the --local option with the database host specified in the dsn, e.g. dbi:mysql:test:db_host The adaptor used is dbi::mysqlopt. There is currently no way to change this. About maxfeature: the default value is 100,000,000 bases. If you have features that are close to or greater that 100Mb in length, then the value of maxfeature should be increased to 1,000,000,000. This value must be a power of 10. Note that Windows users must use the --create option. If the list of GFF or fasta files exceeds the kernel limit for the maximum number of command-line arguments, use the --long_list /path/to/files option. COMMAND-LINE OPTIONS Command-line options can be abbreviated to single-letter options. e.g. -d instead of --database. --database <dsn> Database name (default dbi:mysql:test) --adaptor Adaptor name (default mysql) --create Reinitialize/create data tables without asking --user Username to log in as --fasta File or directory containing fasta files to load --long_list Directory containing a very large number of GFF and/or FASTA files --password Password to use for authentication (Does not work with Postgres, password must be supplied interactively or be left empty for ident authentication) --maxbin Set the value of the maximum bin size --local Flag to indicate that the data source is local --maxfeature Set the value of the maximum feature size (power of 10) --group A list of one or more tag names (comma or space separated) to be used for grouping in the 9th column. --gff3_munge Activate GFF3 name munging (see Bio::DB::GFF) --summary Generate summary statistics for drawing coverage histograms. This can be run on a previously loaded database or during the load. --Temporary Location of a writable scratch directory SEE ALSO
Bio::DB::GFF, fast_load_gff.pl, load_gff.pl AUTHOR
Lincoln Stein, lstein@cshl.org Copyright (c) 2002 Cold Spring Harbor Laboratory This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. See DISCLAIMER.txt for disclaimers of warranty. perl v5.14.2 2012-03-02 BP_BULK_LOAD_GFF(1p)
All times are GMT -4. The time now is 06:16 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy