Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Creating script to read many files and load into database via single control file Post 303042120 by Chubler_XL on Sunday 15th of December 2019 07:20:54 PM
Old 12-15-2019
Rather than using sed to change your original template file and then change it back why not create a new file with the changes and remove once you are done. Here I use $$ which expands to shells processID to ensure unique filename in case script is run simultaneously.

Also assuming TEXT is supposed to be replaced with the table names:

Code:
for table in switch billing
do
   for file in /tmp/*.$table 
   do
       if [ -f "$file" ] 
       then
           sed -e "s|FILENAME|$i|g" -e "s|TEXT|$table|g" /tmp/data1.ctl  > /tmp/$$_data1.ctl
           sqlldr TNS control=/tmp/$$_data1.ctl
           rm /tmp/$$_data1.ctl
       fi
    done
done

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell Script to Load data into the database using a .csv file and .ctl file

Since i'm new to scripting i'm findind it difficult to code a script. The script has to be an executable with 2 paramters passed to it.The Parameters are 1. The Control file name(.ctl file) 2. The Data file name(.csv file) Does anybody have an idea about it? :confused: (3 Replies)
Discussion started by: Csmani
3 Replies

2. Shell Programming and Scripting

shell script to read data from text file and to load it into a table in TOAD

Hi....can you guys help me out in this script?? Below is a portion text file and it contains these: GEF001 000093625 MKL002510 000001 000000 000000 000000 000000 000000 000001 GEF001 000093625 MKL003604 000001 000000 000000 000000 000000 000000 000001 GEF001 000093625 MKL005675 000001... (1 Reply)
Discussion started by: pallavishetty
1 Replies

3. Shell Programming and Scripting

Need shell script to read two file at same time and print out in single file

Need shell script to read two file at same time and print output in single file Example I have two files 1) file1.txt 2) file2.txt File1.txt contains Aaa Bbb Ccc Ddd Eee Fff File2.txt contains Zzz Yyy Xxx (10 Replies)
Discussion started by: sreedhargouda
10 Replies

4. Shell Programming and Scripting

awk: creating a fixed-width single file from 2 different files

I have to create a single file from three files, Please see below for samples: day.txt 20090101 20090102 item.txt 123456789101 12345678910209 1234567891 str.txt 1 12 123 output.txt 20090101123456789101 1 0 2009010112345678910209 12 ... (2 Replies)
Discussion started by: tamahomekarasu
2 Replies

5. Shell Programming and Scripting

Creating a control file for a group of files

Hi, We have almost 45,000 data files created by a script daily. The file names are of format-ODS.POS.<pharmacyid>.<table name>.<timestamp>.dat. There will be one data file like this for each pharmacy and each table.(Totally around 45,000) The requirement is to create a control file for each... (2 Replies)
Discussion started by: Maya_Pillai
2 Replies

6. Shell Programming and Scripting

why do we need UNIX shell script to load data into Oracle database

Hello everyone, I am new to shell scripting/ loading data into a database. I want to load data into Oracle database using SQL loader. Can some one please explain why do we need unix shell script to load the data into the database? Also can someone please explain what has to be in that script?... (5 Replies)
Discussion started by: new_prog
5 Replies

7. Shell Programming and Scripting

How to read file and load data into a script as variable

I need to read a text file that contain columns of data, i need to read 1st column as a function to call, and others are the data i need to get into a ksh script. I am quite new to ksh scripting, i am not very sure how to read each row line by line and the data in each columns of that line, set... (3 Replies)
Discussion started by: gavin_L
3 Replies

8. UNIX for Advanced & Expert Users

Load a file from UNIX into database

Hi, I want your help to see if there is a way wherein we can load a set of files which are lying in unix server into my database using sql loader. Also the person who will be running sql loader does not have access to unix system. So is there a way i can provide any sql loader script wherein... (1 Reply)
Discussion started by: Vivekit82
1 Replies

9. UNIX for Beginners Questions & Answers

Generic script to load file details(ls -ltr) in to a database.

All, I am trying to create a report on the duration of an ETL load from the file arrival to the final dump in to a database for SLA's. Does anyone have any guidance or ideas on how metadata can be extracted; information of a file: like file name, created timestamp, count of records and load... (1 Reply)
Discussion started by: pradeepp
1 Replies

10. UNIX for Beginners Questions & Answers

Script for creating Control file in UNIX

Delete ---- Original post, restored by mod after being deleted by abhilashnair ---- I have a requirement where, I need to create a control file which will have 3 columns in the header row as below: Filename Count Checksum This above control file has to contain metadata as above... (2 Replies)
Discussion started by: abhilashnair
2 Replies
BP_LOAD_GFF(1p) 					User Contributed Perl Documentation					   BP_LOAD_GFF(1p)

NAME
bp_load_gff.pl - Load a Bio::DB::GFF database from GFF files. SYNOPSIS
% bp_load_gff.pl -d testdb -u user -p pw --dsn 'dbi:mysql:database=dmel_r5_1;host=myhost;port=myport' dna1.fa dna2.fa features1.gff features2.gff ... DESCRIPTION
This script loads a Bio::DB::GFF database with the features contained in a list of GFF files and/or FASTA sequence files. You must use the exact variant of GFF described in Bio::DB::GFF. Various command-line options allow you to control which database to load and whether to allow an existing database to be overwritten. This script uses the Bio::DB::GFF interface, and so works with all database adaptors currently supported by that module (MySQL, Oracle, PostgreSQL soon). However, it is slow. For faster loading, see the MySQL-specific bp_bulk_load_gff.pl and bp_fast_load_gff.pl scripts. NOTES If the filename is given as "-" then the input is taken from standard input. Compressed files (.gz, .Z, .bz2) are automatically uncompressed. FASTA format files are distinguished from GFF files by their filename extensions. Files ending in .fa, .fasta, .fast, .seq, .dna and their uppercase variants are treated as FASTA files. Everything else is treated as a GFF file. If you wish to load -fasta files from STDIN, then use the -f command-line swith with an argument of '-', as in gunzip my_data.fa.gz | bp_fast_load_gff.pl -d test -f - On the first load of a database, you will see a number of "unknown table" errors. This is normal. About maxfeature: the default value is 100,000,000 bases. If you have features that are close to or greater that 100Mb in length, then the value of maxfeature should be increased to 1,000,000,000, or another power of 10. COMMAND-LINE OPTIONS Command-line options can be abbreviated to single-letter options. e.g. -d instead of --database. --dsn <dsn> Data source (default dbi:mysql:test) --adaptor <adaptor> Schema adaptor (default dbi::mysqlopt) --user <user> Username for mysql authentication --pass <password> Password for mysql authentication --fasta <path> Fasta file or directory containing fasta files for the DNA --create Force creation and initialization of database --maxfeature Set the value of the maximum feature size (default 100 Mb; must be a power of 10) --group A list of one or more tag names (comma or space separated) to be used for grouping in the 9th column. --upgrade Upgrade existing database to current schema --gff3_munge Activate GFF3 name munging (see Bio::DB::GFF) --quiet No progress reports --summary Generate summary statistics for drawing coverage histograms. This can be run on a previously loaded database or during the load. SEE ALSO
Bio::DB::GFF, bulk_load_gff.pl, load_gff.pl AUTHOR
Lincoln Stein, lstein@cshl.org Copyright (c) 2002 Cold Spring Harbor Laboratory This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. See DISCLAIMER.txt for disclaimers of warranty. perl v5.14.2 2012-03-02 BP_LOAD_GFF(1p)
All times are GMT -4. The time now is 12:38 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy