Korn shell program to parse CSV text file and insert values into Oracle database


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Korn shell program to parse CSV text file and insert values into Oracle database
# 8  
Old 04-28-2011
As @Peasant mentioned sqlloader is THE tool for this job:

SQL*Loader
# 9  
Old 04-28-2011
@pravin : You might face an issue while dealing with date fields to manipulate the same if you use table type insertion..
# 10  
Old 04-29-2011
Table columns & file columns are not in the same order.

---------- Post updated at 10:44 PM ---------- Previous update was at 11:55 AM ----------

How do we tell in awk to ignore spaces in the fields.

If there are spaces in the fields , it is not giving correct output:
-------------------------
Code:
for i in `cat  file.txt`
do
col1=`echo  $i | awk -F ',' '{print $1}'`
done

Output:
Code:
TEST
 
TELEPROCESSING
(ITS)
 
IMPLEMENTATION
 
 
TEST
 
-
ALL
OTHERS
 
 
 
TEST
 
-
ALL
OTHERS

--------------------------------------------------
Used underscore for fields where there is space, i get correct output:
Code:
TEST
TEST
TEST

-------------------------------------------------

Last edited by Franklin52; 04-29-2011 at 03:58 AM.. Reason: Please use code tags
# 11  
Old 05-17-2011
Hi Folks,

Im going through this forum down here looking for some clues as how to insert a CSV file into a table in Oracle. Im in the same situation as the one who posted their question in here.

My input file is like this

395 , LIES CANT BE FOUND IN TRUTH, 20110517 , PRD1
231 , HOW GREEN WAS MY VALLEY, 20110517 , ADJCC

and I need to insert the above lines into a table that has 4 columns in the exact same order as above.

But I cant seem to do it no matter how I did. Here is what I have ... ofcourse a copy of what you have in here anyway....

#!/bin/sh

for i in `cat FILE`
do
col1=`echo $i | awk -F',' '{print $1}'`
col2=`echo $i | awk -F',' '{print $2}'`
col3=`echo $i | awk -F',' '{print $3}'`
col4=`echo $i | awk -F',' '{print $4}'`

sqlplus -s user/pwd@oracledb <<OSQL
INSERT INTO RDD_Book_Unit (COUNT, MESSAGE, PRINTDATE, IBNCODE)
values ($col1, $col2, $col3, $col4)
commit;
exit;
OSQL

done
----

Please advise as I feel like Im cursed now !

thanks,
PGonzalez.
# 12  
Old 05-17-2011
Just as already suggested by Franklin & Peasant, the right way of loading data into your DB is to use the corresponding tools that have been designed for that (In your case - i suppose - it is SQL Loader : sqlldr).

Since you refer to sqlplus, i suppose you have something like an Oracle DB behind, so i suggest you to have a look at Ask Tom "SQL and CSV fields"

Last edited by ctsgnb; 05-17-2011 at 05:32 AM..
# 13  
Old 05-17-2011
Hey Thanks for the quick response. Im as new as they come to Oracle itself. I read couple of posts here pointing me in this direction as it looks pretty simple & easy to follow & ofcourse it doesnt look like so.

At any rate, if you can help, that would great, otherwise, I will sit & learn sqlloader and see how that can be of help for me in this instance.

thanks,
PGonzalez.
# 14  
Old 05-17-2011
SQL Loader has better performance for large loads, but you can also just use a quick awk script to process the CSV file, e.g...
Code:
$ cat file1.csv
395 , LIES CANT BE FOUND IN TRUTH, 20110517 , PRD1
231 , HOW GREEN WAS MY VALLEY, 20110517 , ADJCC

$ cat eg.awk
BEGIN {
  FS = " *, *"
  ins = "INSERT INTO RDD_Book_Unit (COUNT, MESSAGE, PRINTDATE, IBNCODE) VALUES"
}
{
  printf "%s (%s, '%s', '%s', '%s');\n", ins, $1, $2, $3, $4
}

$ awk -f eg.awk file1.csv > eg.sql

$ cat eg.sql
INSERT INTO RDD_Book_Unit (COUNT, MESSAGE, PRINTDATE, IBNCODE) VALUES (395, 'LIES CANT BE FOUND IN TRUTH', '20110517', 'PRD1');
INSERT INTO RDD_Book_Unit (COUNT, MESSAGE, PRINTDATE, IBNCODE) VALUES (231, 'HOW GREEN WAS MY VALLEY', '20110517', 'ADJCC');

$

This User Gave Thanks to Ygor For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parse log file to insert into database

I have a log file that's created daily by this command: sar -u 300 288 >> /var/log/usage/$(date "+%Y-%m-%d")_$(hostname)_cpu.log It that contains data like this: Linux 3.16.0-4-amd64 (myhostname) 08/15/2015 _x86_64_ (1 CPU) 11:34:17 PM CPU %user %nice ... (12 Replies)
Discussion started by: unplugme71
12 Replies

2. Web Development

INSERT data to a Database Table from a text file

If you have a text file and if you want to Insert data to your Database Table, You can do it with these queries LOAD DATA LOCAL INFILE '/path/yourTextFile.txt' INTO TABLE yourTableName FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' ESCAPED BY '\\' LINES TERMINATED BY '\n' (0 Replies)
Discussion started by: sitex
0 Replies

3. Shell Programming and Scripting

insert values into sqlplus database using shell script

hello all, I am new to shell scripting and to unix... so the following is my assignment.. here i am trying to insert a values into sqlplus database using shell script. the following is my shell script InsertDelete.sh #! /bin/sh echo "*********The MENU******** 1.Insert The Values... (2 Replies)
Discussion started by: pradeept
2 Replies

4. Shell Programming and Scripting

how to insert data into database by reading it from a text file??

Hi....can you guys help me out in this script?? Below is a text file and it contains these: GEF001 000093625 MKL002510 000001 000000 000000 000000 000000 000000 000001 GEF001 000093625 MKL003604 000001 000000 000000 000000 000000 000000 000001 GEF001 000093625 MKL005675 000001... (4 Replies)
Discussion started by: pallavishetty
4 Replies

5. Shell Programming and Scripting

How to: Parse text string into variables using Korn shell

I am writing a script to keep check on free disk space, and I would like to find a way to parse $LINE (see code below) into a numeric value (for free disk space percentage) and a string value (for mount point). If possible, I would like to avoid sed or any additional use of awk since I am not very... (7 Replies)
Discussion started by: shew01
7 Replies

6. Shell Programming and Scripting

How to insert data into MYSql database from a text file

Hi, Need to get help from you guys about this issue. I need to insert data into MySql database from a text file which is located in other server. The text file is something look like below: Date | SubscriberNo | Call Duration 20/7/07 | 123456788 | 20 20/7/07 | 123412344 | 30 The... (4 Replies)
Discussion started by: shirleyeow
4 Replies

7. Shell Programming and Scripting

Shell Script: want to insert values in database when update script runs

Hi , I am new to linux and also also to shell scripting. I have one shell script which unpacks .tgz file and install software on machine. When this script runs I want to insert id,filename,description(which will be in readme file),log(which will be in log file) and name of unpacked folder... (1 Reply)
Discussion started by: ring
1 Replies

8. Shell Programming and Scripting

Need Shell Script to upload data from Text file to Oracle database

Hi Does any one have any idea on uploading the data using Unix Shell script from text file to Oracle database. Requirement:- 1. Need to connect to Oracle database from Unix Shell script. 2. Need to pick Text file from some location on Unix Box. 3. Need to upload the data from text file to... (6 Replies)
Discussion started by: chandrashekharj
6 Replies

9. Shell Programming and Scripting

how to insert data in database based on text file?

Hi....can you guys help me out in this script?? Below is a text file script....called Bukom.txt and it contains these: BUKOM 20060101 2.5 2.6 2.7 2.8 2.9 2.3 2.1 BUKOM 20060102 2.4 2.5 2.6 2.7 2.7 2.6 2.4 BUKOM 20060103 2.1 ... (9 Replies)
Discussion started by: forevercalz
9 Replies

10. Shell Programming and Scripting

unix script to export data from csv file to oracle database

Hello people, Need favour. The problem I have is that, I need to develop a unix shell script that performs recurring exports of data from a csv file to an oracle database. Basically, the csv file contains just the first name and last name will be dumped to an Unix server. The data from these... (3 Replies)
Discussion started by: vinayagan
3 Replies
Login or Register to Ask a Question