Reading a pipe delimited file and loading teradata


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Reading a pipe delimited file and loading teradata
# 1  
Old 05-11-2009
Reading a pipe delimited file and loading teradata

Hi,

I am new to UNIX shell scripting (KSH). I have a file in UNIX server which is delimited by |.

I require the basic syntax to:
  • read the file
  • move it to a local variables
  • load a database - Teradata
Continue until end of file.

Note: if the field is have no value inbetween two pipes '||', it should be loaded as NULL in the table.

Please help me out.
# 2  
Old 05-11-2009
you are new to ksh, so have you read anything on that before attempting to solve this?
# 3  
Old 05-11-2009
I am trying to find a proper online help book but could not get a good one.
# 4  
Old 05-11-2009
Hi Shiva,
Please let me know, how you are loading the fine in TERADATA. Using fast load or multiload or ?

As per your question, i think u can find and replace two pipe lines as |NULL|.
For this use sed :

sed 's/||/|NULL|/g' filename > newfilename

Now start loading.
Thanks
Javeed Sunar
# 5  
Old 05-11-2009
Quote:
Originally Posted by javeed7
Hi Shiva,
Please let me know, how you are loading the fine in TERADATA. Using fast load or multiload or ?

As per your question, i think u can find and replace two pipe lines as |NULL|.
For this use sed :

sed 's/||/|NULL|/g' filename > newfilename

Now start loading.
Thanks
Javeed Sunar
and what if the FIRST or the LAST field is empty?
# 6  
Old 05-11-2009
Hi,

I am loading it using MLOAD...

the first and last field would never be empty
# 7  
Old 05-11-2009
Quote:
Originally Posted by shivacbz
I require the basic syntax to:
  • read the file
  • move it to a local variables
  • load a database - Teradata
I know nothing about Teradata, but I assume it has a tool like Oracle SQLLoader capable to load a table from a flat csv or fixed width file.
So, to what purpose would you like to capture values in shell variables?

Anyway, to capture lines composed of 3 fields, the basic loop should be as the following:

Code:
myfile=$1

IFS='|' while read -r f1 f2 f3
do
  print "Do whatever you want with $f1, $f2, $f3"
done < $myfile

Of course if the field values are quoted when they contain the "|" character meaning itself, then the problem becomes much worse.

Last edited by colemar; 05-11-2009 at 06:42 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Need to convert a pipe delimited text file to tab delimited

Hi, I have a rquirement in unix as below . I have a text file with me seperated by | symbol and i need to generate a excel file through unix commands/script so that each value will go to each column. ex: Input Text file: 1|A|apple 2|B|bottle excel file to be generated as output as... (9 Replies)
Discussion started by: raja kakitapall
9 Replies

2. Shell Programming and Scripting

How to ignore Pipe in Pipe delimited file?

Hi guys, I need to know how i can ignore Pipe '|' if Pipe is coming as a column in Pipe delimited file for eg: file 1: xx|yy|"xyz|zzz"|zzz|12... using below awk command awk 'BEGIN {FS=OFS="|" } print $3 i would get xyz But i want as : xyz|zzz to consider as whole column... (13 Replies)
Discussion started by: rohit_shinez
13 Replies

3. Shell Programming and Scripting

Help with converting Pipe delimited file to Tab Delimited

I have a file which was pipe delimited, I need to make it tab delimited. I tried with sed but no use cat file | sed 's/|//t/g' The above command substituted "/t" not tab in the place of pipe. Sample file: abc|123|2012-01-30|2012-04-28|xyz have to convert to: abc 123... (6 Replies)
Discussion started by: karumudi7
6 Replies

4. UNIX for Advanced & Expert Users

Check Whether File is Pipe Delimited

Can anybody help me how to check whether a file is Pipe delimited or not? (1 Reply)
Discussion started by: Allwin333
1 Replies

5. UNIX for Dummies Questions & Answers

Reading a pipe delimited file

Hi Guys, i am reading a pipe delimited file using awk command. I have tested the gawk separately. it was fine. But when i execute the script. i am getting the following error saying command not found. Can somebody point out as what i am doing wrong. Cheers!!! (3 Replies)
Discussion started by: mac4rfree
3 Replies

6. Shell Programming and Scripting

How to convert a space delimited file into a pipe delimited file using shellscript?

Hi All, I have space delimited file similar to the one as shown below.. I need to convert it as a pipe delimited, the values inside the pipe delimited file should be as highlighted... AA ATIU2345098809 009697 005374 BB ATIU2345097809 005445 006518 CC ATIU9685098809 003215 003571 DD... (7 Replies)
Discussion started by: nithins007
7 Replies

7. UNIX for Dummies Questions & Answers

Delete last value from pipe delimited file

I have a large(ish) pipe delimited file. The last line of the file contains a total row count and a checksum: END|1537451|1328569446 After making other adjustments to the file, I need to strip out the checksum and apply a new value - I have a script to generate the checksum and 'cat' it... (3 Replies)
Discussion started by: relentl3ss
3 Replies

8. Shell Programming and Scripting

convert a pipe delimited file to a':" delimited file

i have a file whose data is like this:: osr_pe_assign|-120|wg000d@att.com|4| osr_evt|-21|wg000d@att.com|4| pe_avail|-21|wg000d@att.com|4| osr_svt|-11|wg000d@att.com|4| pe_mop|-13|wg000d@att.com|4| instar_ready|-35|wg000d@att.com|4| nsdnet_ready|-90|wg000d@att.com|4|... (6 Replies)
Discussion started by: priyanka3006
6 Replies

9. UNIX for Dummies Questions & Answers

Extracting from pipe delimited file.

Hey, I am new to regualar expression. I wanted to extract the information from a pipe delimited file which has some entries like L|S2CMG1B|||-11178399||1|-8.65|IRCSH|BOND||||N|S|IRDL|AUD||CRP|STD|CRP|M|0|1|||CSH||||OTHER|01|DE|KFW|50418Y9T5|||||||||||2|||||| In this I want to extract the... (1 Reply)
Discussion started by: leepan2008
1 Replies

10. UNIX for Dummies Questions & Answers

Loading a comma Delimited file into an Array

Hello, I have been stuck on this aspect of loading a comma delimited file into an array. I thought i had the syntax right, but my commands are not working the way I want them to. Basically my cut command is splitting the file up by spaces and commas. I want the cut command to ignore white spaces.... (2 Replies)
Discussion started by: grandtheftander
2 Replies
Login or Register to Ask a Question