Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

How to read from multiple files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to read from multiple files
# 1  
Old 06-18-2007
How to read from multiple files
Hi All,

I have list of multiple files with 7 fields all together. Those are being split to exact lines of 20000 each.

xaa
xab
:
:
:
xhx


Please advise me how to read from those files and in fact I need to invoke and sql update statement for each inputs values..

Regards,
# 2  
Old 06-18-2007
Cedrichiu,
Please specify more details and give us an example of your input and
expected output.
# 3  
Old 06-18-2007
Hi Shell_Life,

If I list out content of

> cat xaa | head

392183921|94389549|A584958|TOM|11, CON STREET|BROADWAY|1122

Somehow, this is what could figure out on how to read from if it is single file

while read file
do
_msisdn=$(echo $file | cut -d "|" -f 1)
_cust_ic=$(echo $file | cut -d "|" -f 2)
_cust_pass=$(echo $file | cut -d "|" -f 3)
_cust_name=$(echo $file | cut -d "|" -f 4)
_cust_street1=$(echo $file | cut -d "|" -f 5)
_cust_street2=$(echo $file | cut -d "|" -f 6)
_cust_post=$(echo $file | cut -d "|" -f 7)
echo "UPDATE customer SET cust_alt_fax='$_cust_ic',cust_bld_nr='$_cust_pass',cust_name='$_cust_name',cust_street_1='$_cust_str eet1',cust_street
_2='$_cust_street2',cust_add_pcode='$_cust_post',chk_upd_dt=CURRENT" >> upd_addl_reg.sql
echo "WHERE cust_msisdn='$_msisdn';" >> upd_addl_reg.sql
done < xaa , xab... ... xhx

Due to db limitation, I have to perform update 20000 rows at a time to reduce load, that's why you could see i split files to several as named below. Smilie


Please advise
# 4  
Old 06-18-2007
Cedrichiu,
See if this works for you:
Code:
for mFile in x?
do
  echo "Now working with file "$mFile" at "`date`
  mOutFile=$mFile".sql"
  while read mLine
  do
    _msisdn=$(echo $mLine | cut -d "|" -f 1)
    .......
    echo "WHERE cust_msisdn='$_msisdn';" >> $mOutFile
  done < $mFile
done

You will have an sql file for each split file.
# 5  
Old 07-10-2007
hi,

I'm not sure whether if conditional statement correct or not? Please advise
Basically, if the field 2 exist and with value , it will be update set as cust_alt_fax column and if field 3 exist and with value, update set as cust_alt_fax , else field 4 exist and with value, update set as cust_alt_fax column

Quote:
for mFile in x???
do
# echo "Now working with file "$mFile" at "`date`
mOutFile=$mFile".sql"
while read mLine
do
_msisdn=$(echo $mLine | cut -d "|" -f 1)
_cust_ic=$(echo $mLine | cut -d "|" -f 2)
_cust_other_ic=$(echo $mLine | cut -d "|" -f 3)
_cust_pass=$(echo $mLine | cut -d "|" -f 4)
_cust_name=$(echo $mLine | cut -d "|" -f 5)
_cust_street1=$(echo $mLine | cut -d "|" -f 6)
_cust_street2=$(echo $mLine | cut -d "|" -f 7)
_cust_post=$(echo $mLine | cut -d "|" -f 8)
if [ -n "wc -l $_cust_ic" ] ; then
echo "UPDATE customer SET cust_alt_fax='$_cust_ic',cust_name='$_cust_name',cust_street_1='$_cust_street1',cust_street_2='$_cus t_street2'
,cust_add_pcode='$_cust_post',chk_upd_dt=CURRENT" >> $mOutFile
echo "WHERE cust_msisdn='$_msisdn';" >> $mOutFile
elif [ -n "wc -l $_cust_other_ic" ] ; then
echo "UPDATE customer SET cust_alt_fax='$_cust_other_ic',cust_name='$_cust_name',cust_street_1='$_cust_street1',cust_street_2= '$_cust_st
reet2',cust_add_pcode='$_cust_post',chk_upd_dt=CURRENT" >> $mOutFile
echo "WHERE cust_msisdn='$_msisdn';" >> $mOutFile
else
echo "UPDATE customer SET cust_alt_fax='$_cust_pass',cust_name='$_cust_name',cust_street_1='$_cust_street1',cust_street_2='$_c ust_street
2',cust_add_pcode='$_cust_post',chk_upd_dt=CURRENT" >> $mOutFile
echo "WHERE cust_msisdn='$_msisdn';" >> $mOutFile
fi
done < $mFile
done
# 6  
Old 07-10-2007
Cedrichiu,
To test if a variable has a value:
Code:
if [ -n "${YourVar}" ]; then
  echo "Variable "${YourVar}" is not empty"
else
  echo "Variable "${YourVar}" is empty"
fi

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to read multiple files at same time through UNIX scripting?

How to read multiple files at simultaneously? (1 Reply)
Discussion started by: Priyanka_M
1 Replies

2. Shell Programming and Scripting

Script to read multiple files...

I have 7 text files of varying sizes for each month of System Maintenance done during the 2013 calendar year (Jan. 134 jobs, Feb. 84 jobs, Apr. 594 jobs, May 158 jobs, July 69 jobs, Aug. 1 job, Oct. 102 jobs) and I have another text file which contains everything from those 7 files. Each of the... (8 Replies)
Discussion started by: CyberOptiq
8 Replies

3. UNIX for Dummies Questions & Answers

awk - how to read multiple files

Hi, is there a ways to read multiple files in a single awk command? For example: awk -f awk_script file1 file2 file3 I've google it, most of them suggest using FNR. But I don't understand how it works. It will be a great help if someone able to explain it in simple term with some example. (4 Replies)
Discussion started by: KCApple
4 Replies

4. Shell Programming and Scripting

read the lines of multiple files

I am trying to create a script which will read 2 files and use the lines of file 1 for each line on file 2. here's my sample code cat $SBox | while read line do cat $Date | while read line do $SCRIPTEXE <line from first file> $2 <line from 2nd file> ... (12 Replies)
Discussion started by: khestoi
12 Replies

5. UNIX for Advanced & Expert Users

Use awk to read multiple files twice

Hi folks I have a situation where I am trying to use awk to compute mean and standard deviation for a variable that spans across multiple files. The layout of each file is same and arranged in 3 columns and uses comma as a delimiter. File1 layout: col1,col2,col3 0,0-1,0.2345... (13 Replies)
Discussion started by: scandy
13 Replies

6. Shell Programming and Scripting

Multiple files read error in Shell

Hi All, there is a script that reads multiple files in a directory starting with CTG_TMPxx where xx is an integer number. i have a problem in readin the last file. For example, if there are 10 files in that directory, 9 files are read and processed. but the 10th file, its saying as cannot read... (1 Reply)
Discussion started by: vkca
1 Replies

7. Shell Programming and Scripting

Read and edit multiple files using a while loop

Hi all, I would like to simply read a file which lists a number of pathnames and files, then search and replace key strings using a few vi commands: :1,$s/search_str/replace_str/g<return> but I am not sure how to automate the <return> of these vis commands when I am putting this in a... (8 Replies)
Discussion started by: cyberfrog
8 Replies

8. Shell Programming and Scripting

How to Read Multiple files in a Shell Script

Hi, Can any one tell me if i can read two files in a shell script... My actual requirement is to read the 1st text file and parse it to get the file code and use this file code to retrieve data from database and print the fetched data in the 2nd text file (I have parsed it and printed the... (2 Replies)
Discussion started by: funonnet
2 Replies

9. Shell Programming and Scripting

Read the data from multiple files and sum the value

Hi all, I have a requirement where i have to read multiple files using Shell Script in Korn Shell. each file will have the 3rd line as the amount field, i have to read this amount field and sum it for all the files. any idea on how to achieve this?? (i think i can achieve it using a loop,... (9 Replies)
Discussion started by: nvuradi
9 Replies

10. Shell Programming and Scripting

Perl program to read from multiple files

Hi, I need to generate a test data file by reading inputs from multiple files. A sample pseudo code for this program to read from three files and write to a output file would be like: open(OUTPUTFILE, "data"); open(INFILE1, "data1"); open(INFILE2, "data2"); open(INFILE3, "data3"); ... (1 Reply)
Discussion started by: jyotipg
1 Replies
Login or Register to Ask a Question