|
|
|
|
|
|||||||
| Forums | Portal | Register | Forum Rules | FAQ | Contribute | Members List | Arcade | Search | Today's Posts | Mark Forums Read |
| Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts here. |
More UNIX and Linux Forum Topics You Might Find Helpful
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| How to parse through a file and based on condition form another output file | sivasu.india | UNIX for Advanced & Expert Users | 6 | 02-28-2008 12:59 AM |
| Need help to parse the file | navsharan | Shell Programming and Scripting | 3 | 01-17-2008 11:58 AM |
| Parse XML file | viki | Shell Programming and Scripting | 5 | 04-13-2007 01:25 AM |
| Parse file | sbasetty | Shell Programming and Scripting | 5 | 03-27-2007 10:27 AM |
| how to insert new coloum | rajan_ka1 | Shell Programming and Scripting | 3 | 12-23-2005 01:22 AM |
 |
|
|
Submit Tools | LinkBack | Thread Tools | Display Modes |
04-29-2008
|
|||
|
|||
parse coloum in a file
I have a file like the format
0000023716272400000237162724555778180300000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000001099001099425%00109925%IPY 2007013120070227Y99991231N YYX SRT0001010125%0000503318882HANDH800 R 0000023716282300000237162823555765956000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000001099001099425%00109925%IPY 2007013100010101Y99991231N YYX SRT0001010125%0000503418302HANDH800 R 0000030805382100000308053821555002938300000010000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000001699001699Q39%00169939%IP 2000093020000905N00010101N NYXNNNSRT0001010139%0000200601622PMPP6007 Y Z here i copy and past my code ############################################# # Parse File ############################################# cut -c2-14 /var/www/html/final_java/IngramHourlyFile/stock@ingram.dat|awk '{print "where isbn=|"$1"|"}'|tr "|" "'">/var/www/html/final_java/IngramHourlyFile/isbn.dat cut -c39-45 /var/www/html/final_java/IngramHourlyFile/stock@ingram.dat|awk '{print "UPDATE item_details SET ib_N_qty="$1}'>/var/www/html/final_java/IngramHourlyFile/OHQTY_N.dat cut -c46-52 /var/www/html/final_java/IngramHourlyFile/stock@ingram.dat|awk '{print "ib_E_qty="$1}'>/var/www/html/final_java/IngramHourlyFile/OHQTY_E.dat cut -c53-59 /var/www/html/final_java/IngramHourlyFile/stock@ingram.dat|awk '{print "ib_D_qty="$1}'>/var/www/html/final_java/IngramHourlyFile/OHQTY_D.dat cut -c60-66 /var/www/html/final_java/IngramHourlyFile/stock@ingram.dat|awk '{print "ib_C_qty="$1}'>/var/www/html/final_java/IngramHourlyFile/OHQTY_C.dat cut -c151-156 /var/www/html/final_java/IngramHourlyFile/stock@ingram.dat|awk '{print "iblastupdate=NOW(),ib_qty=(ib_E_qty+ib_N_qty+ib_D_qty+ib_C_qty),ib_listprice="$1"/100"}'>/var/www/html/final_java/IngramHourlyFile/PRICE.dat cut -c164-166 /var/www/html/final_java/IngramHourlyFile/stock@ingram.dat|awk '{print "ib_discode=|"$1"|"}'|tr "|" "'">/var/www/html/final_java/IngramHourlyFile/DISCODE.dat ############################################ # Change File permission ############################################ chmod 777 /var/www/html/final_java/IngramHourlyFile/isbn.dat chmod 777 /var/www/html/final_java/IngramHourlyFile/OHQTY_N.dat chmod 777 /var/www/html/final_java/IngramHourlyFile/OHQTY_E.dat chmod 777 /var/www/html/final_java/IngramHourlyFile/OHQTY_D.dat chmod 777 /var/www/html/final_java/IngramHourlyFile/OHQTY_C.dat chmod 777 /var/www/html/final_java/IngramHourlyFile/PRICE.dat chmod 777 /var/www/html/final_java/IngramHourlyFile/DISCODE.dat paste -d"," /var/www/html/final_java/IngramHourlyFile/OHQTY_N.dat /var/www/html/final_java/IngramHourlyFile/OHQTY_E.dat /var/www/html/final_java/IngramHourlyFile/OHQTY_D.dat /var/www/html/final_java/IngramHourlyFile/OHQTY_C.dat /var/www/html/final_java/IngramHourlyFile/PRICE.dat /var/www/html/final_java/IngramHourlyFile/DISCODE.dat>/var/www/html/final_java/IngramHourlyFile/tmpfinal_isbn.dat chmod 777 /var/www/html/final_java/IngramHourlyFile/tmpfinal_isbn.dat paste -d" " /var/www/html/final_java/IngramHourlyFile/tmpfinal_isbn.dat /var/www/html/final_java/IngramHourlyFile/isbn.dat>/var/www/html/final_java/IngramHourlyFile/tfinal_isbn.dat chmod 777 /var/www/html/final_java/IngramHourlyFile/tfinal_isbn.dat paste -d";" /var/www/html/final_java/IngramHourlyFile/tfinal_isbn.dat /var/www/html/final_java/IngramHourlyFile/single_quote.dat>/var/www/html/final_java/IngramHourlyFile/final_isbn.dat chmod 777 /var/www/html/final_java/IngramHourlyFile/final_isbn.dat It takes too long time because the file size is too big.I have to do it with in 15 min. please help me. thanks! vaskar 
|
| Forum Sponsor | ||
|
|
|
04-29-2008
|
|||
|
|||
|
not sure if this will increase performance by a significant amuont but you could always copy those files to the /tmp dir and read from there. also, a suggestion for you. when you have a dir that is common in your code, i would write it once to help others and you read your script a little easier:
Code:
#############################################
# Parse File
#############################################
dir=/var/www/html/final_java/IngramHourlyFile
cut -c2-14 $dir/stock@ingram.dat|awk '{print "where isbn=|"$1"|"}'|tr "|" "'">$dir/isbn.dat
cut -c39-45 $dir/stock@ingram.dat|awk '{print "UPDATE item_details SET ib_N_qty="$1}'>$dir/OHQTY_N.dat
cut -c46-52 $dir/stock@ingram.dat|awk '{print "ib_E_qty="$1}'>$dir/OHQTY_E.dat
cut -c53-59 $dir/stock@ingram.dat|awk '{print "ib_D_qty="$1}'>$dir/OHQTY_D.dat
cut -c60-66 $dir/stock@ingram.dat|awk '{print "ib_C_qty="$1}'>$dir/OHQTY_C.dat
cut -c151-156 $dir/stock@ingram.dat|awk '{print "iblastupdate=NOW(),ib_qty=(ib_E_qty+ib_N_qty+ib_D_qty+ib_C_qty),ib_listprice="$1"/100"}'>$dir/PRICE.dat
cut -c164-166 $dir/stock@ingram.dat|awk '{print "ib_discode=|"$1"|"}'|tr "|" "'">$dir/DISCODE.dat
|
|
04-29-2008
|
|||
|
|||
|
This is a good example of Useless Use of cut, tr , chmod, paste and temp files when you can use only awk.
Code:
#cat awk.code
{
a="UPDATE item_details SET ib_N_qty="substr($0,39,7)
b="ib_E_qty="substr($0,46,7)
c="ib_D_qty="substr($0,53,7)
d="ib_C_qty="substr($0,60,7)
e="iblastupdate=NOW(),ib_qty=(ib_E_qty+ib_N_qty+ib_D_qty+ib_C_qty),ib_listprice="substr($0,151,6)"/100"
f="ib_discode=\47"substr($0,164,3)"\47"
g="where isbn=\47" substr($0,2,13)"\47"
print a","b","c","d","e","f" "g";"
}
Code:
awk -f awk.code data.file > sql.file Last edited by danmero; 04-29-2008 at 08:01 AM. Reason: Typo
|
|
04-29-2008
|
|||
|
|||
|
Thanks!
 it is working....
|
|||
| Google The UNIX and Linux Forums |
Linear Mode
