Tuning function


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Tuning function
# 1  
Old 05-20-2010
Tuning function

This is my function in UNIX file. In this function I am

-> first replacing spaces in character 19-27 with 0
-> then if it's all zeros ( 9 zeros) replace it with space

The reason I have to make it to 0 first is that my requirement is that if this field is having value of 0 , replace it with spaces. The zeros in coming in various ways like-

All zeros. (000000000)
Spaces in between. (0000 0)
Only 1 zero.( 0)

This function is taking 2 minutes 25 seconds for 2000 records which is very very slow. We can have time window of 25-30 minutes and we can get upto 50000 records in1 day. Could someone suggest some ways to tune this fucntion.

Code:
modiify_customer_feed_id()
{
   while IFS='' read line
   do
      f1="$(echo "$line" | cut -c1-18)"
      f2="$(echo "$line" | cut -c19-27)"
      f3="$(echo "$line" | cut -c28-854)"
      f2=$(echo "$f2"|sed 's/ /0/g')
      printf "%s%s%s\n" "$f1" "$f2" "$f3"
   done < ${SOURCEFILE} > $id_interim_file
   echo " ID PROCESS 1 "
   while IFS='' read line
   do
      f1="$(echo "$line" | cut -c1-18)"
      f2="$(echo "$line" | cut -c19-27)"
      f3="$(echo "$line" | cut -c28-854)"
      f2=$(echo "$f2"|sed 's/000000000/ /g')
      printf "%s%s%s\n" "$f1" "$f2" "$f3"
   done < $id_interim_file > $id_final_file
   echo " ID PROCESS 2 "
   if [[ $? -ne 0 ]]
   then
      rm -f $id_interim_file $id_final_file
      return 1
   else
      mv -f $id_final_file $SOURCEFILE
      rm -f $id_interim_file
      return 0
   fi
}

Moderator's Comments:
Mod Comment Some indention of your code next time would be fine, ty.

Moderator's Comments:
Mod Comment Neo: Removed font formatting in code.
# 2  
Old 05-20-2010
try this:

Code:
echo 'some_data_blablah_0       00' | awk '{
b=substr($0,1,18)
s=substr($0,19,9)
e=substr($0,28,length($0)-27)
gsub (" ","0",s)
if (s ~ /0{9}/) {gsub (" ","0",s)}
print b""s""e}'

not tested yet.
# 3  
Old 05-20-2010
tuning in final function

This is my final function. For 2000 records running in 3 minutes. Can this be tuned . Many Thanks in advance.

Code:
modify_customer_feed()
{
while IFS='' read line
do
f1="$(echo "$line" | cut -c1-18)"
f2="$(echo "$line" | cut -c19-27)"
f2=$(echo "$f2"|sed 's/ /0/g')
f2=$(echo "$f2"|sed 's/000000000/         /g')
f3="$(echo "$line" | cut -c28-686)"
f4="$(echo "$line" | cut -c687-702)"
f4=$(echo "$f4"|sed 's/ /0/g')
f4=$(echo "$f4"|sed 's/0000000000000000/                /g')

f5="$(echo "$line" | cut -c703-718)"
f5=$(echo "$f5"|sed 's/ /0/g')
f5=$(echo "$f5"|sed 's/0000000000000000/                /g')
f6="$(echo "$line" | cut -c719-854)"
printf "%s%s%s%s%s%s\n" "$f1" "$f2" "$f3" "$f4" "$f5" "$f6"
done < ${SOURCEFILE}    >  $id_final_file

if [[ $? -ne 0 ]]
then
  rm -f  $id_final_file
  return 1
else
  mv -f $id_final_file $SOURCEFILE
  return 0
fi
}

# 4  
Old 05-20-2010
Are you using bash?
# 5  
Old 05-20-2010
you are having lots of redirection and command substitution on each line which makes this bulky.

If you can try the awk solution, that would be faster.

with some correction and removed typos,

Code:
awk '{
b=substr($0,1,18)
s=substr($0,19,9)
e=substr($0,28,length($0)-27)
gsub (" ","0",s)
if (s ~ /000000000/) {gsub ("0"," ",s);print b s e} else {print $0}}' file

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Solaris

nxge tuning

Hi all, I would like to tune the nxge card as suggested by this link, but got some confusion. Can anyone advise me ? We have SunOS hsbc02 5.10 Generic_137137-09 sun4v sparc SUNW,Netra-CP3260 Do I have to install any patches ? The tunning link Networks - Siwiki In our nxge.conf, I... (0 Replies)
Discussion started by: dehetoxic
0 Replies

2. AIX

I/O tuning for oracle

is it a good practice to enable AIO (Async I/O) and mount the oracle file system with DIO with JFS2 (Direct I/O) option? please help (3 Replies)
Discussion started by: pchangba
3 Replies

3. Web Development

MySQL Tuning Tools with mysqltuner.pl and tuning-primer.sh

We have been tuning MySQL lately and I ran accoss two useful tools that you might be interested in: mysqltuner.pl tuning-primer.sh Both of these scripts are quite useful for MySQL tuning. Here is some sample output of mysqltuner.pl >> MySQLTuner 0.9.8 - Major Hayden... (3 Replies)
Discussion started by: Neo
3 Replies

4. Shell Programming and Scripting

Performance Tuning

Hi All, In last one week, i have posted many questions in this portal. At last i am succeeded to make my 1st unix script. following are 2 points where my script is taking tooooo long. 1. Print the total number of records excluding header & footer. I have found that awk 'END{print NR -... (2 Replies)
Discussion started by: Amit.Sagpariya
2 Replies

5. Solaris

tuning

hw to increse nfs perfomence tuning? (4 Replies)
Discussion started by: naresh.sun
4 Replies

6. UNIX for Advanced & Expert Users

Tuning AIX IO

Hi I am trying to investigate a disk performance issue, and we are not seem to be hitting the right direction in our analysis. This is a FC disk running on USP1000 HDS system. The application is an IO intensive application, but our opinion is that it is not performing due to perceived disk... (1 Reply)
Discussion started by: theerthan
1 Replies

7. UNIX for Advanced & Expert Users

SAS Tuning

Does anyone had perfomed a tuning with SAS on Solaris??? Performance is not so good and I found out that Share Memory an Semaphores are the same that initial instalation, I havent found info at internet Please help (1 Reply)
Discussion started by: alex blanco
1 Replies

8. UNIX for Dummies Questions & Answers

Performance Tuning

Hi to all, I'm interested in finding an introduction about Performance Tuning under Unix (or Linux); can somebody please point me in the right direction? Best regards (1 Reply)
Discussion started by: domyalex
1 Replies

9. UNIX for Advanced & Expert Users

Tuning Apache

I have an Apache 1.3.27 on a Solaris 8 Sun box that I am trying to tweek out for max performance. The only documentation I have run accross is on apache.org. I also have the O'Reilly Apache book. Anyone have another source I can refer to? (2 Replies)
Discussion started by: edkung
2 Replies

10. UNIX for Dummies Questions & Answers

Performance tuning.

can someone tell me a good site to go to in order to learn this. please do not recommen nay books because i dont have interest in that. if you know of any good sites with good straight forward explanation on how to split loads on machines that has excessive loading, please let me know Also,... (1 Reply)
Discussion started by: TRUEST
1 Replies
Login or Register to Ask a Question