Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Tune my script

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Tune my script
# 1  
Old 05-25-2012
Tune my script
Hi !

My script read out data out of 144 files per day - every ten minutes a file with data.
data-file
Code:
 
WR030B      306.71       0
WR050B      315.13       0
WR120B      308.34       0
WV030B        3.52        0
WV050B        5.06        0
WV120B        6.65       0
TLUFT02B      8.60        0
TLUFT12B      7.94        0
NSCHLB        0.00         0

my script is working like this for files with the namestructure YYYYMMDDHHMM
Code:
 
#!/bin/bash
M=`date +%m`
Y=`date +%Y`
home=$HOME/METE
cd $home
FLIST=`ls $Y$M*`
for i in $FLIST
do
     if test -f $i; then
  Dy=`echo $i | cut -c3-4`
  Dm=`echo $i | cut -c5-6`
  Dd=`echo $i | cut -c7-8`
 UZh=`echo $i | cut -c9-10` 
 UZm=`echo $i | cut -c11-12`
  WV=`grep WV120B $i | awk '{print $2}'`
  WR=`grep WR120B $i | awk '{print $2}'`
  NS=`grep NSCHLB $i | awk '{print $2}'`
T120=`grep TLUFT12B $i | awk '{print $2}'`
# echo $i " --> "$UZh":"$UZm " --> "$Dm"/"$Dd"/"$Dy"-"$UZh":"$UZm "\t" $WV
 printf "$Dm/$Dd/$Dy-$UZh:$UZm\t$WV\t$WR\t$NS\t$T120\n"
 else
 DAT=`date +%Y%m%d`
        printf "$DAT - Datei $i is not here" >> /$HOME/Logfiles/Log_`date +%Y%m%d`.log
    fi
done

so at the end i got one file like this
Code:
 
...
05/25/12-03:30 7.28 82.58 0.00 17.29
05/25/12-03:40 7.90 82.08 0.00 17.17
05/25/12-03:50 8.78 78.06 0.00 17.06
05/25/12-04:00 8.73 79.78 0.00 16.88
05/25/12-04:10 8.13 81.56 0.00 16.64
...

Code:
 
time /script.sh

will bring
real 0m41.111s
user 0m1.400s
sys 0m4.134s
So far it is working properly, but i want to speed it up.
How can i change the script to make it more fast?

Thanks in advance!
IMPe
# 2  
Old 05-25-2012
Hi


Few points:

1. The commands with grep and awk combination can be done with awk only, like:
Instead of
Code:
 WV=`grep WV120B $i | awk '{print $2}'`

Use
Code:
WV=`awk '/WV120B/{print $2}' $i`

This will get rid of those 4 grep commands.

2. The date command is used twice in the beginning +"as many files you have" times, whereas just one date would have done the thing for you. also the one inside the loop is not needed since the result is fixed. Instead of these 3 commands, use the below at the very beginning of the script:

Code:
      DAT=`date +%Y%m%d`
      Y=`echo ${DAT:0:4}`
      M=`echo ${DAT:4:2}`

This step will get rid of your umpteen date commands.

3. Instead of the statement
Code:
Dy=`echo $i | cut -c3-4`

Use:
Code:
 Dy=` echo ${i:2:2}`

This will get rid of those 4 cut commands.

4. In place of the command:
Code:
FLIST=`ls $Y$M*`

Use:
Code:
 FLIST=`echo $Y$M*`

ls is an external command.

Guru.
This User Gave Thanks to guruprasadpr For This Post:
IMPe
# 3  
Old 05-25-2012
Like guruprasadr suggests, it is important to limit the number of external programs as much as possible, especially within the loop. Try:
Code:
#!/bin/bash
fpat=$(date +%Y%m)*
home=$HOME/METE
cd $home
for i in $fpat
do
  if test -f "$i"
  then
    Dy=${i:2:2} Dm=${i:4:2} Dd=${i:6:2}
    UZh=${i:8:2} UZm=${i:10:2}
    res=$(awk '{A[$1]=$2}END{print A["WV120B"], A["WR120B"], A["NSCHLB"], A["TLUFT12B"]}' OFS='\t' "$i")
    printf "%s/%s/%s-%s:%s\t%s\n" $Dm $Dd $Dy $UZh $UZm "$res"
  else
    DAT=$(date +%Y%m%d)
    printf "$DAT - Datei $i is not here" >> "$HOME/Logfiles/Log_$(date +%Y%m%d).log"
  fi
done


--
Guru: these suggestions could be further improved by leaving out some of the echo + back tick combinations.
Last edited by Scrutinizer; 05-25-2012 at 10:07 AM..
This User Gave Thanks to Scrutinizer For This Post:
IMPe
# 4  
Old 05-25-2012
Thanks a lot!
With the advancement of @guruprasadpr the script takes only half the time, bud with @Scrutinizer improvement it only needs 5.4 sec.
really great!! Smilie

IMPe
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. HP-UX

Top cmd showing NICE value 97% -what to tune?

Running 2 VM Guests on an HPUX Integrity Server. One Guest runs great, the other is always at a high NICE value and 0% idle as shown in TOP: What do you think should be tuned to bring down the NICE and increase IDLE %? Thanks in advance -hpuxadmin slow VM GUEST Load averages: 2.56,... (5 Replies)
Discussion started by: hpuxadmin
5 Replies

2. Shell Programming and Scripting

Fine tune this perl script to add router

Hi, I have this routine that reads a microsoft dhcp.netsh dump. Where it finds optionvalue 3 STRING "0.0.0.0" Replace it with the router IP based on the network !/usr/bin/perl while ( <> ) { if ( /\# NET / ) { $net = $'; $net =~ s///g; } else { s/set optionvalue 3... (1 Reply)
Discussion started by: richsark
1 Replies

3. Programming

SQL : Fine tune Insert by query

i would like to know how can i fine tune the following query since the cost of the query is too high .. insert into temp temp_1 select a,b,c,d from xxxx .. database used is IDS.. (1 Reply)
Discussion started by: expert
1 Replies

4. UNIX for Dummies Questions & Answers

Need tune my command occupying 90% CPU

Can some body tune the below command, its occupyinh more than 90% of CPU some times. tail -n 1000 /logs/trace.log | awk 'BEGIN{OOM = 0; ScE = 0; NaE = 0; Jms = 0} /OutOfMemoryException/{OOM = 1} /StaleConnectionException/{ScE = 1} /NamingException/{NaE = 1} /JmsTimeOutException/{Jms = 1}... (17 Replies)
Discussion started by: senthil.ak
17 Replies

5. Solaris

Please tune my script for Solaris

I have very big log file around 2-3 GB in that it contians 24 hours log data. My work is extract only 5-5 data and count the patterns from them. I worte a script in linux and we're using that. sed -n "/2009 05:/,/2009 17:/p" trace.log | grep -f patterns.txt > temp.log while read string ;do... (5 Replies)
Discussion started by: senthil.ak
5 Replies

6. Shell Programming and Scripting

Tune my query

I have a requirement to separate only some numbers from the input file and produce it in a format. The input is ( i have took a sample, the actual file contains more than 50000 rows around 840 MB in size) $cat temp.txt 001 08 002 08 003 06 004 11 005 11 006 08 007 08 008 92* 009 92 010... (1 Reply)
Discussion started by: senthil.ak
1 Replies

7. Cybersecurity

How to fine Tune and Harden the Linux kernel

Hi, As a a security audit, how can I proceed further with Fine tuning and Hardening the linux kernel... I am not sure with the steps how to proceed further... If i do some thing wrong, then its comes with the Kernel panic error. So, I am afraid, how to do the tuning with the kernel.. (1 Reply)
Discussion started by: gsiva
1 Replies

8. UNIX for Dummies Questions & Answers

Tune my logic of script

I have big log file, which contains the netstat output from my application server to a particular DB server. I aim is to plot a daily graph for this. Please find the sample log file below. @ - ........................................................... @ - Total number of connection to the ... (3 Replies)
Discussion started by: senthilkumar_ak
3 Replies
Login or Register to Ask a Question