Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Optimize awk code
Top Forums Shell Programming and Scripting Optimize awk code Post 302966000 by Scrutinizer on Saturday 6th of February 2016 03:07:35 AM
Old 02-06-2016
Hi, you did not specify the shell, since you are using GNU utilities, I presumed it to be bash, this would functionally be these equivalent, but should be a bit more efficient:

Code:
ZPROCC=$(
  while read ALLF 
  do
    IFS=_- read ALL EPTIME x <<< "$ALLF"
    ZSCORE=$(( STDEVIATE>0 ? ( ALL - AVERAGE ) / STDEVIATE : 0 ))
    read x mon day time year x <<< $(perl -le "print scalar(localtime($EPTIME))")
    ACSCORE="[$mon-$day-($time)-$year]"
    echo "frq=${ALL},std=${ZSCORE},time=${ACSCORE},epoch=${EPTIME},avg=${AVERAGE}"
  done <<< "$VALUESA"
)

It is unsorted, since
$(echo ${VALUESA} | sort -r | xargs) produces the same output as ${VALUESA}

So, as is, it could be further reduced to:
Code:
ZPROCC=$(
  while IFS=_- read ALL EPTIME x
  do
    ZSCORE=$(( STDEVIATE>0 ? ( ALL - AVERAGE ) / STDEVIATE : 0 ))
    read x mon day time year x <<< $(perl -le "print scalar(localtime($EPTIME))")
    ACSCORE="[$mon-$day-($time)-$year]"
    echo "frq=${ALL},std=${ZSCORE},time=${ACSCORE},epoch=${EPTIME},avg=${AVERAGE}"
  done <<< "$VALUESA"
)

Which leaves one external call to perl per iteration. To eliminate that one as well the whole loop would need to be eliminated in favor of -for example- one awk or perl program...

I don't know where AVERAGE and STDEVIATE are determined ? Is that is n a similar loop, if so I suspect similar gains could be made there?

---edit---

This would be a gawk equivalent:

Code:
ZPROCC=$(
  gawk -F'[_-]' -v av="$AVERAGE" -v sd="$STDEVIATE" '
    {
      zscore=(sd>0) ? ($1-av)/sd : 0
      acscore=strftime("%b-%e-(%H:%M:%S)-%Y",$2)
      printf "frq=%s,std=%s,time=%s,epoch=%s,avg=%s\n", $1, zscore, acscore, $2, av
    }
  ' <<< "$VALUESA"
)

Last edited by Scrutinizer; 02-06-2016 at 04:50 AM..
This User Gave Thanks to Scrutinizer For This Post:
SkySmart
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

optimize the script

Hi, I have this following script below. Its searching a log file for 2 string and if found then write the strings to success.txt and If not found write strings to failed.txt . if one found and not other...then write found to success.txt and not found to failed.txt. I want to optimize this... (3 Replies)
Discussion started by: amitrajvarma
3 Replies

2. Shell Programming and Scripting

can we optimize this command

can we optimize this command ? sed 's#AAAA##g' /study/i.txt | sed '1,2d' | tr -d '\n\' > /study/i1.txt; as here i am using two files ...its overhead..can we optimise to use only 1 file sed 's#AAAA##g' /study/i.txt | sed '1,2d' | tr -d '\n\' > /study/i.txt; keeping them same but it... (9 Replies)
Discussion started by: crackthehit007
9 Replies

3. Shell Programming and Scripting

Optimize shell code

#!/usr/bin/perl use strict; use warnings; use Date::Manip; my $date_converted = UnixDate(ParseDate("3 days ago"),"%e/%h/%Y"); open FILE,">$ARGV"; while(<DATA>){ my @tab_delimited_array = split(/\t/,$_); $tab_delimited_array =~ s/^\ =~ s/^\-//; my $converted_date =... (2 Replies)
Discussion started by: sandy1028
2 Replies

4. Shell Programming and Scripting

pl help me to Optimize the given code

Pl help to me to write the below code in a simple way ... i suupose to use this code 3 to 4 places in my makefile(gnu) .. **************************************** @for i in $(LIST_A); do \ for j in $(LIST_B); do\ if ;then\ echo "Need to sign"\ echo "List A = $$i , List B =$$j"\ ... (2 Replies)
Discussion started by: pk_arun
2 Replies

5. Shell Programming and Scripting

Optimize the nested IF

Hi, I have to assign a value for a varaiable based on a Input. I have written the below code: if then nf=65 elif then nf=46 elif then nf=164 elif then nf=545 elif then nf=56 elif then (3 Replies)
Discussion started by: machomaddy
3 Replies

6. Shell Programming and Scripting

Can someone please help me optimize my code (script searches subdirectories)?

Here is my code. What it does is it reads an input file (input.txt which contains roughly 2,000 search phrases) and searches a directory for files that contains the search phrase. The directory contains roughly 1900 files and 84 subdirectories. The output is a file (output.txt) that shows only the... (23 Replies)
Discussion started by: jl487
23 Replies

7. Shell Programming and Scripting

Optimize awk command

WARNING=${1} CRITICAL=${2} echo ${OUTPUT} | gawk -F'' ' { V = $2 R = $0 } END { for ( i = 1; i <= n; i++) { if((V > 0) && (V < V)) print R, ((V - V) / V) * 100 else if ((V > V) && (V > 0)) ... (6 Replies)
Discussion started by: SkySmart
6 Replies

8. Shell Programming and Scripting

Optimize my mv script

Hello, I'm wondering if there is a quicker way of doing this. Here is my mv script. d=/conversion/program/out cd $d ls $d > /home/tempuser/$$tmp while read line ; do a=`echo $line|cut -c1-5|sed "s/_//g"` b=`echo $line|cut -c16-21` if ;then mkdir... (13 Replies)
Discussion started by: whegra
13 Replies

9. Shell Programming and Scripting

Looking to optimize code

Hi guys, I feel a bit comfortable now doing bash scripting but I am worried that the way I do it is not optimized and I can do much better as to how I code. e.g. I have a whole line in a file from which I want to extract some values. Right now what I am doing is : STATE=`cat... (5 Replies)
Discussion started by: Junaid Subhani
5 Replies

10. Shell Programming and Scripting

Optimize multiple awk variable assignment

how can i optimize the following: TOTALRESULT="total1=4 total2=9 total3=89 TMEMORY=1999" TOTAL1=$(echo "${TOTALRESULT}" | egrep "total1=" | awk -F"=" '{print $NF}') TOTAL2=$(echo "${TOTALRESULT}" | egrep "total2=" | awk -F"=" '{print $NF}') TOTAL3=$(echo... (4 Replies)
Discussion started by: SkySmart
4 Replies
All times are GMT -4. The time now is 10:24 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy