Sponsored Content
Full Discussion: Optimize awk code
Top Forums Shell Programming and Scripting Optimize awk code Post 302965996 by SkySmart on Friday 5th of February 2016 10:48:50 PM
Old 02-05-2016
Thanks RudiC. I took your suggestions into consideration and combined all those commands into one awk command. Thanks so much.

In doing the above, i discovered the code i originally pasted in this thread is not the reason why the script was slow. I found out that it is the for loop below that takes at least 4 seconds to complete.

can anyone help me optimize the below code?

Content of variable VALUESA:

Code:
VALUESA="1751-1451549113_0--1751
1445-1451549413_0--1445
1864-1451549713_0--1864
1410-1451550013_0--1410
655-1451550313_0--655
147-1451550613_0--147
209-1451550913_0--209
1472-1451551213_0--1472
1984-1451551513_0--1984
690-1451551813_0--690
652-1451552113_0--652
1161-1451552413_0--1161
1314-1451552713_0--1314
1030-1451553013_0--1030
428-1451553313_0--428
262-1451553613_0--262
95-1451553913_0--95"

The slow for loop:

Code:
                                        ZPROCC=$(
                                        for ALLF in $(echo ${VALUESA} | sort -r | xargs)
                                        do
                                                ALL=$(echo "${ALLF}" | gawk -F"-" '{print $1}') ; ZSCORE=$(gawk "BEGIN {if($STDEVIATE>0) {print (${ALL} - ${AVERAGE}) / ${STDEVIATE}} else {print 0}}")
                                                EPTIME=$(echo "${ALLF}" | gawk -F"-" '{print $2}' | awk -F"_" '{print $1}')
                                                FIXED=$(gawk -v c="perl -le 'print scalar(localtime("${EPTIME}"))'" 'BEGIN{c|getline; close(c); print $0;}')
                                                ACSCORE=$(echo ${FIXED} ${EPTIME} | gawk '{print "["$2"-"$3"-""("$4")""-"$5"]"}')
                                                echo "frq=${ALL},std=${ZSCORE},time=${ACSCORE},epoch=${EPTIME},avg=${AVERAGE}"
                                        done)

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

optimize the script

Hi, I have this following script below. Its searching a log file for 2 string and if found then write the strings to success.txt and If not found write strings to failed.txt . if one found and not other...then write found to success.txt and not found to failed.txt. I want to optimize this... (3 Replies)
Discussion started by: amitrajvarma
3 Replies

2. Shell Programming and Scripting

can we optimize this command

can we optimize this command ? sed 's#AAAA##g' /study/i.txt | sed '1,2d' | tr -d '\n\' > /study/i1.txt; as here i am using two files ...its overhead..can we optimise to use only 1 file sed 's#AAAA##g' /study/i.txt | sed '1,2d' | tr -d '\n\' > /study/i.txt; keeping them same but it... (9 Replies)
Discussion started by: crackthehit007
9 Replies

3. Shell Programming and Scripting

Optimize shell code

#!/usr/bin/perl use strict; use warnings; use Date::Manip; my $date_converted = UnixDate(ParseDate("3 days ago"),"%e/%h/%Y"); open FILE,">$ARGV"; while(<DATA>){ my @tab_delimited_array = split(/\t/,$_); $tab_delimited_array =~ s/^\ =~ s/^\-//; my $converted_date =... (2 Replies)
Discussion started by: sandy1028
2 Replies

4. Shell Programming and Scripting

pl help me to Optimize the given code

Pl help to me to write the below code in a simple way ... i suupose to use this code 3 to 4 places in my makefile(gnu) .. **************************************** @for i in $(LIST_A); do \ for j in $(LIST_B); do\ if ;then\ echo "Need to sign"\ echo "List A = $$i , List B =$$j"\ ... (2 Replies)
Discussion started by: pk_arun
2 Replies

5. Shell Programming and Scripting

Optimize the nested IF

Hi, I have to assign a value for a varaiable based on a Input. I have written the below code: if then nf=65 elif then nf=46 elif then nf=164 elif then nf=545 elif then nf=56 elif then (3 Replies)
Discussion started by: machomaddy
3 Replies

6. Shell Programming and Scripting

Can someone please help me optimize my code (script searches subdirectories)?

Here is my code. What it does is it reads an input file (input.txt which contains roughly 2,000 search phrases) and searches a directory for files that contains the search phrase. The directory contains roughly 1900 files and 84 subdirectories. The output is a file (output.txt) that shows only the... (23 Replies)
Discussion started by: jl487
23 Replies

7. Shell Programming and Scripting

Optimize awk command

WARNING=${1} CRITICAL=${2} echo ${OUTPUT} | gawk -F'' ' { V = $2 R = $0 } END { for ( i = 1; i <= n; i++) { if((V > 0) && (V < V)) print R, ((V - V) / V) * 100 else if ((V > V) && (V > 0)) ... (6 Replies)
Discussion started by: SkySmart
6 Replies

8. Shell Programming and Scripting

Optimize my mv script

Hello, I'm wondering if there is a quicker way of doing this. Here is my mv script. d=/conversion/program/out cd $d ls $d > /home/tempuser/$$tmp while read line ; do a=`echo $line|cut -c1-5|sed "s/_//g"` b=`echo $line|cut -c16-21` if ;then mkdir... (13 Replies)
Discussion started by: whegra
13 Replies

9. Shell Programming and Scripting

Looking to optimize code

Hi guys, I feel a bit comfortable now doing bash scripting but I am worried that the way I do it is not optimized and I can do much better as to how I code. e.g. I have a whole line in a file from which I want to extract some values. Right now what I am doing is : STATE=`cat... (5 Replies)
Discussion started by: Junaid Subhani
5 Replies

10. Shell Programming and Scripting

Optimize multiple awk variable assignment

how can i optimize the following: TOTALRESULT="total1=4 total2=9 total3=89 TMEMORY=1999" TOTAL1=$(echo "${TOTALRESULT}" | egrep "total1=" | awk -F"=" '{print $NF}') TOTAL2=$(echo "${TOTALRESULT}" | egrep "total2=" | awk -F"=" '{print $NF}') TOTAL3=$(echo... (4 Replies)
Discussion started by: SkySmart
4 Replies
ECHO(3) 								 1								   ECHO(3)

echo - Output one or more strings

SYNOPSIS
void echo (string $arg1, [string $...]) DESCRIPTION
Outputs all parameters. echo is not actually a function (it is a language construct), so you are not required to use parentheses with it. echo (unlike some other language constructs) does not behave like a function, so it cannot always be used in the context of a function. Additionally, if you want to pass more than one parameter to echo, the parameters must not be enclosed within parentheses. echo also has a shortcut syntax, where you can immediately follow the opening tag with an equals sign. Prior to PHP 5.4.0, this short syn- tax only works with the short_open_tag configuration setting enabled. I have <?=$foo?> foo. PARAMETERS
o $arg1 - The parameter to output. o $... - RETURN VALUES
No value is returned. EXAMPLES
Example #1 echo examples <?php echo "Hello World"; echo "This spans multiple lines. The newlines will be output as well"; echo "This spans multiple lines. The newlines will be output as well."; echo "Escaping characters is done "Like this"."; // You can use variables inside of an echo statement $foo = "foobar"; $bar = "barbaz"; echo "foo is $foo"; // foo is foobar // You can also use arrays $baz = array("value" => "foo"); echo "this is {$baz['value']} !"; // this is foo ! // Using single quotes will print the variable name, not the value echo 'foo is $foo'; // foo is $foo // If you are not using any other characters, you can just echo variables echo $foo; // foobar echo $foo,$bar; // foobarbarbaz // Some people prefer passing multiple parameters to echo over concatenation. echo 'This ', 'string ', 'was ', 'made ', 'with multiple parameters.', chr(10); echo 'This ' . 'string ' . 'was ' . 'made ' . 'with concatenation.' . " "; echo <<<END This uses the "here document" syntax to output multiple lines with $variable interpolation. Note that the here document terminator must appear on a line with just a semicolon. no extra whitespace! END; // Because echo does not behave like a function, the following code is invalid. ($some_var) ? echo 'true' : echo 'false'; // However, the following examples will work: ($some_var) ? print 'true' : print 'false'; // print is also a construct, but // it behaves like a function, so // it may be used in this context. echo $some_var ? 'true': 'false'; // changing the statement around ?> NOTES
Note Because this is a language construct and not a function, it cannot be called using variable functions. SEE ALSO
print(3), printf(3), flush(3), Heredoc syntax. PHP Documentation Group ECHO(3)
All times are GMT -4. The time now is 08:04 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy