Improve program efficiency (awk)


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Improve program efficiency (awk)
# 1  
Old 07-27-2009
Improve program efficiency (awk)

Hi !! I've finished an awk exercise. Here it is:

Code:
#!/bin/bash

function calcula
{

# Imprimimos el mayor tamaño de fichero

ls -l $1 | awk '
    BEGIN {
       max = $5; # Inicializamos la variable que nos guardará el máximo con el tamaño del primer archivo
    }
    {
       if ($5 > max){  # Vamos comparando la columna filesizes para cada línea (archivo) que compone el ls -l
          max = $5;
       }
    }
    END {
    print "Tamanio mayor fichero = " max; }'


# Imprimimos ahora el menor tamaño de fichero

ls -l $1 | awk '
    NR==2{
      min = $5; next # Inicializamos la variable que nos guardará el máximo con el tamaño del primer archivo
             # Y saltamos los elementos restantes de la línea de entrada puesto que no nos interesan y nos vamos a la siguiente línea para procesar la columna filesize
    }
    {
      if ($5 < min){   # Vamos comparando la columa filesizes para cada línea (archivo) que compone el ls -l
        min = $5;
      }
    }
    END {
    print "Tamanio menor fichero = " min; }'

# NOTA: La ejecución del comando ls -l produce como primera línea "total xxxxxxx". Por eso, necesitamos, para calcular el mínimo, empezar a procesar los datos desde la segunda línea
# y por eso ejecutamos NR == 2, para que nos empiece en la segunda línea, puesto que de no hacerlo, la variable min tomaría un valor vacío.


# Y ahora sumamos todos los filesizes y los mostramos

ls -l $1 | awk '{ suma += $5; } END { print "Total bytes ruta : " suma; }' 

# Nota: para hacerlo más claro, podríamos expresar el tamaño total en megabytes, dividiendo dos veces
# suma entre 1024 --> ls -l | awk '{ suma += $5; } END { print suma/1024/1024; }' 


}
# Fin funcion

# Inicio del programa

i=1
while [ $# -gt 0 ]; do # Mientras existan rutas...

   # Imprimimos la ruta
   echo 
   echo La ruta es $1

   # Llamamos a la función que nos calcula todo
   calcula $1

   # Desplazamos los parámetros
   i=$(($i+1))
   shift
done
echo

My question is how can I improve the efficiency calling awk just one time and not 3.

Thanks !!
# 2  
Old 07-27-2009
No hablo Espanol pero you can try this:

Code:
ls -l $1 | awk '
BEGIN {
 max = $5 # Inicializamos la variable que nos guardará el máximo con el tamaño del primer archivo
}
NR==2{
  min = $5 next # Inicializamos la variable que nos guardará el máximo con el tamaño del primer archivo
  # Y saltamos los elementos restantes de la línea de entrada puesto que no nos interesan y nos vamos a la siguiente línea para procesar la columna filesize
}
{
  if ($5 > max){  # Vamos comparando la columna filesizes para cada línea (archivo) que compone el ls -l
  max = $5
  }
}
{
  if ($5 < min && NR > 2){   # Vamos comparando la columa filesizes para cada línea (archivo) que compone el ls -l
    min = $5
  }
}
{ suma += $5 }
END {
  print "Tamanio mayor fichero = " max
  print "Tamanio menor fichero = " min
  print "Total bytes ruta : " suma 
}'

Regards

Last edited by Franklin52; 07-27-2009 at 10:57 AM.. Reason: Placed wrong code
# 3  
Old 07-27-2009
something like:

Code:
#  ls -l | awk 'BEGIN{max=0;min=99999}NR>1{sum+=$5;if (min>$5){min=$5};if ($5>max){max=$5}}END{print sum;print max;print min}'
36420041
15763946
0

Should be a step inn the right direction..
# 4  
Old 07-27-2009
Quote:
Originally Posted by Phass
My question is how can I improve the efficiency calling awk just one time and not 3.
Code:
#!/bin/bash
eval $(ls -l $1 | awk 'NR>1{min=min?min:$5}{if(min >$5)min=$5}{if(max<$5)max=$5;suma+=$5}END{printf "min=%d max=%d suma=%d", min,max,suma}')

echo $min
echo $max
echo $suma

You can work from here.

---------- Post updated at 10:53 AM ---------- Previous update was at 10:13 AM ----------

Quote:
Originally Posted by Tytalus
something like:

Code:
#  ls -l | awk 'BEGIN{max=0;min=99999}NR>1{sum+=$5;if (min>$5){min=$5};if ($5>max){max=$5}}END{print sum;print max;print min}'
36420041
15763946
0

What if all files size on directory are greater that 99999 byte/98 kilobyte Smilie
# 5  
Old 07-27-2009
Danmero, i've tried ur code, but files with 0 size aren't caughted like min, just the file with the min size > 0.

PD: can u explain me a bit ur code? dont understand a lot "min=min?min:$5" and why u compare max without initialize it hehe

Last edited by Phass; 07-27-2009 at 12:15 PM..
# 6  
Old 07-27-2009
Quote:
Originally Posted by Phass
Danmero, i've tried ur code, but files with 0 size aren't caughted like min, just the file with the min size > 0.
Works for me.
Code:
# ls -l
total 10
-rw-r--r--  1 root  wheel   135 Jul 27 10:58 1.txt
-rw-r--r--  1 root  wheel   273 Jul 27 10:58 2.txt
-rw-r--r--  1 root  wheel   411 Jul 27 10:58 3.txt
-rw-r--r--  1 root  wheel   552 Jul 27 10:58 4.txt
-rw-r--r--  1 root  wheel  1371 Jul 27 10:57 file
-rw-r--r--  1 root  wheel     0 Jul 27 11:04 test.file
# ls -l $1 | awk 'NR>1{min=min?min:$5}{if(min >$5)min=$5}{if(max<$5)max=$5;suma+=$5}END{printf "min=%d max=%d suma=%d", min,max,suma}'
min=0 max=1371 suma=2742[11:04:49]#

# 7  
Old 07-27-2009
I don't understand why it doesn't work for me :S :S

Anyway, can u explain me what u do in the code? I want to understand what I do

I don't understand " min = min ? min : $5 ". min = min it's always true.. not? :S

Thanks man

Last edited by Phass; 07-27-2009 at 12:37 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

About efficiency of parallel memory allocation

Hello, there. I'm a new beginner to Linux kernel and curious about its memory management. When multiple applications apply for memory space at the same time, how Linux kernel solve the resource contending problem for high performance? I have known that there is a buddy system for allocating and... (4 Replies)
Discussion started by: blackwall
4 Replies

2. Shell Programming and Scripting

Improve awk code that has three separate parts

I have a very inefficient awk below that I need some help improving. Basically, there are three parts, that ideally, could be combined into one search and one output file. Thank you :). Part 1: Check if the user inputted string contains + or - in it and if it does the input is writting to a... (4 Replies)
Discussion started by: cmccabe
4 Replies

3. Shell Programming and Scripting

File or Folder Efficiency?

I've got this program set up so that it creates files whose unique names specify the jobs their contents describe. In order to retrieve the information inside those files, I have to do a "grep" and awk or sed to extract it. I've just assumed that making a directory with that unique name that... (1 Reply)
Discussion started by: gmark99
1 Replies

4. Shell Programming and Scripting

Looking to improve the output of this awk one-liner

I have the following awk one-liner I came up with last night to gather some data. and it works pretty well (apologies, I'm quite new with awk, and don't know how to format this pretty-printed). You can see the output with it. awk '{if ($8 == 41015 && $21 == "requests") arr+=$20;if ($8 == 41015... (3 Replies)
Discussion started by: DeCoTwc
3 Replies

5. Shell Programming and Scripting

Improve performance of echo |awk

Hi, I have a script which looks like this. Input file data1^20 data2^30 #!/bin/sh file"/home/Test.txt" while read line do echo $line |awk 'BEGIN { FS = "^" } ; { print $2 }' echo $line |awk 'BEGIN { FS = "^" } ; { print $1 }' | gzip | wc -c done <"$file" How can i... (4 Replies)
Discussion started by: chetan.c
4 Replies

6. Shell Programming and Scripting

The efficiency between GREP and SED???

Hello Everyone! I am a newbie. I'd like to get key lines from a big txt file by Reg Exp, The file is nearly 22MB. GREP or SED?which may be the best choice,more efficient way? or any other best practise? Thank you in advance. Ever:) (5 Replies)
Discussion started by: ever
5 Replies

7. Shell Programming and Scripting

Problem with awk awk: program limit exceeded: sprintf buffer size=1020

Hi I have many problems with a script. I have a script that formats a text file but always prints the same error when i try to execute it The code is that: { if (NF==17){ print $0 }else{ fields=NF; all=$0; while... (2 Replies)
Discussion started by: fate
2 Replies

8. Shell Programming and Scripting

Perl: code efficiency for gmtime

I have the following Perl snippet: # get datetime @dt = gmtime(); $strdate = 1900 + $dt . addleadingzero(++$dt) . addleadingzero($dt) . addleadingzero($dt) . addleadingzero($dt) . addleadingzero($dt); # write to file $outfile = $strdate . ".txt"; getstore($url, $outfile) or die "Error:... (3 Replies)
Discussion started by: figaro
3 Replies

9. Shell Programming and Scripting

efficiency..

how efficient is it, and how practical is it to call outside programs in a shell script (bash) for small tasks? for example, say i have a script that might preform many tasks, one of those tasks may require root access; rather than implementing inside the script a method to use su or sudo to... (11 Replies)
Discussion started by: norsk hedensk
11 Replies
Login or Register to Ask a Question