Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

script to add numbers is slow

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting script to add numbers is slow
# 1  
Old 03-23-2009
script to add numbers is slow
Hi,

I am running a BASH shell with the following script. The script works and gives me correct output but is very slow with large files. The more rows and columns (width and height) the slower as you can probably see.

How can I do what I want more efficiently? Any ideas welcome. It has been ages since I have gone down the scripting road and - wow - it is slow to come back.

Thanks in advance.

i=$(($WIDTH * $HEIGHT - $WIDTH - 1))
k=$(($WIDTH - 1))
for ((j=0;j<$i;j++))
do
if [[ $j -eq $k ]]
then echo $((k=$(($k + $WIDTH))))
else
second=$(( j + 1 ))
third=$(( j + WIDTH + 1))
fourth=$(( j + WIDTH))
echo "4 $j $second $third $fourth" >> file.out
fi
done
# 2  
Old 03-23-2009
can you provide the sample i/p and o/p
and is it ok if its a ksh script???
Last edited by vidyadhar85; 03-23-2009 at 08:48 PM..
# 3  
Old 03-23-2009
if we make the WIDTH = 5 and the HEIGHT = 5
then the output is
4 0 1 6 5
4 1 2 7 6
4 2 3 8 7
4 3 4 9 8
4 5 6 11 10
4 6 7 12 11
4 7 8 13 12
4 8 9 14 13
4 10 11 16 15
4 11 12 17 16
4 12 13 18 17
4 13 14 19 18
4 15 16 21 20
4 16 17 22 21
4 17 18 23 22
4 18 19 24 23

which is correct but if the WIDTH = 5000 and the HEIGHT = 5000 then it can take hours to create the output file, and yes, a large output file
# 4  
Old 03-23-2009
Have you considered just writing that up as a C program? Multimegabyte output, character by character, seems to be better suited to a more powerful language.

Of course finding ways to do it more efficiently in a shell script would be interesting, and may be your point, but if all you want is a way to generate those files quickly...
# 5  
Old 03-23-2009
you can try out the following one liner
provide WIDTH and HEIGHT to it. since you have such a complicated calculation its tough to make it run fast anyways try it outSmilie
Code:
awk -v var1="50" -v var2="50" 'BEGIN{
i=(var1*var2-var1-1)
k=(var2-1)
       {
          for(j=0;j<i;j++)
          {
                if(j==k)
                {
                     print (k+var1)
                }else
                    {
                     print "4 " j" "(j+1)" "(j+var1+1)" "(j+var1)
                     }
           }
       } 
}'

Last edited by vidyadhar85; 03-23-2009 at 09:31 PM..
# 6  
Old 03-23-2009
That is much faster!!

Thank you vidyadhar85

I was trying to figure out an awk solution and then got distracted trying to see if I could add columns of numbers quicker.

Your solution will work for now - again Thank you very much!!

macsurveyr
# 7  
Old 03-23-2009
Lightbulb
That is an improvement. I timed it at 2m13.027s doing 5000x5000. But there's a tenfold improvement with a direct translation to C (11.885s):
Code:
#include <stdio.h>

main(int argc, char *argv[])
{
    int height = atoi(argv[1]);
    int width = atoi(argv[2]);
    int i = width * height - width - 1;
    int k = width - 1;
    int j;
    for (j = 0; j < i; j++) {
        if (j == k) {
            //printf("%i\n", k + width);
        } else {
            int tmp = j + width;
            printf("4 %i %i %i %i\n", j, j + 1, tmp + 1, tmp);
        }
    }
    return 0;
}

An optimized version might be even better.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Improve script - slow process with big files

Gents, Please can u help me to improve this script to be more faster, it works perfectly but for big files take a lot time to end the job.. I see the problem is in the step (while) and in this part the script takes a lot time.. Please if you can find a best way to do will be great. ... (13 Replies)
Discussion started by: jiam912
13 Replies

2. Shell Programming and Scripting

Shell script reading file slow

I have shell program as below #!/bin/sh echo ======= LogManageri start ========== #This directory is getting the raw data from remote server Raw_data=/opt/ftplogs # This directory is ready for process the data Processing_dir=/opt/processing_dir # This directory is prcoessed files and... (4 Replies)
Discussion started by: Chenchireddy
4 Replies

3. Shell Programming and Scripting

Script to alert about a slow link on the website

Hello all, Currently I am using a script with "curl" to get the an alert if 200 ok would not be grepped.and the link is down. is it possible to get an alert mail if a particular link on a website is not completely down but SLOW?? (0 Replies)
Discussion started by: chirag991
0 Replies

4. Shell Programming and Scripting

Slow down output from dhclient-script to screen

Hi I know the basic about script and sleep processes. However this is more tricky: I would like to run sh -x /sbin/dhclient-script and slow down the output of the script as a whole. How would you do it? I would like to delay output on the screen with 1 second for every line for the output... (3 Replies)
Discussion started by: medium_linux
3 Replies

5. Shell Programming and Scripting

Slow Script Execution.

Basically my requirement is to know the total number of free anonymous ports. anonymous port range is 32768- 65535. i wrote a script for that ********************************************** for i in {32768..65535} do netstat -an | grep $i > /dev/null if ... (21 Replies)
Discussion started by: mohtashims
21 Replies

6. UNIX for Dummies Questions & Answers

Help with slow KSH script

My script builds a lot of these array lists, then compares their sizes which solves my problem, but runs very slow. :( set -A comboSorted -- $( for x in ${IDs} do nawk -v s=$x ' BEGIN { testPattern="^" s "$" } { if ( $2 ~ testPattern ) { getline;getline; if ($1 == "IMAGE_SIZE") print... (1 Reply)
Discussion started by: nerdcurious
1 Replies

7. Shell Programming and Scripting

Slow Perl script: how to speed up?

I had written a perl script to compare two files: new and master and get the output of the first file i.e. the first file: words that are not in the master file STRUCTURE OF THE TWO FILES The first file is a series of names ramesh sushil jonga sudesh lugdi whereas the second file (could be... (4 Replies)
Discussion started by: gimley
4 Replies

8. Shell Programming and Scripting

Bash script too slow

I have a bash script that will take approx. 130 days to complete. I am trying to grep a list of 1,144 user ID's out of 41 (1 GB each) files. The 41 files were originally one 41 G file, but that was horrendously too slow.:eek: This is my current file: #!/bin/bash for i in `cat... (11 Replies)
Discussion started by: tigta09
11 Replies

9. Shell Programming and Scripting

add numbers in shell script

cat dailyreports | grep "Important list" | awk -F":" '{print $2}' | awk -F" " '{print $1}' hey guys, after running the above combination of cat and awk, i get the below output: 3 4 2 9 now, i need to add these numbers up all in one line. i dont know what to add to that cat and awk one... (2 Replies)
Discussion started by: Terrible
2 Replies

10. UNIX for Advanced & Expert Users

My script runs too slow :-(...

Hello experts, I have a series issue in script that result with bad peformence and I wonder if you can assist me. For example I have two files: File-New, size 15Mb. File-Old, size 1Mb. File-New content: a b c k File-Old content: d f a b (0 Replies)
Discussion started by: roybe
0 Replies
Login or Register to Ask a Question