Help with script


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help with script
# 1  
Old 05-29-2013
Help with script

Hi

the following code if working fine for small file
Code:
 
for i in 200 202 203 
do
SEQ=$[i]
gawk -v S="$SEQ" '
        {
                A[++c] = $1
        }
        END {
                for ( i = 1; i <= c; i++ )
                {
                        if ( A[i] == S )
                        {
                                C[A[i],A[i+1],A[i-1]]++
                                V[A[i],A[i+1],A[i-1]] = "else if  ( A[i] == 236 && A[i+1] ==" A[i+1] " && A[i-1] ==" A[i-1] ") {print A[i] = " A[i]"}"
                        }
                }
                for ( k in V )
                {
                        T[++j] = C[k]
                }
                n = asort(T)
                for ( i = n; i >= 1; i-- )
                {
                        for ( k in V )
                        {
                                if ( C[k] == T[i] )
                                {
                                        print C[k] " times"
                                        print V[k]
                                }
                        }
                }
        }
' "$1"ascii >>200condition

but with large files it showed the following error


Code:
 
gawk: cmd. line:3: (FILENAME=all.txtascii FNR=18498193) fatal: more_nodes: nextfree: can't allocate 4000 bytes of memory (Cannot allocate memory)
gawk: cmd. line:3: (FILENAME=all.txtascii FNR=18498243) fatal: more_nodes: nextfree: can't allocate 4000 bytes of memory (Cannot allocate memory)
gawk: cmd. line:3: (FILENAME=all.txtascii FNR=18498193) fatal: more_nodes: nextfree: can't allocate 4000 bytes of memory (Cannot allocate memory)


Any help?

Thanks
# 2  
Old 05-29-2013
You read in the whole file into memory here:

Code:
A[++c] = $1

It is natural outcome of a limited resource like memory, that, as the files grow larger, at some point this memory is exhausted. What follows is the allocation error you see.

Try to use a less memory-intensive algorithm.

I hope this helps.

bakunin
Login or Register to Ask a Question

Previous Thread | Next Thread

5 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to block first bash script until second bash script script launches web server/site?

I'm new to utilities like socat and netcat and I'm not clear if they will do what I need. I have a "compileDeployStartWebServer.sh" script and a "StartBrowser.sh" script that are started by emacs/elisp at the same time in two different processes. I'm using Cygwin bash on Windows 10. My... (3 Replies)
Discussion started by: siegfried
3 Replies

2. Shell Programming and Scripting

Shell script works fine as a standalone script but not as part of a bigger script

Hello all, I am facing a weird issue while executing a code below - #!/bin/bash cd /wload/baot/home/baotasa0/sandboxes_finance/ext_ukba_bde/pset sh UKBA_publish.sh UKBA 28082015 3 if then echo "Param file conversion for all the areas are completed, please check in your home directory"... (2 Replies)
Discussion started by: ektubbe
2 Replies

3. UNIX for Dummies Questions & Answers

Calling a script from master script to get value from called script

I am trying to call a script(callingscript.sh) from a master script(masterscript.sh) to get string type value from calling script to master script. I have used scripts mentioned below. #masterscript.sh ./callingscript.sh echo $fileExist #callingscript.sh echo "The script is called"... (2 Replies)
Discussion started by: Raj Roy
2 Replies

4. Shell Programming and Scripting

Script will keep checking running status of another script and also restart called script at night

I am using blow script :-- #!/bin/bash FIND=$(ps -elf | grep "snmp_trap.sh" | grep -v grep) #check snmp_trap.sh is running or not if then # echo "process found" exit 0; else echo "process not found" exec /home/Ketan_r /snmp_trap.sh 2>&1 & disown -h ... (1 Reply)
Discussion started by: ketanraut
1 Replies

5. Shell Programming and Scripting

create a shell script that calls another script and and an awk script

Hi guys I have a shell script that executes sql statemets and sends the output to a file.the script takes in parameters executes sql and sends the result to an output file. #!/bin/sh echo " $2 $3 $4 $5 $6 $7 isql -w400 -U$2 -S$5 -P$3 << xxx use $4 go print"**Changes to the table... (0 Replies)
Discussion started by: magikminox
0 Replies
Login or Register to Ask a Question