Out of memory message Post: 303044752

Sponsored Content

Top Forums UNIX for Beginners Questions & Answers Out of memory message Post 303044752 by gimley on Tuesday 3rd of March 2020 08:43:01 AM

03-03-2020

Registered User

Out of memory message

I have written a script to generate trigrams. The script runs perfectly on a relatively small file. When a large file is given, around 700 KB, I get the message

Code:

Counting trigrams ... 1614375 lines done.gawk32: trigrams.gk:12: (FILENAME=urmono.txt FNR=1614376) fatal: newnode: nextfree: can't allocate memory (No error)

I have Windows 10 with 32 Gb ram. I believe DOS cannot access all that ram and hence gives the above message.
Just in case my program has an error I am giving below the program

Code:

{ 
# $0 = tolower($0)                                   
   gsub(/[.,:;!?"<>\[\]#(){}]/,"")                     
    for(i=1; i<=NF; i++){ 
       trigram = word1 " " word2" " $i         
       word1 = word2                                  
       word2 = $i                                        
       count[trigram]++                               
      }
       printf "\rCounting trigrams ... %6d lines done.", NR > "CON"
  }
END {for (w in count)
      print count[w], "\t" w

Many thanks for any solution to this problem. Truncating the file or dividing it into small chunks and running the script on them is not very satisfactory, since trigrams are repeated in different outputs and I need to combine all of them using another awk script, which makes things cumbersome.

gimley

View Public Profile for gimley

Find all posts by gimley

10 More Discussions You Might Find Interesting

1. AIX

Printer Memory Message

Hello Everyone, I received the following (root) email. Does anyone know what causes this and how I can find the offending printer? Thanks in advance. Jim Message 2: From daemon Wed Nov 30 09:51:07 2005 Date: Wed, 30 Nov 2005 09:51:07 -0800 From: daemon To: root

2. Programming

shared memory and message queues

Hi, According to my understanding.. When message queues are used, when a process post a message in the queue and if another process reads it from the queue then the queue will be empty unlike shared memory where n number of processess can access the shared memory and still the contents remain...

3. Programming

How to limit max no of message in a posix message queue

Hii can anyone pls tell how to limit the max no of message in a posix message queue. I have made changes in proc/sys/fs/mqueue/msg_max But still whenever i try to read the value of max. message in the queue using attr.mq_curmsgs (where struct mq_attr attr) its giving the default value as 10....

4. Shell Programming and Scripting

memory full warning message

Hi I wrote a script #!/usr/bin/ksh #set -x for fs in `df -k|awk '{print $1}'|sed -n "3,14 p"` do x=`df -kl | grep $fs | awk '{ print $5 }'` y=50% if then message="File System `df -k |grep $fs |awk '{print $6\", \"$5}'`...

5. Solaris

How to find Total and Free Physical Memory and Logical Memory in SOLARIS 9

Hi, Im working on Solaris 9 on SPARC-32 bit running on an Ultra-80, and I have to find out the following:- 1. Total Physical Memory in the system(total RAM). 2. Available Physical Memory(i.e. RAM Usage) 3. Total (Logical) Memory in the system 4. Available (Logical) Memory. I know...

6. Programming

How to deal with lots of data in memory in order not to run out of memory

Hi, I'm trying to learn how to manage memory when I have to deal with lots of data. Basically I'm indexing a huge file (5GB, but it can be bigger), by creating tables that holds offset <-> startOfSomeData information. Currently I'm mapping the whole file at once (yep!) but of course the...

7. Solaris

restrcit physical memory with zone.max-locked-memory

Is it possible to restrict physical memory in solaris zone with zone.max-locked-memory just like we can do with rcapd ? I do not want to used rcapd

8. Programming

POSIX Message Queue Memory Allocation

Hi, I wanted to know whether the POSIX message queues are statically allocated memory by the kernel based on the parameters specified in the open or as and when we send messages, memory are allocated? Does the kernel reserve the specified memory for the message queue irrespective of whether...

9. Solaris

[DOUBT] Memory high in idle process on Solaris 10 (Memory Utilization > 90%)

Hi Experts, Our servers running Solaris 10 with SAP Application. The memory utilization always >90%, but the process on SAP is too less even nothing. Why memory utilization on solaris always looks high? I have statement about memory on solaris, is this true: Memory in solaris is used for...

10. UNIX and Linux Applications

Ssmtp -t < /path/to/the/message.txt (How to format message.txt for html email)

ssmtp has been running well under Kubuntu 12.04.1 for plain text messages. I would like to send html messages with ssmtp -t < /path/to/the/message.txt, but I cannot seem to get the message.txt file properly formatted. I have tried various charsets, Content-Transfer-Encoding, rearranging the...

LEARN ABOUT LINUX

irsend

IRSEND(1)							   User Commands							 IRSEND(1)

NAME

       irsend - basic LIRC program to send infra-red commands

SYNOPSIS

       irsend [options] DIRECTIVE REMOTE CODE [CODE...]

DESCRIPTION

       Asks the lircd daemon to send one or more CIR (Consumer Infra-Red) commands. This is intended for remote control of electronic devices such
       as TV boxes, HiFi sets, etc.

       DIRECTIVE can be:
	  SEND_ONCE	    - send CODE [CODE ...] once
	  SEND_START	    - start repeating CODE
	  SEND_STOP	    - stop repeating CODE
	  LIST		    - list configured remote items
	  SET_TRANSMITTERS  - set transmitters NUM [NUM ...]
	  SIMULATE	    - simulate IR event

       REMOTE is the name of a remote, as described in the lircd configuration file.

       CODE is the name of a remote control key of REMOTE, as it appears in the lircd configuration file.

       NUM is the transmitter number of the hardware device.

       For the LIST DIRECTIVE, REMOTE and/or CODE can be empty:

	  LIST	 ""    ""   - list all configured remote names
	  LIST REMOTE  ""   - list all codes of REMOTE
	  LIST REMOTE CODE  - list only CODE of REMOTE

       The SIMULATE command only works if it has been explicitly enabled in lircd.

       -h --help
	      display usage summary

       -v --version
	      display version

       -d --device
	      use given lircd socket [/var/run/lirc/lircd]

       -a --address=host[:port]
	      connect to lircd at this address

       -# --count=n
	      send command n times

EXAMPLES

	  irsend LIST DenonTuner ""
	  irsend SEND_ONCE  DenonTuner PROG-SCAN
	  irsend SEND_ONCE  OnkyoAmpli VOL-UP VOL-UP VOL-UP VOL-UP
	  irsend SEND_START OnkyoAmpli VOL-DOWN ; sleep 3
	  irsend SEND_STOP  OnkyoAmpli VOL-DOWN
	  irsend SET_TRANSMITTERS 1
	  irsend SET_TRANSMITTERS 1 3 4
	  irsend SIMULATE "0000000000000476 00 OK TECHNISAT_ST3004S"

FILES

       /etc/lirc/lircd.conf
	      Default lircd configuration file.  It should contain all the remotes, their infra-red codes and the corresponding timing	and  wave-
	      form details.

DIAGNOSTICS

       If lircd is not running (or /var/run/lirc/lircd lacks write permissions) irsend aborts with the following diagnostics:
       "irsend: could not connect to socket"
       "irsend: Connection refused" (or "Permission denied").

SEE ALSO

       The documentation for lirc is maintained as html pages. They are located under html/ in the documentation directory.

       lircd(8), mode2(1), smode2(1), xmode2(1), irrecord(1), irw(1), http://www.lirc.org.

irsend 0.9.0-pre1						   October 2010 							 IRSEND(1)

10 More Discussions You Might Find Interesting

1. AIX

Printer Memory Message

Discussion started by: jlslhills

2. Programming

shared memory and message queues

Discussion started by: rvan

3. Programming

How to limit max no of message in a posix message queue

Discussion started by: mohit3884

4. Shell Programming and Scripting

memory full warning message

Discussion started by: namishtiwari