Sponsored Content
Full Discussion: Out of memory message
Top Forums UNIX for Beginners Questions & Answers Out of memory message Post 303044752 by gimley on Tuesday 3rd of March 2020 08:43:01 AM
Old 03-03-2020
Out of memory message

I have written a script to generate trigrams. The script runs perfectly on a relatively small file. When a large file is given, around 700 KB, I get the message
Code:
Counting trigrams ... 1614375 lines done.gawk32: trigrams.gk:12: (FILENAME=urmono.txt FNR=1614376) fatal: newnode: nextfree: can't allocate memory (No error)

I have Windows 10 with 32 Gb ram. I believe DOS cannot access all that ram and hence gives the above message.
Just in case my program has an error I am giving below the program
Code:
{ 
# $0 = tolower($0)                                   
   gsub(/[.,:;!?"<>\[\]#(){}]/,"")                     
    for(i=1; i<=NF; i++){ 
       trigram = word1 " " word2" " $i         
       word1 = word2                                  
       word2 = $i                                        
       count[trigram]++                               
      }
       printf "\rCounting trigrams ... %6d lines done.", NR > "CON"
  }
END {for (w in count)
      print count[w], "\t" w

Many thanks for any solution to this problem. Truncating the file or dividing it into small chunks and running the script on them is not very satisfactory, since trigrams are repeated in different outputs and I need to combine all of them using another awk script, which makes things cumbersome.
 

10 More Discussions You Might Find Interesting

1. AIX

Printer Memory Message

Hello Everyone, I received the following (root) email. Does anyone know what causes this and how I can find the offending printer? Thanks in advance. Jim Message 2: From daemon Wed Nov 30 09:51:07 2005 Date: Wed, 30 Nov 2005 09:51:07 -0800 From: daemon To: root (2 Replies)
Discussion started by: jlslhills
2 Replies

2. Programming

shared memory and message queues

Hi, According to my understanding.. When message queues are used, when a process post a message in the queue and if another process reads it from the queue then the queue will be empty unlike shared memory where n number of processess can access the shared memory and still the contents remain... (2 Replies)
Discussion started by: rvan
2 Replies

3. Programming

How to limit max no of message in a posix message queue

Hii can anyone pls tell how to limit the max no of message in a posix message queue. I have made changes in proc/sys/fs/mqueue/msg_max But still whenever i try to read the value of max. message in the queue using attr.mq_curmsgs (where struct mq_attr attr) its giving the default value as 10.... (0 Replies)
Discussion started by: mohit3884
0 Replies

4. Shell Programming and Scripting

memory full warning message

Hi I wrote a script #!/usr/bin/ksh #set -x for fs in `df -k|awk '{print $1}'|sed -n "3,14 p"` do x=`df -kl | grep $fs | awk '{ print $5 }'` y=50% if then message="File System `df -k |grep $fs |awk '{print $6\", \"$5}'`... (1 Reply)
Discussion started by: namishtiwari
1 Replies

5. Solaris

How to find Total and Free Physical Memory and Logical Memory in SOLARIS 9

Hi, Im working on Solaris 9 on SPARC-32 bit running on an Ultra-80, and I have to find out the following:- 1. Total Physical Memory in the system(total RAM). 2. Available Physical Memory(i.e. RAM Usage) 3. Total (Logical) Memory in the system 4. Available (Logical) Memory. I know... (4 Replies)
Discussion started by: 0ktalmagik
4 Replies

6. Programming

How to deal with lots of data in memory in order not to run out of memory

Hi, I'm trying to learn how to manage memory when I have to deal with lots of data. Basically I'm indexing a huge file (5GB, but it can be bigger), by creating tables that holds offset <-> startOfSomeData information. Currently I'm mapping the whole file at once (yep!) but of course the... (1 Reply)
Discussion started by: emitrax
1 Replies

7. Solaris

restrcit physical memory with zone.max-locked-memory

Is it possible to restrict physical memory in solaris zone with zone.max-locked-memory just like we can do with rcapd ? I do not want to used rcapd (1 Reply)
Discussion started by: fugitive
1 Replies

8. Programming

POSIX Message Queue Memory Allocation

Hi, I wanted to know whether the POSIX message queues are statically allocated memory by the kernel based on the parameters specified in the open or as and when we send messages, memory are allocated? Does the kernel reserve the specified memory for the message queue irrespective of whether... (1 Reply)
Discussion started by: sumtata
1 Replies

9. Solaris

[DOUBT] Memory high in idle process on Solaris 10 (Memory Utilization > 90%)

Hi Experts, Our servers running Solaris 10 with SAP Application. The memory utilization always >90%, but the process on SAP is too less even nothing. Why memory utilization on solaris always looks high? I have statement about memory on solaris, is this true: Memory in solaris is used for... (4 Replies)
Discussion started by: edydsuranta
4 Replies

10. UNIX and Linux Applications

Ssmtp -t < /path/to/the/message.txt (How to format message.txt for html email)

ssmtp has been running well under Kubuntu 12.04.1 for plain text messages. I would like to send html messages with ssmtp -t < /path/to/the/message.txt, but I cannot seem to get the message.txt file properly formatted. I have tried various charsets, Content-Transfer-Encoding, rearranging the... (0 Replies)
Discussion started by: Ronald B
0 Replies
pods::SDL::GFX::ImageFilter(3pm)			User Contributed Perl Documentation			  pods::SDL::GFX::ImageFilter(3pm)

NAME
SDL::GFX::ImageFilter - image filtering functions CATEGORY
TODO, GFX METHODS
MMX_detect int gfx_image_MMX_detect() CODE: SDL_imageFilterMMXdetect(); MMX_off void gfx_image_MMX_off() CODE: SDL_imageFilterMMXoff(); MMX_on void gfx_image_MMX_on() CODE: SDL_imageFilterMMXon(); add int gfx_image_add(Src1, Src2, Dest, length) unsigned char *Src1 unsigned char *Src2 unsigned char *Dest int length CODE: RETVAL = SDL_imageFilterAdd(Src1, Src2, Dest, length); OUTPUT: RETVAL mean int gfx_image_mean(Src1, Src2, Dest, length) unsigned char *Src1 unsigned char *Src2 unsigned char *Dest int length CODE: RETVAL = SDL_imageFilterMean(Src1, Src2, Dest, length); OUTPUT: RETVAL sub int gfx_image_sub(Src1, Src2, Dest, length) unsigned char *Src1 unsigned char *Src2 unsigned char *Dest int length CODE: RETVAL = SDL_imageFilterSub(Src1, Src2, Dest, length); OUTPUT: RETVAL abs_diff int gfx_image_abs_diff(Src1, Src2, Dest, length) unsigned char *Src1 unsigned char *Src2 unsigned char *Dest int length CODE: RETVAL = SDL_imageFilterAbsDiff(Src1, Src2, Dest, length); OUTPUT: RETVAL mult int gfx_image_mult(Src1, Src2, Dest, length) unsigned char *Src1 unsigned char *Src2 unsigned char *Dest int length CODE: RETVAL = SDL_imageFilterMult(Src1, Src2, Dest, length); OUTPUT: RETVAL mult_nor int gfx_image_mult_nor(Src1, Src2, Dest, length) unsigned char *Src1 unsigned char *Src2 unsigned char *Dest int length CODE: RETVAL = SDL_imageFilterMultNor(Src1, Src2, Dest, length); OUTPUT: RETVAL mult_div_by_2 int gfx_image_mult_div_by_2(Src1, Src2, Dest, length) unsigned char *Src1 unsigned char *Src2 unsigned char *Dest int length CODE: RETVAL = SDL_imageFilterMultDivby2(Src1, Src2, Dest, length); OUTPUT: RETVAL mult_div_by_4 int gfx_image_mult_div_by_4(Src1, Src2, Dest, length) unsigned char *Src1 unsigned char *Src2 unsigned char *Dest int length CODE: RETVAL = SDL_imageFilterMultDivby4(Src1, Src2, Dest, length); OUTPUT: RETVAL bit_and int gfx_image_bit_and(Src1, Src2, Dest, length) unsigned char *Src1 unsigned char *Src2 unsigned char *Dest int length CODE: RETVAL = SDL_imageFilterBitAnd(Src1, Src2, Dest, length); OUTPUT: RETVAL bit_or int gfx_image_bit_or(Src1, Src2, Dest, length) unsigned char *Src1 unsigned char *Src2 unsigned char *Dest int length CODE: RETVAL = SDL_imageFilterBitOr(Src1, Src2, Dest, length); OUTPUT: RETVAL div int gfx_image_div(Src1, Src2, Dest, length) unsigned char *Src1 unsigned char *Src2 unsigned char *Dest int length CODE: RETVAL = SDL_imageFilterDiv(Src1, Src2, Dest, length); OUTPUT: RETVAL bit_negation int gfx_image_bit_negation(Src1, Dest, length) unsigned char *Src1 unsigned char *Dest int length CODE: RETVAL = SDL_imageFilterBitNegation(Src1, Dest, length); OUTPUT: RETVAL add_byte int gfx_image_add_byte(Src1, Dest, length, C) unsigned char *Src1 unsigned char *Dest int length unsigned char C CODE: RETVAL = SDL_imageFilterAddByte(Src1, Dest, length, C); OUTPUT: RETVAL add_uint int gfx_image_add_uint(Src1, Dest, length, C) unsigned char *Src1 unsigned char *Dest int length unsigned int C CODE: RETVAL = SDL_imageFilterAddUint(Src1, Dest, length, C); OUTPUT: RETVAL add_byte_to_half int gfx_image_add_byte_to_half(Src1, Dest, length, C) unsigned char *Src1 unsigned char *Dest int length unsigned char C CODE: RETVAL = SDL_imageFilterAddByteToHalf(Src1, Dest, length, C); OUTPUT: RETVAL sub_byte int gfx_image_sub_byte(Src1, Dest, length, C) unsigned char *Src1 unsigned char *Dest int length unsigned char C CODE: RETVAL = SDL_imageFilterSubByte(Src1, Dest, length, C); OUTPUT: RETVAL sub_uint int gfx_image_sub_uint(Src1, Dest, length, C) unsigned char *Src1 unsigned char *Dest int length unsigned int C CODE: RETVAL = SDL_imageFilterSubUint(Src1, Dest, length, C); OUTPUT: RETVAL shift_right int gfx_image_shift_right(Src1, Dest, length, N) unsigned char *Src1 unsigned char *Dest int length unsigned char N CODE: RETVAL = SDL_imageFilterShiftRight(Src1, Dest, length, N); OUTPUT: RETVAL shift_right_uint int gfx_image_shift_right_uint(Src1, Dest, length, N) unsigned char *Src1 unsigned char *Dest int length unsigned char N CODE: RETVAL = SDL_imageFilterShiftRightUint(Src1, Dest, length, N); OUTPUT: RETVAL mult_by_byte int gfx_image_mult_by_byte(Src1, Dest, length, C) unsigned char *Src1 unsigned char *Dest int length unsigned char C CODE: RETVAL = SDL_imageFilterMultByByte(Src1, Dest, length, C); OUTPUT: RETVAL shift_right_and_mult_by_byte int gfx_image_shift_right_and_mult_by_byte(Src1, Dest, length, N, C) unsigned char *Src1 unsigned char *Dest int length unsigned char N unsigned char C CODE: RETVAL = SDL_imageFilterShiftRightAndMultByByte(Src1, Dest, length, N, C); OUTPUT: RETVAL shift_left_byte int gfx_image_shift_left_byte(Src1, Dest, length, N) unsigned char *Src1 unsigned char *Dest int length unsigned char N CODE: RETVAL = SDL_imageFilterShiftLeftByte(Src1, Dest, length, N); OUTPUT: RETVAL shift_left_uint int gfx_image_shift_left_uint(Src1, Dest, length, N) unsigned char *Src1 unsigned char *Dest int length unsigned char N CODE: RETVAL = SDL_imageFilterShiftLeftUint(Src1, Dest, length, N); OUTPUT: RETVAL shift_left int gfx_image_shift_left(Src1, Dest, length, N) unsigned char *Src1 unsigned char *Dest int length unsigned char N CODE: RETVAL = SDL_imageFilterShiftLeft(Src1, Dest, length, N); OUTPUT: RETVAL binarize_using_threshold int gfx_image_binarize_using_threshold(Src1, Dest, length, T) unsigned char *Src1 unsigned char *Dest int length unsigned char T CODE: RETVAL = SDL_imageFilterBinarizeUsingThreshold(Src1, Dest, length, T); OUTPUT: RETVAL clip_to_range int gfx_image_clip_to_range(Src1, Dest, length, Tmin, Tmax) unsigned char *Src1 unsigned char *Dest int length unsigned char Tmin unsigned char Tmax CODE: RETVAL = SDL_imageFilterClipToRange(Src1, Dest, length, Tmin, Tmax); OUTPUT: RETVAL normalize_linear int gfx_image_normalize_linear(Src1, Dest, length, Cmin, Cmax, Nmin, Nmax) unsigned char *Src1 unsigned char *Dest int length int Cmin int Cmax int Nmin int Nmax CODE: RETVAL = SDL_imageFilterNormalizeLinear(Src1, Dest, length, Cmin, Cmax, Nmin, Nmax); OUTPUT: RETVAL convolve_kernel_3x3_divide int gfx_image_convolve_kernel_3x3_divide(Src, Dest, rows, columns, Kernel, Divisor) unsigned char *Src unsigned char *Dest int rows int columns Sint16 *Kernel unsigned char Divisor CODE: RETVAL = SDL_imageFilterConvolveKernel3x3Divide(Src, Dest, rows, columns, Kernel, Divisor); OUTPUT: RETVAL convolve_kernel_5x5_divide int gfx_image_convolve_kernel_5x5_divide(Src, Dest, rows, columns, Kernel, Divisor) unsigned char *Src unsigned char *Dest int rows int columns Sint16 *Kernel unsigned char Divisor CODE: RETVAL = SDL_imageFilterConvolveKernel5x5Divide(Src, Dest, rows, columns, Kernel, Divisor); OUTPUT: RETVAL convolve_kernel_7x7_divide int gfx_image_convolve_kernel_7x7_divide(Src, Dest, rows, columns, Kernel, Divisor) unsigned char *Src unsigned char *Dest int rows int columns Sint16 *Kernel unsigned char Divisor CODE: RETVAL = SDL_imageFilterConvolveKernel7x7Divide(Src, Dest, rows, columns, Kernel, Divisor); OUTPUT: RETVAL convolve_kernel_9x9_divide int gfx_image_convolve_kernel_9x9_divide(Src, Dest, rows, columns, Kernel, Divisor) unsigned char *Src unsigned char *Dest int rows int columns Sint16 *Kernel unsigned char Divisor CODE: RETVAL = SDL_imageFilterConvolveKernel9x9Divide(Src, Dest, rows, columns, Kernel, Divisor); OUTPUT: RETVAL convolve_kernel_3x3_shift_right int gfx_image_convolve_kernel_3x3_shift_right(Src, Dest, rows, columns, Kernel, NRightShift) unsigned char *Src unsigned char *Dest int rows int columns Sint16 *Kernel unsigned char NRightShift CODE: RETVAL = SDL_imageFilterConvolveKernel3x3ShiftRight(Src, Dest, rows, columns, Kernel, NRightShift); OUTPUT: RETVAL convolve_kernel_5x5_shift_right int gfx_image_convolve_kernel_5x5_shift_right(Src, Dest, rows, columns, Kernel, NRightShift) unsigned char *Src unsigned char *Dest int rows int columns Sint16 *Kernel unsigned char NRightShift CODE: RETVAL = SDL_imageFilterConvolveKernel5x5ShiftRight(Src, Dest, rows, columns, Kernel, NRightShift); OUTPUT: RETVAL convolve_kernel_7x7_shift_right int gfx_image_convolve_kernel_7x7_shift_right(Src, Dest, rows, columns, Kernel, NRightShift) unsigned char *Src unsigned char *Dest int rows int columns Sint16 *Kernel unsigned char NRightShift CODE: RETVAL = SDL_imageFilterConvolveKernel7x7ShiftRight(Src, Dest, rows, columns, Kernel, NRightShift); OUTPUT: RETVAL convolve_kernel_9x9_shift_right int gfx_image_convolve_kernel_9x9_shift_right(Src, Dest, rows, columns, Kernel, NRightShift) unsigned char *Src unsigned char *Dest int rows int columns Sint16 *Kernel unsigned char NRightShift CODE: RETVAL = SDL_imageFilterConvolveKernel9x9ShiftRight(Src, Dest, rows, columns, Kernel, NRightShift); OUTPUT: RETVAL sobel_x int gfx_image_sobel_x(Src, Dest, rows, columns) unsigned char *Src unsigned char *Dest int rows int columns CODE: RETVAL = SDL_imageFilterSobelX(Src, Dest, rows, columns); OUTPUT: RETVAL sobel_x_shift_right int gfx_image_sobel_x_shift_right(Src, Dest, rows, columns, NRightShift) unsigned char *Src unsigned char *Dest int rows int columns unsigned char NRightShift CODE: RETVAL = SDL_imageFilterSobelXShiftRight(Src, Dest, rows, columns, NRightShift); OUTPUT: RETVAL align_stack void gfx_image_align_stack() CODE: SDL_imageFilterAlignStack(); restore_stack void gfx_image_restore_stack() CODE: SDL_imageFilterRestoreStack(); AUTHORS
See "AUTHORS" in SDL. perl v5.14.2 2012-05-28 pods::SDL::GFX::ImageFilter(3pm)
All times are GMT -4. The time now is 02:56 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy