Sponsored Content
Top Forums Shell Programming and Scripting Gawk: error memory allocation in HP-UX Post 302916042 by siramitsharma on Saturday 6th of September 2014 09:03:28 AM
Old 09-06-2014
Gawk: error memory allocation in HP-UX

i have following code

Code:
gawk -F, '{FS=","}
{
if (substr($1,3,1)=="-") array[$5","$1","$5","$6","substr($1,1,2)","$2","$7","$4]++
else arraytmp[$5","$1","$5","$6","$1","$2","$7","$4]++
}
END { for (counter in array) {print counter","array[counter] }
{ for (count in arraytmp) {print count","arraytmp[count]}} }
' Infile > OutFile

the Infile has more that 400 million records, by virute of this operation outfile will be reduced to 60million.

Error coming is:
Code:
gawk: cmd. line:4: (FILENAME=- FNR=8226772) fatal: more_blocks: freep: can't allocate 4000 bytes of memory (Not enough space)

Additional Information:
Code:
$ uname -a
HP-UX smsciuc0 B.11.31 U ia64 3651711977 unlimited-user license
 
$ swapinfo
Kb Kb Kb PCT START/ Kb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 67108864 0 67108864 0% 0 - 1 /dev/vg00/lvol2
reserve - 455428 -455428
memory 63799772 8378200 55421572 13%
 
$ kctune maxdsiz maxdsiz_64bit maxssiz maxssiz_64bit maxtsiz maxtsiz_64bit
Tunable Value Expression Changes
maxdsiz 1073741824 Default Immed
maxdsiz_64bit 4294967296 Default Immed
maxssiz 8388608 Default Immed
maxssiz_64bit 268435456 Default Immed
maxtsiz 100663296 Default Immed
maxtsiz_64bit 1073741824 Default Immed

PLZ HELP
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

memory allocation

I would like to know how I could allocate some more memory to a process. Please note that I am not the root user. (1 Reply)
Discussion started by: sagar
1 Replies

2. UNIX for Advanced & Expert Users

threads and memory allocation

Hello! First of all, forgive me for bad English. When I starts new thread (pthread_create), system allocates some memory for it (for example, for thread's stack). I wonder when does it deallocate this memory? The problem is that I have a program which sometimes creates new threads and sometimes... (3 Replies)
Discussion started by: prankster
3 Replies

3. HP-UX

HP-UX memory usage allocation

Hi all, I have a HP-UX Server with 4 gigabytes of physical RAM. When I use the 'Glance' utility to see what my memory utilization is, my memory usage shows up maxed out at 99%. I shut off all the known processes that I'm running on that box and the memory utilization is still at 78% (with Swap... (3 Replies)
Discussion started by: dehuang83
3 Replies

4. Programming

Memory allocation problem

I have a program that will fetch some particular lines and store it in a buffer for further operations.The code which is given below works but with some errors.I couldn't trace out the error.Can anybody help on this plz?? #include <stdio.h> #include <stdlib.h> #include<string.h> #define... (1 Reply)
Discussion started by: vigneshinbox
1 Replies

5. Programming

Is there a problem with the memory allocation???

I have a scenario like the client has to search for the active server.There will be many servers.But not all server are active.And at a time not more than one server will be active. The client will be in active state always i.e, it should always search for an active server until it gets one.I... (1 Reply)
Discussion started by: vigneshinbox
1 Replies

6. Programming

Memory allocation in C

Hi Experts I need some help in static memory allocation in C. I have a program in which I declared 2 variables, one char array and one integer. I was little surprised to see the addresses of the variables. First: int x; char a; printf("%u %u\n', &x, a); I got the addresses displayed... (2 Replies)
Discussion started by: unx_freak
2 Replies

7. Programming

Memory Allocation Query

When we dynamically allocate the memory say 100 integers say int *x = new int(1000); then does entire chunk of memory gets allocated at once after the completion of the statement? I mean will the the concept of page fault come into picture over here? (3 Replies)
Discussion started by: rupeshkp728
3 Replies

8. Shell Programming and Scripting

memory allocation to a variable

hello all.. i'm a beginner in shell scripting. I need to know what is really happening when we are creating a variable in shell scripting? how memory is allocated for that variable? (3 Replies)
Discussion started by: aarathy
3 Replies

9. Programming

memory allocation for string in C

hi in the following code, how the memory is allocated for a1 which holds the values of a2 after cpy function call. #include <stdio.h> #include <string.h> void cpy(char* d, const char* s){ while(*d++=*s++); } main(){ char* a1; char* a2="done"; cpy(a1,a2); ... (3 Replies)
Discussion started by: mprakasheee
3 Replies

10. Programming

C++/ROOT Memory Allocation?

Hello, I am new to C++ programming, so I'm still getting a feel for things. I recently wrote a simple C++ program (to be used as a ROOT Macro) to conduct a statistical analysis of a varied version of the Monty Hall problem (code below). Basically, the programs runs a few simple calculations to... (7 Replies)
Discussion started by: Tyler_92
7 Replies
maxssiz(5)							File Formats Manual							maxssiz(5)

NAME
maxssiz, maxssiz_64bit - maximum size (in bytes) of the stack for any user process VALUES
Default Allowed values DESCRIPTION
User programs on HP-UX systems are composed of five discrete segments of virtual memory: text (or code), data, stack, shared, and I/O. Each segment occupies an architecturally defined range of the virtual address space which sets the upper limit to their size. However, text, data, and stack segments may have a smaller maximum enforced via the and tunables. and define the maximum size of the stack segment for 32-bit and 64-bit processes. The stack segment contains the actual program stack and the storage space for registers on a process or thread context switch. Who is Expected to Change This Tunable? Anyone. Restrictions on Changing Changes to this tunable take effect only for processes started after the modification. In addition, a process which modifies its for the stack segment propagates the modified limit to all child processes, thereby exempting them from any future modification of The value speci- fied is expected to be a multiple of the base page size. See the description of getconf(1) for more details. If the value specified is not a multiple of the base page size, it will be rounded down to the nearest multiple of the base page size. When Should the Value of This Tunable Be Raised? should be raised if user processes are generating the console error message: Processes generating this error message will likely terminate with the segmentation violation error and dump core. What Are the Side Effects of Raising the Value? Raising this tunable by definition allows larger stack segments for every process. This means that and function as limitations on the amount of swap space that can be reserved or used by each process. Therefore, using more virtual address space does not translate directly to using more physical address space because virtual pages can be swapped out. If swap space on the machine is near capacity, raising this tunable increases the amount of reservable swap per process. This could exhaust the swap space on the system by allowing a process with a memory leak or a malicious program that uses huge amounts of memory to reserve too much swap space. It is also important to realize that for 32 bit user processes, data and stack are located contiguously. Raising the amount of virtual address space reserved for the stack segment implies lowering the amount of virtual address space for the data segment. In other words, raising may cause user processes which use all (or nearly all) of the previously available data area to fail allocation with the error even with set above the current amount of memory allocated for data by this process. On will cause an increase in the kernel data structures used to represent the larger stack space. This may use enough additional swap space that the user may see performance degradation or application failure due to lack of reservable swap space. One method to minimize impact is to use a script which launches the applications needing maximum stack size. Within the script, raise the value of launch the application, and then lower to its previous value. This will allow the specific application to benefit from the increased stack size but will not cause additional stack growth for applications which do not need it. When Should the Value of This Tunable Be Lowered? This tunable should be lowered if swap space is at a premium on the machine and programs that are using too much swap space are affecting the execution of other critical user processes. What Are the Side Effects of Lowering the Value? Lowering this tunable will limit the amount of memory available for stack usage per process. This could cause processes with large stack requirements to terminate with a error. What Other Tunable Values Should Be Changed at the Same Time? The tunable should be considered because it also limits swap usage by process data segment. WARNINGS
All HP-UX kernel tunable parameters are release specific. This parameter may be removed or have its meaning changed in future releases of HP-UX. Installation of optional kernel software, from HP or other vendors, may cause changes to tunable parameter values. After installation, some tunable parameters may no longer be at the default or recommended values. For information about the effects of installation on tun- able values, consult the documentation for the kernel software being installed. For information about optional kernel software that was factory installed on your system, see at AUTHOR
was developed by HP. SEE ALSO
getconf(1), maxdsiz(5), maxtsiz(5). Tunable Kernel Parameters maxssiz(5)
All times are GMT -4. The time now is 10:55 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy