Sponsored Content
Top Forums Programming How to deal with lots of data in memory in order not to run out of memory Post 302320561 by jim mcnamara on Thursday 28th of May 2009 10:09:34 AM
Old 05-28-2009
Yes memory mapping counts as part of your process. But it is possible to map a file shared, and have another process actually process the file. That does not solve the memory usage problem necessarily. If you are running out of memory see how ulimit is set.

ulimit example -
Code:
/home/jmcnama> ulimit -a
time(seconds)        unlimited
file(blocks)         unlimited
data(kbytes)         2015464
stack(kbytes)        256000
memory(kbytes)       unlimited
coredump(blocks)     4194303

The two red lines apply to your question.

getrusage() returns resource usage for a parent process and it's children.
setrusage allows you to change ulimit soft values - you cannot go beyond the hard limits unless the sysadmin reconfigures your account/kernel.

Also you can increase virtual memory simply by adding swap space. Virtual memory (if ulimit for memory == unlimited) is the actual limit for process memory space.

check out vmstat for more information.
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Shared memory shortage but lots of unused memory

I am running HP-UX B.11.11. I'm increasing a parameter for a database engine so that it uses more memory to buffer the disk drive (to speed up performance). I have over 5GB of memory not being used. But when I try to start the DB with the increased buffer parameter I get told. "Not... (1 Reply)
Discussion started by: cjcamaro
1 Replies

2. Programming

how to round up a memory address(memory alignment problem)

Hi, I try to marshal a unsigned int and a char * into a buffer, and then unmarshal them later to get them out. I need to put the char * in the front and unsigned int at the end of the buffer. However, my system always give me "BUS ERROR". I am using Sun Sparcs Sloris 2.10. My code to marshal... (6 Replies)
Discussion started by: nj302
6 Replies

3. HP-UX

Order process by consumed memory

Hi! I am new to HP-UX. :o By using the command glance, I found the user memory usage was very high. I would like to know is there any command can show the process which consume most available memory ? (Just like the command top, but order by memory, not CPU) (1 Reply)
Discussion started by: alfredo
1 Replies

4. Programming

malloc gives the same memory to two different nodes. How to deal with it?

When allocating memory for two different nodes, the resulting memory are the same. Clearly, this will lead to a mistake. This happened in a function. And the process must be in a function. (gdb) p tree->list $43 = (node *) 0x8be4180 (gdb) p tree->list $44 = (node *) 0x8be4180 At the... (2 Replies)
Discussion started by: cdbug
2 Replies

5. Solaris

How to find Total and Free Physical Memory and Logical Memory in SOLARIS 9

Hi, Im working on Solaris 9 on SPARC-32 bit running on an Ultra-80, and I have to find out the following:- 1. Total Physical Memory in the system(total RAM). 2. Available Physical Memory(i.e. RAM Usage) 3. Total (Logical) Memory in the system 4. Available (Logical) Memory. I know... (4 Replies)
Discussion started by: 0ktalmagik
4 Replies

6. Solaris

restrcit physical memory with zone.max-locked-memory

Is it possible to restrict physical memory in solaris zone with zone.max-locked-memory just like we can do with rcapd ? I do not want to used rcapd (1 Reply)
Discussion started by: fugitive
1 Replies

7. AIX

Lots of page faults and free memory

Hello, I've been reading your forums for quite a while and the great amount of information I find here always come in hand.This time however, I need some specific help... I have a doubt with an AIX server which I'm failing to understand as I'm new to its concept of memory management... ... (8 Replies)
Discussion started by: flpgdt
8 Replies

8. Solaris

relationship or difference between entitled memory and locked memory

Hello solaris experts, Being new to solaris containers, from Linux, feeling difficulty in understanding certain concepts. Hope somebody can help me here. I understand that, & some questions .... Locked memory -- memory which will not be swapped out at any cause. is this for... (0 Replies)
Discussion started by: thegeek
0 Replies

9. Solaris

[DOUBT] Memory high in idle process on Solaris 10 (Memory Utilization > 90%)

Hi Experts, Our servers running Solaris 10 with SAP Application. The memory utilization always >90%, but the process on SAP is too less even nothing. Why memory utilization on solaris always looks high? I have statement about memory on solaris, is this true: Memory in solaris is used for... (4 Replies)
Discussion started by: edydsuranta
4 Replies

10. AIX

High Paging when lots of free memory AIX 5.3

I am new to AIX, I have few AIX 5.3 servers and I could see there are significant difference in paging space utilization on servers even though they are running same applications below server is working fine which shows 2-5 % paging usage throuh out the day cpu_scale_memp = 8... (12 Replies)
Discussion started by: bibish
12 Replies
limit(1)																  limit(1)

NAME
limit, ulimit, unlimit - set or get limitations on the system resources available to the current shell and its descendents SYNOPSIS
/usr/bin/ulimit [-f] [blocks] sh ulimit [ - [HS] [ a | cdfnstv]] ulimit [ - [HS] [ c | d | f | n | s | t | v]] limit csh limit [-h] [ resource [limit]] unlimit [-h] [resource] ksh ulimit [-HSacdfnstv] [limit] /usr/bin/ulimit The ulimit utility sets or reports the file-size writing limit imposed on files written by the shell and its child processes (files of any size may be read). Only a process with appropriate privileges can increase the limit. sh The Bourne shell built-in function, ulimit, prints or sets hard or soft resource limits. These limits are described in getrlimit(2). If limit is not present, ulimit prints the specified limits. Any number of limits may be printed at one time. The -a option prints all lim- its. If limit is present, ulimit sets the specified limit to limit. The string unlimited requests the largest valid limit. Limits may be set for only one resource at a time. Any user may set a soft limit to any value below the hard limit. Any user may lower a hard limit. Only a super-user may raise a hard limit. See su(1M). The -H option specifies a hard limit. The -S option specifies a soft limit. If neither option is specified, ulimit will set both limits and print the soft limit. The following options specify the resource whose limits are to be printed or set. If no option is specified, the file size limit is printed or set. -c maximum core file size (in 512-byte blocks) -d maximum size of data segment or heap (in kbytes) -f maximum file size (in 512-byte blocks) -n maximum file descriptor plus 1 -s maximum size of stack segment (in kbytes) -t maximum CPU time (in seconds) -v maximum size of virtual memory (in kbytes) csh The C-shell built-in function, limit, limits the consumption by the current process or any process it spawns, each not to exceed limit on the specified resource. If limit is omitted, print the current limit; if resource is omitted, display all limits. -h Use hard limits instead of the current limits. Hard limits impose a ceiling on the values of the current limits. Only the privi- leged user may raise the hard limits. resource is one of: cputime Maximum CPU seconds per process. filesize Largest single file allowed. Limited to the size of the filesystem (see df(1M)). datasize The maximum size of a process's heap in kilobytes. stacksize Maximum stack size for the process. The default stack size is 2**64. coredumpsize Maximum size of a core dump (file). This is limited to the size of the filesystem. descriptors Maximum number of file descriptors. Run the sysdef(1M) command to obtain the maximum possible limits for your system. The values reported are in hexadecimal, but can be translated into decimal numbers using the bc(1) command. memorysize Maximum size of virtual memory. limit is a number, with an optional scaling factor, as follows: nh Hours (for cputime). nk n kilobytes. This is the default for all but cputime. nm n megabytes or minutes (for cputime). mm:ss Minutes and seconds (for cputime). unlimit removes a limitation on resource. If no resource is specified, then all resource limitations are removed. See the description of the limit command for the list of resource names. -h Remove corresponding hard limits. Only the privileged user may do this. ksh The Korn shell built-in function, ulimit, sets or displays a resource limit. The available resources limits are listed below. Many systems do not contain one or more of these limits. The limit for a specified resource is set when limit is specified. The value of limit can be a number in the unit specified below with each resource, or the value unlimited. The -H and -S flags specify whether the hard limit or the soft limit for the given resource is set. A hard limit cannot be increased once it is set. A soft limit can be increased up to the value of the hard limit. If neither the -H or -S options is specified, the limit applies to both. The current resource limit is printed when limit is omitted. In this case, the soft limit is printed unless -H is specified. When more than one resource is specified, then the limit name and unit is printed before the value. -a Lists all of the current resource limits. -c The number of 512-byte blocks on the size of core dumps. -d The number of K-bytes on the size of the data area. -f The number of 512-byte blocks on files written by child processes (files of any size may be read). -n The number of file descriptors plus 1. -s The number of K-bytes on the size of the stack area. -t The number of seconds (CPU time) to be used by each process. -v The number of K-bytes for virtual memory. If no option is given, -f is assumed. Per-Shell Memory Parameters The heapsize, datasize, and stacksize parameters are not system tunables. The only controls for these are hard limits, set in a shell startup file, or system-wide soft limits, which, for the current version of the Solaris OS, is 2**64bytes. The following option is supported by ulimit: -f Sets (or reports, if no blocks operand is present), the file size limit in blocks. The -f option is also the default case. The following operand is supported by ulimit: blocks The number of 512-byte blocks to use as the new file size limit. /usr/bin/ulimit Example 1: Limiting the Stack Size The following example limits the stack size to 512 kilobytes: example% ulimit -s 512 example% ulimit -a time(seconds) unlimited file(blocks) 100 data(kbytes) 523256 stack(kbytes) 512 coredump(blocks) 200 nofiles(descriptors) 64 memory(kbytes) unlimited sh/ksh Example 2: Limiting the Number of File Descriptors The following command limits the number of file descriptors to 12: example$ ulimit -n 12 example$ ulimit -a time(seconds) unlimited file(blocks) 41943 data(kbytes) 523256 stack(kbytes) 8192 coredump(blocks) 200 nofiles(descriptors) 12 vmemory(kbytes) unlimited csh Example 3: Limiting the Core Dump File Size The following command limits the size of a core dump file size to 0 kilobytes: example% limit coredumpsize 0 example% limit cputime unlimited filesize unlimited datasize 523256 kbytes stacksize 8192 kbytes coredumpsize 0 kbytes descriptors 64 memorysize unlimited Example 4: Removing the limitation for core file size The following command removes the above limitation for the core file size: example% unlimit coredumpsize example% limit cputime unlimited filesize unlimited datasize 523256 kbytes stacksize 8192 kbytes coredumpsize unlimited descriptors 64 memorysize unlimited See environ(5) for descriptions of the following environment variables that affect the execution of ulimit: LANG, LC_ALL, LC_CTYPE, LC_MES- SAGES, and NLSPATH. The following exit values are returned by ulimit: 0 Successful completion. >0 A request for a higher limit was rejected or an error occurred. See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ |Interface Stability |Standard | +-----------------------------+-----------------------------+ bc(1), csh(1), ksh(1), sh(1), df(1M), su(1M), swap(1M), sysdef(1M), getrlimit(2), attributes(5), environ(5), standards(5) 19 Aug 2005 limit(1)
All times are GMT -4. The time now is 05:09 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy