Sponsored Content
Top Forums Programming How to deal with lots of data in memory in order not to run out of memory Post 302320561 by jim mcnamara on Thursday 28th of May 2009 10:09:34 AM
Old 05-28-2009
Yes memory mapping counts as part of your process. But it is possible to map a file shared, and have another process actually process the file. That does not solve the memory usage problem necessarily. If you are running out of memory see how ulimit is set.

ulimit example -
Code:
/home/jmcnama> ulimit -a
time(seconds)        unlimited
file(blocks)         unlimited
data(kbytes)         2015464
stack(kbytes)        256000
memory(kbytes)       unlimited
coredump(blocks)     4194303

The two red lines apply to your question.

getrusage() returns resource usage for a parent process and it's children.
setrusage allows you to change ulimit soft values - you cannot go beyond the hard limits unless the sysadmin reconfigures your account/kernel.

Also you can increase virtual memory simply by adding swap space. Virtual memory (if ulimit for memory == unlimited) is the actual limit for process memory space.

check out vmstat for more information.
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Shared memory shortage but lots of unused memory

I am running HP-UX B.11.11. I'm increasing a parameter for a database engine so that it uses more memory to buffer the disk drive (to speed up performance). I have over 5GB of memory not being used. But when I try to start the DB with the increased buffer parameter I get told. "Not... (1 Reply)
Discussion started by: cjcamaro
1 Replies

2. Programming

how to round up a memory address(memory alignment problem)

Hi, I try to marshal a unsigned int and a char * into a buffer, and then unmarshal them later to get them out. I need to put the char * in the front and unsigned int at the end of the buffer. However, my system always give me "BUS ERROR". I am using Sun Sparcs Sloris 2.10. My code to marshal... (6 Replies)
Discussion started by: nj302
6 Replies

3. HP-UX

Order process by consumed memory

Hi! I am new to HP-UX. :o By using the command glance, I found the user memory usage was very high. I would like to know is there any command can show the process which consume most available memory ? (Just like the command top, but order by memory, not CPU) (1 Reply)
Discussion started by: alfredo
1 Replies

4. Programming

malloc gives the same memory to two different nodes. How to deal with it?

When allocating memory for two different nodes, the resulting memory are the same. Clearly, this will lead to a mistake. This happened in a function. And the process must be in a function. (gdb) p tree->list $43 = (node *) 0x8be4180 (gdb) p tree->list $44 = (node *) 0x8be4180 At the... (2 Replies)
Discussion started by: cdbug
2 Replies

5. Solaris

How to find Total and Free Physical Memory and Logical Memory in SOLARIS 9

Hi, Im working on Solaris 9 on SPARC-32 bit running on an Ultra-80, and I have to find out the following:- 1. Total Physical Memory in the system(total RAM). 2. Available Physical Memory(i.e. RAM Usage) 3. Total (Logical) Memory in the system 4. Available (Logical) Memory. I know... (4 Replies)
Discussion started by: 0ktalmagik
4 Replies

6. Solaris

restrcit physical memory with zone.max-locked-memory

Is it possible to restrict physical memory in solaris zone with zone.max-locked-memory just like we can do with rcapd ? I do not want to used rcapd (1 Reply)
Discussion started by: fugitive
1 Replies

7. AIX

Lots of page faults and free memory

Hello, I've been reading your forums for quite a while and the great amount of information I find here always come in hand.This time however, I need some specific help... I have a doubt with an AIX server which I'm failing to understand as I'm new to its concept of memory management... ... (8 Replies)
Discussion started by: flpgdt
8 Replies

8. Solaris

relationship or difference between entitled memory and locked memory

Hello solaris experts, Being new to solaris containers, from Linux, feeling difficulty in understanding certain concepts. Hope somebody can help me here. I understand that, & some questions .... Locked memory -- memory which will not be swapped out at any cause. is this for... (0 Replies)
Discussion started by: thegeek
0 Replies

9. Solaris

[DOUBT] Memory high in idle process on Solaris 10 (Memory Utilization > 90%)

Hi Experts, Our servers running Solaris 10 with SAP Application. The memory utilization always >90%, but the process on SAP is too less even nothing. Why memory utilization on solaris always looks high? I have statement about memory on solaris, is this true: Memory in solaris is used for... (4 Replies)
Discussion started by: edydsuranta
4 Replies

10. AIX

High Paging when lots of free memory AIX 5.3

I am new to AIX, I have few AIX 5.3 servers and I could see there are significant difference in paging space utilization on servers even though they are running same applications below server is working fine which shows 2-5 % paging usage throuh out the day cpu_scale_memp = 8... (12 Replies)
Discussion started by: bibish
12 Replies
BIO_s_mem(3)							      OpenSSL							      BIO_s_mem(3)

NAME
BIO_s_mem, BIO_set_mem_eof_return, BIO_get_mem_data, BIO_set_mem_buf, BIO_get_mem_ptr, BIO_new_mem_buf - memory BIO SYNOPSIS
#include <openssl/bio.h> BIO_METHOD * BIO_s_mem(void); BIO_set_mem_eof_return(BIO *b,int v) long BIO_get_mem_data(BIO *b, char **pp) BIO_set_mem_buf(BIO *b,BUF_MEM *bm,int c) BIO_get_mem_ptr(BIO *b,BUF_MEM **pp) BIO *BIO_new_mem_buf(void *buf, int len); DESCRIPTION
BIO_s_mem() return the memory BIO method function. A memory BIO is a source/sink BIO which uses memory for its I/O. Data written to a memory BIO is stored in a BUF_MEM structure which is extended as appropriate to accommodate the stored data. Any data written to a memory BIO can be recalled by reading from it. Unless the memory BIO is read only any data read from it is deleted from the BIO. Memory BIOs support BIO_gets() and BIO_puts(). If the BIO_CLOSE flag is set when a memory BIO is freed then the underlying BUF_MEM structure is also freed. Calling BIO_reset() on a read write memory BIO clears any data in it. On a read only BIO it restores the BIO to its original state and the read only data can be read again. BIO_eof() is true if no data is in the BIO. BIO_ctrl_pending() returns the number of bytes currently stored. BIO_set_mem_eof_return() sets the behaviour of memory BIO b when it is empty. If the v is zero then an empty memory BIO will return EOF (that is it will return zero and BIO_should_retry(b) will be false. If v is non zero then it will return v when it is empty and it will set the read retry flag (that is BIO_read_retry(b) is true). To avoid ambiguity with a normal positive return value v should be set to a negative value, typically -1. BIO_get_mem_data() sets pp to a pointer to the start of the memory BIOs data and returns the total amount of data available. It is implemented as a macro. BIO_set_mem_buf() sets the internal BUF_MEM structure to bm and sets the close flag to c, that is c should be either BIO_CLOSE or BIO_NOCLOSE. It is a macro. BIO_get_mem_ptr() places the underlying BUF_MEM structure in pp. It is a macro. BIO_new_mem_buf() creates a memory BIO using len bytes of data at buf, if len is -1 then the buf is assumed to be null terminated and its length is determined by strlen. The BIO is set to a read only state and as a result cannot be written to. This is useful when some data needs to be made available from a static area of memory in the form of a BIO. The supplied data is read directly from the supplied buffer: it is not copied first, so the supplied area of memory must be unchanged until the BIO is freed. NOTES
Writes to memory BIOs will always succeed if memory is available: that is their size can grow indefinitely. Every read from a read write memory BIO will remove the data just read with an internal copy operation, if a BIO contains a lots of data and it is read in small chunks the operation can be very slow. The use of a read only memory BIO avoids this problem. If the BIO must be read write then adding a buffering BIO to the chain will speed up the process. BUGS
There should be an option to set the maximum size of a memory BIO. There should be a way to "rewind" a read write BIO without destroying its contents. The copying operation should not occur after every small read of a large BIO to improve efficiency. EXAMPLE
Create a memory BIO and write some data to it: BIO *mem = BIO_new(BIO_s_mem()); BIO_puts(mem, "Hello World "); Create a read only memory BIO: char data[] = "Hello World"; BIO *mem; mem = BIO_new_mem_buf(data, -1); Extract the BUF_MEM structure from a memory BIO and then free up the BIO: BUF_MEM *bptr; BIO_get_mem_ptr(mem, &bptr); BIO_set_close(mem, BIO_NOCLOSE); /* So BIO_free() leaves BUF_MEM alone */ BIO_free(mem); SEE ALSO
TBA 50 2013-03-05 BIO_s_mem(3)
All times are GMT -4. The time now is 08:17 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy