Sponsored Content
Top Forums Shell Programming and Scripting Help with implementing available memory status script Post 302808815 by DGPickett on Friday 17th of May 2013 02:59:07 PM
Old 05-17-2013
This is a recurring stress dream from the pre-UNIX, pre-MULTIX O/S including many IBM O/S. UNIX dies of low swap space, but low RAM is quite desirable (you paid for it, so sw that uses it all, all the time is great), since it is a cache for things on disk. When a process exits, it creates some free pages from freed swap or could-swap-but-swap-never-allocated pages, but mmap() pages like linked dynamic libraries remain in RAM awaiting other mmap() of the same file, like libc.so. If lots of stuff ends and nothing new runs, the free RAM may be substantial and persistent, but it just says you are not using the RAM and the whole machine fully, in ways that exploit the RAM you bought. Just booted systems can be a bit slugish as RAM pages get written from disk, then perk up as more of the pages needed for each new process are already in RAM. A process that mmap()'s files and reads them in excess can roll every other app's pages from RAM, a vulnerability; when it stops, they all gradually return to full responsiveness as they roll in their pages.

Some O/S map swap and /tmp together, so either files in /tmp or many programs malloc()ing lots of space (new'ing objects) can draw down the swap space to dangerous levels. You can make more swap using any local files space using mkswap or the like, in an emergency, but usually the trick is to kill the hog. Sometimes a programmer needs to do things in file space or with mmap() not malloc(), or needs to not run 983 parallel processes, but 32 at a time until all are run (I like 2x the CPU Core count, assuming 50% i/o bound processing). Disk is so cheap, so not being stingy on swap is a great direction, too. Fussing over lack of VM is not worth your time.

Last edited by DGPickett; 05-17-2013 at 04:07 PM..
 

10 More Discussions You Might Find Interesting

1. Solaris

Process Status Without Shared Memory

Hello Folks, On a Solaris 8 system I am trying to diagnosis memory usage of the running processes. I've been using prstat, and top , but the outputs of those include the shared memory used by the process. Hence, more a dozen of running processes are using the shared memory, and the total memory... (1 Reply)
Discussion started by: altinkaya
1 Replies

2. HP-UX

Memory dimm status

Hi, How to check the memory dimm status from OS leavel!! My system model HP N4000. Thanks, ARumugam. (2 Replies)
Discussion started by: arumsun
2 Replies

3. Programming

Solaris CPU/memory status monitoring (Shell script or c++)

i'm trying to find a way to monitor the CPU/Memory status of a solaris station using vmstat. I like to write a small script to periodically run vmstat and store the output. Can anyone show me how (preferrably in C++ if possible)? Thanks in advance. (6 Replies)
Discussion started by: shingpui
6 Replies

4. Shell Programming and Scripting

Implementing Ctrl +C in a script as a part of automation

Hi, I am working on Solaris SPARC where we used start the BEA WebLogic Server for some project module. The server is started very simply by changing to its home directory and starting the script called ./startWebLogic.sh. Upto this no big deal. Once the server get started and comes to... (6 Replies)
Discussion started by: bhaskar_m
6 Replies

5. AIX

Memory status

hi, how to checck memory and paging utilization in mb in aix 6.1 vmstat output is not clear thanks to all (3 Replies)
Discussion started by: sunnybee
3 Replies

6. UNIX for Advanced & Expert Users

prtdiag -v problem :Memory Module Groups status not Showing

Hi Friends, I need a help from you all. In my machine which is on Solaris 9. the command prtdiag -v shows the complete output but it doesn't show "Memory Module Groups status" status. I have tried restarting the picl daemon, but still it doesn't work. Memory Module Groups:... (2 Replies)
Discussion started by: vivek.goel.piet
2 Replies

7. Shell Programming and Scripting

Shell script to find Memory status in AIX

Hi All, There is a shell script that captures Memory status in AIX 6.1 64 bits! I need it to be validated by shell script experts for the following: Shell Script: cat memusageAIX.sh #!/usr/bin/ksh # # Memory usage under AIX # USED=`svmon -G | head -2 | tail -1 | awk '{ print $3... (1 Reply)
Discussion started by: a1_win
1 Replies

8. Shell Programming and Scripting

Script will keep checking running status of another script and also restart called script at night

I am using blow script :-- #!/bin/bash FIND=$(ps -elf | grep "snmp_trap.sh" | grep -v grep) #check snmp_trap.sh is running or not if then # echo "process found" exit 0; else echo "process not found" exec /home/Ketan_r /snmp_trap.sh 2>&1 & disown -h ... (1 Reply)
Discussion started by: ketanraut
1 Replies

9. Shell Programming and Scripting

Need help implementing a timout in my Shell Script for RHEL6

Hey Guys, My problem: I have a script that will be querying the database every minute to see if it gets a response, the response its querying for is "UP" in a table i made called dbup in the database. Now, I am trying to add the component to implement a timeout if the script does not get a... (2 Replies)
Discussion started by: mo_VERTICASQL
2 Replies

10. Shell Programming and Scripting

Help with delaying script and implementing checks before completion

Hello everyone, I was wondering if any of you could help me with this. I am an absolute beginner and don't know how to program, but I can follow a tutorial and tweak code sometimes. My understanding of programing is limitted to what for and while loops do, and how if then else logic works. That... (2 Replies)
Discussion started by: tomeurp
2 Replies
MLOCK(2)						     Linux Programmer's Manual							  MLOCK(2)

NAME
mlock, munlock, mlockall, munlockall - lock and unlock memory SYNOPSIS
#include <sys/mman.h> int mlock(const void *addr, size_t len); int munlock(const void *addr, size_t len); int mlockall(int flags); int munlockall(void); DESCRIPTION
mlock() and mlockall() respectively lock part or all of the calling process's virtual address space into RAM, preventing that memory from being paged to the swap area. munlock() and munlockall() perform the converse operation, respectively unlocking part or all of the calling process's virtual address space, so that pages in the specified virtual address range may once more to be swapped out if required by the kernel memory manager. Memory locking and unlocking are performed in units of whole pages. mlock() and munlock() mlock() locks pages in the address range starting at addr and continuing for len bytes. All pages that contain a part of the specified address range are guaranteed to be resident in RAM when the call returns successfully; the pages are guaranteed to stay in RAM until later unlocked. munlock() unlocks pages in the address range starting at addr and continuing for len bytes. After this call, all pages that contain a part of the specified memory range can be moved to external swap space again by the kernel. mlockall() and munlockall() mlockall() locks all pages mapped into the address space of the calling process. This includes the pages of the code, data and stack seg- ment, as well as shared libraries, user space kernel data, shared memory, and memory-mapped files. All mapped pages are guaranteed to be resident in RAM when the call returns successfully; the pages are guaranteed to stay in RAM until later unlocked. The flags argument is constructed as the bitwise OR of one or more of the following constants: MCL_CURRENT Lock all pages which are currently mapped into the address space of the process. MCL_FUTURE Lock all pages which will become mapped into the address space of the process in the future. These could be for instance new pages required by a growing heap and stack as well as new memory mapped files or shared memory regions. If MCL_FUTURE has been specified, then a later system call (e.g., mmap(2), sbrk(2), malloc(3)), may fail if it would cause the number of locked bytes to exceed the permitted maximum (see below). In the same circumstances, stack growth may likewise fail: the kernel will deny stack expansion and deliver a SIGSEGV signal to the process. munlockall() unlocks all pages mapped into the address space of the calling process. RETURN VALUE
On success these system calls return 0. On error, -1 is returned, errno is set appropriately, and no changes are made to any locks in the address space of the process. ERRORS
ENOMEM (Linux 2.6.9 and later) the caller had a nonzero RLIMIT_MEMLOCK soft resource limit, but tried to lock more memory than the limit permitted. This limit is not enforced if the process is privileged (CAP_IPC_LOCK). ENOMEM (Linux 2.4 and earlier) the calling process tried to lock more than half of RAM. EPERM (Linux 2.6.9 and later) the caller was not privileged (CAP_IPC_LOCK) and its RLIMIT_MEMLOCK soft resource limit was 0. EPERM (Linux 2.6.8 and earlier) The calling process has insufficient privilege to call munlockall(). Under Linux the CAP_IPC_LOCK capa- bility is required. For mlock() and munlock(): EAGAIN Some or all of the specified address range could not be locked. EINVAL len was negative. EINVAL (Not on Linux) addr was not a multiple of the page size. ENOMEM Some of the specified address range does not correspond to mapped pages in the address space of the process. For mlockall(): EINVAL Unknown flags were specified. For munlockall(): EPERM (Linux 2.6.8 and earlier) The caller was not privileged (CAP_IPC_LOCK). CONFORMING TO
POSIX.1-2001, SVr4. AVAILABILITY
On POSIX systems on which mlock() and munlock() are available, _POSIX_MEMLOCK_RANGE is defined in <unistd.h> and the number of bytes in a page can be determined from the constant PAGESIZE (if defined) in <limits.h> or by calling sysconf(_SC_PAGESIZE). On POSIX systems on which mlockall() and munlockall() are available, _POSIX_MEMLOCK is defined in <unistd.h> to a value greater than 0. (See also sysconf(3).) NOTES
Memory locking has two main applications: real-time algorithms and high-security data processing. Real-time applications require determin- istic timing, and, like scheduling, paging is one major cause of unexpected program execution delays. Real-time applications will usually also switch to a real-time scheduler with sched_setscheduler(2). Cryptographic security software often handles critical bytes like pass- words or secret keys as data structures. As a result of paging, these secrets could be transferred onto a persistent swap store medium, where they might be accessible to the enemy long after the security software has erased the secrets in RAM and terminated. (But be aware that the suspend mode on laptops and some desktop computers will save a copy of the system's RAM to disk, regardless of memory locks.) Real-time processes that are using mlockall() to prevent delays on page faults should reserve enough locked stack pages before entering the time-critical section, so that no page fault can be caused by function calls. This can be achieved by calling a function that allocates a sufficiently large automatic variable (an array) and writes to the memory occupied by this array in order to touch these stack pages. This way, enough pages will be mapped for the stack and can be locked into RAM. The dummy writes ensure that not even copy-on-write page faults can occur in the critical section. Memory locks are not inherited by a child created via fork(2) and are automatically removed (unlocked) during an execve(2) or when the process terminates. The memory lock on an address range is automatically removed if the address range is unmapped via munmap(2). Memory locks do not stack, that is, pages which have been locked several times by calls to mlock() or mlockall() will be unlocked by a sin- gle call to munlock() for the corresponding range or by munlockall(). Pages which are mapped to several locations or by several processes stay locked into RAM as long as they are locked at least at one location or by at least one process. Linux Notes Under Linux, mlock() and munlock() automatically round addr down to the nearest page boundary. However, POSIX.1-2001 allows an implementa- tion to require that addr is page aligned, so portable applications should ensure this. The VmLck field of the Linux-specific /proc/PID/status file shows how many kilobytes of memory the calling process has locked using mlock(), mlockall(), shmctl(2) SHM_LOCK, and mmap(2) MAP_LOCKED. Limits and permissions In Linux 2.6.8 and earlier, a process must be privileged (CAP_IPC_LOCK) in order to lock memory and the RLIMIT_MEMLOCK soft resource limit defines a limit on how much memory the process may lock. Since Linux 2.6.9, no limits are placed on the amount of memory that a privileged process can lock and the RLIMIT_MEMLOCK soft resource limit instead defines a limit on how much memory an unprivileged process may lock. BUGS
In the 2.4 series Linux kernels up to and including 2.4.17, a bug caused the mlockall() MCL_FUTURE flag to be inherited across a fork(2). This was rectified in kernel 2.4.18. Since kernel 2.6.9, if a privileged process calls mlockall(MCL_FUTURE) and later drops privileges (loses the CAP_IPC_LOCK capability by, for example, setting its effective UID to a nonzero value), then subsequent memory allocations (e.g., mmap(2), brk(2)) will fail if the RLIMIT_MEMLOCK resource limit is encountered. SEE ALSO
mmap(2), setrlimit(2), shmctl(2), sysconf(3), proc(5), capabilities(7) COLOPHON
This page is part of release 3.25 of the Linux man-pages project. A description of the project, and information about reporting bugs, can be found at http://www.kernel.org/doc/man-pages/. Linux 2010-03-05 MLOCK(2)
All times are GMT -4. The time now is 01:12 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy