11-02-2008
i-nodes - out of disk space on /tmp
Usage:
We run test build jobs that login to our AIX machines and create many small files in /tmp. After these jobs complete, they delete their temporary files that they have created.
Issue:
After approximately a week /tmp appears to become full. Issuing the command “df –g /tmp” shows that there is free disk space (almost the entire disk is free) however the “%Iused” displays 99%, which prevents us from writing any new files to /tmp.
Workaround:
The only solution I’ve been able to find, is to reboot all of our AIX machines, once a week.
Environment:
Operating System Model Series CPU Type
AIX 5300-07-03-0811 JS20 Blade PowerPC_970FX
AIX 5200-08-00 JS21 Blade PowerPC_970MP
AIX 5200-08-00 JS21 Blade PowerPC_970MP
AIX 6100-01-01-0823 JS21 Blade PowerPC_970MP
AIX 5200-08-CSP-0000 JS21 Blade PowerPC_970MP
AIX 6100-01-01-0823 JS22 Blade PowerPC_POWER6
AIX 6100-01-01-0823 JS22 Blade PowerPC_POWER6
AIX 5300-07-00-0000 JS22 Blade PowerPC_POWER6
AIX 6100-01-01-0823 JS22 Blade PowerPC_POWER6
AIX 6100-01-01-0823 JS22 Blade PowerPC_POWER6
AIX 6100-01-01-0823 JS22 Blade PowerPC_POWER6
AIX 6100-01-01-0823 JS22 Blade PowerPC_POWER6
AIX 5200-08-CSP-0000 eServer pSeries PowerPC_POWER4
AIX 6100-01-01-0823 JS22 Blade PowerPC_POWER6
AIX 5200-08-01-0000 eServer pSeries PowerPC_POWER4
AIX 5200-09-04-0000 JS21 Blade PowerPC_970FX
AIX 5300-04-03 JS21 Blade PowerPC_970FX
AIX 5300-06-03-0732 eServer p5 PowerPC_POWER5
AIX 5300-04-03 eServer p5 PowerPC_POWER5
AIX 5300-04-CSP-0000 JS21 Blade PowerPC_970FX
More Information:
When I run the command “ncheck -i /tmp” when there are no files in /tmp:
# pwd
/tmp
# ls -la
total 216
drwxrwsrwt 2 sys sys 102912 Nov 02 08:12 .
drwxr-xr-x 26 root system 4096 Nov 02 07:51 ..
#
ncheck will respond with over 25000 lines of files, which looks like this:
664822 /bld_17450_aix_ppc-64/test/java/jre/bin/shared_test/copyright.o
664832 /bld_17450_aix_ppc-64/test/java/jre/bin/shared_test/jvmti/.
664808 /bld_17450_aix_ppc-64/test/java/jre/bin/shared_test/main.c
664809 /bld_17450_aix_ppc-64/test/java/jre/bin/shared_test/main.h
664842 /bld_17450_aix_ppc-64/test/java/jre/bin/shared_test/main.o
664810 /bld_17450_aix_ppc-64/test/java/jre/bin/shared_test/makefile
These are in fact files/folders which our test jobs created and then deleted.
Is my issue related to some feature of AIX that allows for file recovery?
If so, is there a way to disable this on /tmp?
Or is there a way to release these used i-nodes, without rebooting the system?
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hello,
does anyone have a script that can check the contents of the /tmp directory and for example e-mail the directory content if anything other than session files are present?
Maybe there are better ways to monitor suspicous /tmp and /var/tmp activity, if so I'm listening :) (1 Reply)
Discussion started by: jamesbond
1 Replies
2. UNIX for Dummies Questions & Answers
Hello,
Can someone please tell me which command to use to determine the available disk space on a given disk device?
I have to write a shell script that compresses files and stores them in a specific location but I am not sure how "conservative" I should be?
Thanks in advance!
Al. (4 Replies)
Discussion started by: alan
4 Replies
3. Solaris
Can you help. My server sunning solaris 9 on x86 platform pretty much hung for a few hours... I could not use telnet or ssh to the box - it kept refusing connection. A few hours later - I was able to log in again.
The server has not rebooted but here are the first errors in the messages log... (5 Replies)
Discussion started by: frustrated1
5 Replies
4. Shell Programming and Scripting
Hi ,
I am a documentum consultant and facing a problem with a shell script that launches an installer to install documentum . The problem i am facing is that , when i invoke the below script to install the software it says -
The directory /tmp does not contain enough space to extract... (1 Reply)
Discussion started by: h.shergill
1 Replies
5. Shell Programming and Scripting
HI ...
I am New to the Unix...I am trying to write a script to check the disk space.
But i am not able to write it.
I know the command to check the disk space df -k,but unable to write the script..Can any body help me...
Thanks in advance... (3 Replies)
Discussion started by: Kingkon
3 Replies
6. Shell Programming and Scripting
Hi Guys
i have a nice little piece of code then i need to modify so that is does not look at /Voulmes/*
thanks
sub disk_full {
my $i = 0;
open( DF, "df -l|" );
while (<DF>) {
#chomp();
next if (/^\/proc\b/);
$i++;
next if ( $i == 1 );
... (3 Replies)
Discussion started by: ab52
3 Replies
7. Shell Programming and Scripting
Hi,
I am new to shell scripting, and want to monitor disk space using shell script continously on server, which will shoot mail after crossing threshold limit
Please suggest.
Regards
Manoj (1 Reply)
Discussion started by: manoj.solaris
1 Replies
8. Solaris
Hi ,
In a server /tmp has almost reached 75% and i can see the File system utilization is 48Mb only , so i believe some process is using the /tmp space. I would like to know which process is using /tmp space.
# df -h /tmp
Filesystem size used avail capacity Mounted on
swap ... (9 Replies)
Discussion started by: chidori
9 Replies
9. Solaris
Hello All,
I have solaris server running,
uname -a
SunOS host 5.9 Generic_112233-12 sun4u sparc SUNW,Sun-Fire-280R
Filesystem Size Used Available Capacity Mounted on
/dev/md/dsk/d0 9.8G 8.7G 1.0G 90% /
/dev/dsk/c1t1d0s3 4.3G 7.7M 4.2G ... (17 Replies)
Discussion started by: gull05
17 Replies
10. UNIX for Advanced & Expert Users
Hi Team
when I boot the server I get this 2 errors :
the disk drive for /tmp is not ready yet or not present
the disk drive for /boot is not ready yet or not present
and its stay like that , I m using Ubuntu 12.04
please if someone have any idea how to fix that problem . (1 Reply)
Discussion started by: SULTAN01
1 Replies
LEARN ABOUT DEBIAN
orte-clean
orte-clean(1) Open MPI orte-clean(1)
NAME
orte-clean - Cleans up any stale processes and files leftover from Open MPI jobs.
SYNOPSIS
orte-clean [--verbose]
mpirun --pernode [--host | --hostfile file] orte-clean [--verbose]
OPTIONS
[-v | --verbose] This argument will run the command in verbose mode and print out the universes that are getting cleaned up as well as pro-
cesses that are being killed.
DESCRIPTION
orte-clean attempts to clean up any processes and files left over from Open MPI jobs that were run in the past as well as any currently
running jobs. This includes OMPI infrastructure and helper commands, any processes that were spawned as part of the job, and any temporary
files. orte-clean will only act upon processes and files that belong to the user running the orte-clean command. If run as root, it will
kill off processes belonging to any users.
When run from the command line, orte-clean will attempt to clean up the local node it is run from. When launched via mpirun, it will clean
up the nodes selected by mpirun.
EXAMPLES
Example 1: Clean up local node only.
example% orte-clean
Example 2: To clean up on a specific set of nodes specified on command line, it is recommended to use the pernode option. This will run
one orte-clean for each node.
example% mpirun --pernode --host node1,node2,node3 orte-clean
To clean up on a specific set of nodes from a file.
example% mpirun --pernode --hostfile nodes_file orte-clean
Example 3: Within a resource managed environment like N1GE, SLURM, or Torque. The following example is from N1GE.
First, we see that we have two nodes with two CPUs each.
example% qsh -pe orte 4
example% mpirun -np 4 hostname
node1
node1
node2
node2
Clean up all the nodes in the cluster.
example% mpirun --pernode orte-clean
Clean up a subset of the nodes in the cluster.
example% mpirun --pernode --host node1 orte-clean
SEE ALSO
orterun(1), orte-ps(1)
1.4.5 Feb 10, 2012 orte-clean(1)