/tmp filling up


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting /tmp filling up
# 1  
Old 08-31-2011
Network /tmp filling up

Does anyone know of a way to redirect the ksh default of processing data in /tmp to another file system or / something else?

My ksh script is parsing large DB files and it keeps filling up /tmp on the root disk.
I have a 1 Tb disk with most of its space.

How do I re-direct the /tmp ksh processes to that 1 Tb disk?
# 2  
Old 08-31-2011
exporting TMPDIR=/path/to/whatever/ in the script or in a profile the script loads, before running the tmp-intensive processes, may do it.

If that doesn't work, it depends on what your system is, what your script is, and what program's creating these temp files.
# 3  
Old 08-31-2011
its awk / gawk and sed that are doing it. Do you know how to redirect the output.
For instance sort -T wil redirect soreting out of /vat/tmp.
But i do not see anything in awk's man page like that.

I will try your method as soon as I can.

Thanks for the idea
# 4  
Old 08-31-2011
Quote:
Originally Posted by cchelten
its awk / gawk and sed that are doing it.
I wasn't aware of awk/sed creating and using temp files on their own. Sort obviously needs a temp dir, but awk/sed hold data in memory. So it's probably the script that's redirecting their data into temporary files, not awk/sed themselves; options or environment vars fed into them would have no effect, the script decides where they go.

And unless they were extremely thoughtful in building that script, it might not obey TMPDIR.

Can we see the script?
# 5  
Old 08-31-2011
It looks like it doesn't use much ram or swap (running topas).
Is there a way to attach the script. I would be happy to show you.
Its pretty big.

---------- Post updated at 02:47 PM ---------- Previous update was at 02:44 PM ----------

Here's the main part that is causing the /tmp to fill.
/tmp is : AIX 6.1 obviously since i referenced topas

Code:
MB-Blocks
/dev/hd3        2048.00   2043.71    1%       16     1% /tmp




Code:
#!/bin/ksh
set -x
#
var=5600
out=./check.out
LIST=/data/$var/tests/*.txt
COUNT=0
rg=
kg=
jg=
mailer1=chris.cheltenham@fisglobal.com
mailer2=cchelten@yahoo.com
rm -f ./*.out
##############################
#find duplicate lines field 2#
##############################
for i in /data/$var/tests/*.txt
do
dups ()
{
awk '/^01/ || /^05/ {if (i[$2]) { mf[$2]++; print $0; if (mf[$2] == 1) { print i[$2] } } i[$2] = $0}' |sort -k
2,2 -k 1,1
}

prse ()
{
sed -e 's%0\\\\\\%\:%g' |tr ':' '\n' | gawk -F'\' '$1==01{print $1,$4,$6,$2} $1==05{print $1,$4,$9}' $LIST
}
done
############################################################################################
dups | prse | grep -vi "eprprde|mtm|$var" |  while read aa bb cc dd
do
echo $dd >> ./check1.out
if [ $COUNT -eq 0 ]
        then
        rg=$bb
        kg=$aa
        jg=$cc
        fi

if [ $bb = $rg ]
                then
                ((COUNT=$COUNT+1))

                        if [ $aa -eq $kg ]
                        then
                        :
                        else
                        fld=`echo $dd | awk -F'^' '{print $2}' | awk -F'=' '{print $1}'`
                        dte=`echo $bb | cut -c11`
                        print Date/Time >> $out
                        print $dte >> $out
                        print $kg $bb $jg >> $out
                        print $aa $rg $fld >> $out
                        echo >> $out
                        fi
                else
                echo
                COUNT=1
                fi
        kg=$aa
        rg=$bb
        jg=$cc
done

---------- Post updated at 04:16 PM ---------- Previous update was at 02:47 PM ----------

the TMPDIR thing did not work by the way, good try though.

Last edited by pludi; 08-31-2011 at 08:56 PM..
# 6  
Old 08-31-2011
I see a sort -k in there. That could be doing it, and you know how to redirect that.
# 7  
Old 08-31-2011
I will try it and let you know.I forgot i had that sort in there, Age is catching up wit me.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Filling empty cells

How do you fill empty cells that do not have any data in them with "X" in a tab delimited text file? Thanks! (4 Replies)
Discussion started by: evelibertine
4 Replies

2. Shell Programming and Scripting

Filling file with 10 zeros

I am stuck with a problem. I have some 100 files with extension .txt. The files look like this: 0 3 0 0 4 All files have variable number of characters and some files are completely EMPTY. I want to make columns of length 10 by putting zeros (0) after the numbers end and also 10... (1 Reply)
Discussion started by: shoaibjameel123
1 Replies

3. Solaris

Root partition filling up

I have a T1000 Sparc server that has a relatively small root partition which is 24Gb and a larger partition dedicated to /export/home that is approximately 100 Gb. We have a lot of data going to /var/audit and to /var/core/corefiles. Is there any non-destructive way to redirect files from... (4 Replies)
Discussion started by: goose25
4 Replies

4. UNIX for Dummies Questions & Answers

Need help filling in ranges

I have a list of about 200,000 lines in a text file that look like this: 1 1 120 1 80 200 1 150 270 5 50 170 5 100 220 5 300 420 The first column is an identifier, the next 2 columns are a range (always 120 value range) I'm trying fill in the values of those ranges, and remove... (4 Replies)
Discussion started by: knott76
4 Replies

5. Shell Programming and Scripting

filling a character

in my file data is like this 1,2,3 3,4,5,6,7,8 10,11,23,24 i want to make as 1,2,3,?,?,? 3,4,5,6,7,8 10,11,23,24,?,? here max no of words(separated by comma) in a line is 6.so every line contains 6 words.Line which have less than 6 words replaced with '?' as a word i have... (3 Replies)
Discussion started by: new2ubuntulinux
3 Replies

6. Emergency UNIX and Linux Support

Something is filling hard disk on its own.

I came in this morning to find that our mail server was down. Couldn't connect. I looked at logs. The logs complained about no space on device. I run df and it comfirms that the system disk (mounted on /) is at 100% capacity. I try to delete some files before attempting to look at cyrus. I rotate... (23 Replies)
Discussion started by: timgolding
23 Replies

7. UNIX for Dummies Questions & Answers

filling variable with ls

hello All, Need some further help. This will make my live easier. Instead of copy and pasting I think I can automate some website building. when I do a ls from a directory I need the file names placed into a sentence. it is going about wordts like: word-AB-1234.jpg... (1 Reply)
Discussion started by: ToXiQ
1 Replies

8. Shell Programming and Scripting

Filling out a text

Hello, I have a problem with filling out a text. I have different lenghts in a file and would like to see that all the lines becomes the same length by putting a zero in front off the line. Please advice. Old File: ---------- 5432 233 3455 4432 New File: ----------- 5432 0233... (6 Replies)
Discussion started by: peterk
6 Replies

9. UNIX for Dummies Questions & Answers

monitoring /tmp and /var/tmp for suspicous activity

Hello, does anyone have a script that can check the contents of the /tmp directory and for example e-mail the directory content if anything other than session files are present? Maybe there are better ways to monitor suspicous /tmp and /var/tmp activity, if so I'm listening :) (1 Reply)
Discussion started by: jamesbond
1 Replies

10. UNIX for Advanced & Expert Users

Process responsible for filling up /var/tmp

Hi, Help ! - I have a process which I cannot find that is writing to /var/tmp every 10 minutes and filling up my partition, it is also filling up my wtmpx file. I have some software error correction for a faulty DIMM at the moment - is this likely to be causing this as well as over-loading my... (3 Replies)
Discussion started by: Mal
3 Replies
Login or Register to Ask a Question