Delete big directory issue


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Delete big directory issue
# 1  
Old 01-15-2014
Delete big directory issue

Hello folks,

I am deleting a directory with script it is taking 11Hour and also increase the IO on server. I am using below command, inside date directory there are hour directories, which i am deleting after archiving. Archiving is not taking long time, only "rm -rf" is taking alot of time with IOPS, please advise what should do to make it fast with less IO effect.

Code:
rm -rf $date

# 2  
Old 01-15-2014
What is the value of $date?

What permissions were your script run with?

Make very sure that your script is only deleting what its supposed to be deleting!
# 3  
Old 01-15-2014
1)date is one day old date 2014-01-14
2) i am running with root user
3) i am deleting correct directories.
# 4  
Old 01-15-2014
If it's taking 11 hours to delete a directory, it's either a very, very full directory, or rm is deleting things you didn't intend.

If it's just a very full directory, rm doesn't have a "go faster" button for that. It's the filesystem that's slow, from having far too many files piled into one folder.
# 5  
Old 01-15-2014
I saw a link and found that we can delete multiple files with this command, is it possible we can do something like this for date directory which is also having subdirectories and we if delete directory proper with less IOPS, how, i am confuse.

Code:
perl -e 'for(<*>){((stat)[9]<(unlink))}'

# 6  
Old 01-15-2014
Running multiple rm's at once won't make the delete happen faster because rm is not the thing being slow here.
Using other utilities to emulate rm won't make the delete happen faster because rm is not the thing being slow here.

Your disk and filesystem are performing badly because, probably, far too man files have been crammed into a single directory.
# 7  
Old 01-15-2014
Are there any network-mounted directories in there?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Solaris

Big /etc/group creating issue

I have Solaris-10 with kernel patch 142900-03, Update 7 Sparc server. root@ddlomps40:/# uname -a SunOS ddlomps40 5.10 Generic_142900-03 sun4u sparc SUNW,SPARC-Enterprise root@ddlomps40:/# cat /etc/release Solaris 10 5/09 s10s_u7wos_08 SPARC Copyright 2009 Sun... (5 Replies)
Discussion started by: solaris_1977
5 Replies

2. UNIX for Dummies Questions & Answers

Delete records from a big file based on some condition

Hi, To load a big file in a table,I have a make sure that all rows in the file has same number of the columns . So in my file if I am getting any rows which have columns not equal to 6 , I need to delete it . Delimiter is space and columns are optionally enclosed by "". This can be ... (1 Reply)
Discussion started by: hemantraijain
1 Replies

3. UNIX for Advanced & Expert Users

Delete first 100 lines from a BIG File

Hi, I need a unix command to delete first n (say 100) lines from a log file. I need to delete some lines from the file without using any temporary file. I found sed -i is an useful command for this but its not supported in my environment( AIX 6.1 ). File size is approx 100MB. Thanks in... (18 Replies)
Discussion started by: unohu
18 Replies

4. Shell Programming and Scripting

Delete rows from big file

Hi all, I have a big file (about 6 millions rows) and I have to delete same occurrences, stored in a small file (about 9000 rews). I have tried this: while read line do grep -v $line big_file > ok_file.tmp mv ok_file.tmp big_file done < small_file It works, but is very slow. How... (2 Replies)
Discussion started by: Tibbeche
2 Replies

5. Shell Programming and Scripting

Hello - Have a big issue need help please ASAP

I have been using this simple grep command to extract fields from a file that ALPHANUMERIC field has been set to 16 grep -w 'ALPHANUMERIC *16' filename ------------------------- sample data file SAMP_RICS "SAMPLE RICS" 5694 NULL ALPHANUMERIC 60 RMTES_STRING ... (11 Replies)
Discussion started by: mnassiri
11 Replies

6. HP-UX

Empty Directory, big size

Hello, Can you please explain why I have various empty directories with large size ? OS is B.11.11 (3 Replies)
Discussion started by: drbiloukos
3 Replies

7. UNIX for Dummies Questions & Answers

Empty Directory, big size

Hello, Can somebody please explain why I have EMPTY directories on HP-UX with BIG SIZE ? Thank you ! Double post, continued here (0 Replies)
Discussion started by: drbiloukos
0 Replies

8. Shell Programming and Scripting

Find which dir is big size in a directory

Hi all, Could you please tellme the commadn which sorts the list of directories in a parent dir by their size. Thanks. (2 Replies)
Discussion started by: firestar
2 Replies

9. HP-UX

Processing very big directory

Hello, I have a very big directory with lots of small files. Just doing a single "ls" takes eternity, so I even can't know how many files the directory has. I need to get the newest files in the last 15 minutes, but I can't get an eficient way of doing it. I've tried find, perl, shell script,... (4 Replies)
Discussion started by: psimoes79
4 Replies

10. Shell Programming and Scripting

Need script to make big directory structure

Hi, I'm a novice and I'd like to make a directory structure with a hundred or so folders. I've tried mkdir /foo/foo1/etc... mkdir /foo/foo2/etc mkdir /foo/foo3/etc mkdir /foo/foo4/etc ...but it appends '@' to each folder name and then fails on the subdirectories. Is it better to use a... (2 Replies)
Discussion started by: kamur
2 Replies
Login or Register to Ask a Question