Deleting a million of files .. Post: 302897598

Sponsored Content

Top Forums UNIX for Dummies Questions & Answers Deleting a million of files .. Post 302897598 by rbatte1 on Tuesday 15th of April 2014 04:55:42 AM

04-15-2014

Moderator

If actual elapse time is critical because of ongoing processing, could I suggest:-

Code:

mv /path /path-to-del
mkdir -m 777 /path
chown user:group /path

Doing this will be a near instantaneous operation and then with the directory renamed and a new empty one created, your processing can continue whilst you dispose of what is now called /path-to-del at your leisure.

The -m 777 on the mkdir sets the permissions for the directory. The user:group on the chown sets both the owner and group in one operation, rather than having to do a chown and a chgrp

I might be way off course, but I was confused as to why the timing matters so much. If you still want a choice for your original question, I would go with rm -f /path

Another alternate might be available if this is a filesystem you can unmount for a few moments:-

Code:

umount /path
newfs your-options /dev/logical-volume
mount /path

You don't give us an OS or any clue about what devices the disks are on, so I've tried to be generic here. You make have to use mkfs, crfs, fscreate or other things depending what you are running. Of course, this may not be an option.

I hope that these suggestions might help.

Robin
Liverpool/Blackburn
UK

This User Gave Thanks to rbatte1 For This Post:

rbatte1

View Public Profile for rbatte1

Visit rbatte1's homepage!

Find all posts by rbatte1

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need help comparing two files and deleting some things in those files!

So I have two files: File1 pictures.txt 1.1 1.3 dance.txt 1.2 1.4 treehouse.txt 1.3 1.5 File2 pictures.txt 1.5 ref2313 1.4 ref2345 1.3 ref5432 1.2 ref4244 dance.txt 1.6 ref2342 1.5 ref2352 1.4 ref0695 1.3 ref5738 1.2 ref4948 1.1 treehouse.txt 1.6 ref8573 1.5 ref3284 1.4 ref5838...

2. Solaris

Need to know command to delete more than 3 million files from /var/spool/clientmqueue

Hi I need to delete more than 3 million files from /var/spool/clientmqueue. When I give the following command to delete the files, I get the error # pwd /var/spool/clientmqueue # rm -f * /usr/bin/rm: arg list too long Please tell me how can I delete the files

3. Shell Programming and Scripting

AIX system.... deleting files in remote directory after retrieving files

Hi Friends, I am new to this , I am working on AIX system and my scenario is to retrive the files from remote system and remove the files from the remote system after retreving files. I can able to retrieve the files but Can't remove files in remote system. Please check my code and help me out...

4. Shell Programming and Scripting

Deleting files

Hi all, I have developed a shell script to copy the files from source to destination and simultaneously to delete the copied files in source. I can copy the files but the files cannot be deleted in source side.

5. Shell Programming and Scripting

Matching 10 Million file records with 10 Million in other file

Dear All, I have two files both containing 10 Million records each separated by comma(csv fmt). One file is input.txt other is status.txt. Input.txt-> contains fields with one unique id field (primary key we can say) Status.txt -> contains two fields only:1. unique id and 2. status ...

6. Shell Programming and Scripting

Fast processing(mv command) of 1 million+ files using find, mv and xargs

Hi, I'd like to ask if anybody can help improve my code to move 1 million+ files from a directory to another: find /source/dir -name file* -type f | xargs -I '{}' mv {} /destination/dir I learned this line of code from this forum as well and it works fine. However, file movement is kinda...

7. UNIX for Dummies Questions & Answers

Pls. help with script to remove million files

Hi, one of the server, log directory was never cleaned up. We have so many files. I want to remove all the files that starts with dfr* but I get error message when I use the *. rm qfr* bash: /usr/bin/rm: Arg list too long I am trying to write this script but not working. ...

8. Shell Programming and Scripting

Bash script deleting my files, and editing files in subdirectories question

#!/bin/bash # name=$1 type=$2 number=1 for file in ./** do if then filenumber=00$number elif then filenumber=0$number fi tempname="$name""$filenumber"."$type" if

9. UNIX for Advanced & Expert Users

Zip million files taking 12 hours or more

Hi I have task to zip files based on modified time but they are in millions and it is taking lot of time more than 12 hours and also eating up high cpu is there any other / better way to handle it quickly with less cpu consumptionfind . ! -name \"*.gz\" -mtime +7 -type f | grep -v '/.*/' |...

LEARN ABOUT DEBIAN

lfc-chown

LFC-CHOWN(1)							 LFC User Commands						      LFC-CHOWN(1)

NAME

       lfc-chown - change owner and group of a LFC directory/file in the name server

SYNOPSIS

       lfc-chown [-h] [-R] owner[:group] path...

       lfc-chown [-h] [-R] :group path...

DESCRIPTION

       lfc-chown sets the owner and/or the group of a LFC directory/file in the name server to the values in owner and group respectively.

       To change the owner ID, if the group ID does not change and if the caller and the new owner ID belong to that group, GRP_ADMIN privilege is
       needed, otherwise the caller must have ADMIN privilege in the Cupv database.  To change the group ID, the effective user ID of the  process
       must  match  the  owner	ID  of the file and the new group must be in the list of groups the caller belong to or the caller must have ADMIN
       privilege in the Cupv database.

       owner  is either a valid username or a valid numeric ID.

       group  is either a valid group name or a valid numeric ID.

       path   specifies the LFC pathname.  If path does not start with /, it is prefixed by the content of the LFC_HOME environment variable.

OPTIONS

       The following options are supported:

       -h     If path is a symbolic link, changes the ownership of the link itself.

       -R     Recursive mode.

EXIT STATUS

       This program returns 0 if the operation was successful or >0 if the operation failed.

SEE ALSO

       Castor_limits(4), lfc_chown(3), Cupvlist(1)

AUTHOR

       LCG Grid Deployment Team

LFC
							   $Date: 2007/01/15 08:05:17 $ 					      LFC-CHOWN(1)