Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Deleting a million of files .. Post 302897407 by Don Cragun on Monday 14th of April 2014 03:47:54 AM
Old 04-14-2014
Quote:
Originally Posted by cain82
I suppose - once rm is faster than million times Smilie

---------- Post updated at 02:29 AM ---------- Previous update was at 02:28 AM ----------

What about
Code:
find /path/ -type f -delete

is this a fastest method to remove a million files more than 200gb ?
You're comparing apples and oranges:
Code:
rm -rf directory

attempts to remove directory and any links to files (of any type) in the file hierarchy rooted in directory.

On the other hand:
Code:
find directory -type f -delete

attempts to remove only links to regular files in the file hierarchy rooted in directory (leaving the directory structure and all other file types as is). Furthermore, the -delete primary in find is an extension to the standards that is available on some systems, but is not available on other systems.
These 2 Users Gave Thanks to Don Cragun For This Post:
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need help comparing two files and deleting some things in those files!

So I have two files: File1 pictures.txt 1.1 1.3 dance.txt 1.2 1.4 treehouse.txt 1.3 1.5 File2 pictures.txt 1.5 ref2313 1.4 ref2345 1.3 ref5432 1.2 ref4244 dance.txt 1.6 ref2342 1.5 ref2352 1.4 ref0695 1.3 ref5738 1.2 ref4948 1.1 treehouse.txt 1.6 ref8573 1.5 ref3284 1.4 ref5838... (24 Replies)
Discussion started by: linuxkid
24 Replies

2. Solaris

Need to know command to delete more than 3 million files from /var/spool/clientmqueue

Hi I need to delete more than 3 million files from /var/spool/clientmqueue. When I give the following command to delete the files, I get the error # pwd /var/spool/clientmqueue # rm -f * /usr/bin/rm: arg list too long Please tell me how can I delete the files (5 Replies)
Discussion started by: sb200
5 Replies

3. Shell Programming and Scripting

AIX system.... deleting files in remote directory after retrieving files

Hi Friends, I am new to this , I am working on AIX system and my scenario is to retrive the files from remote system and remove the files from the remote system after retreving files. I can able to retrieve the files but Can't remove files in remote system. Please check my code and help me out... (3 Replies)
Discussion started by: vinayparakala
3 Replies

4. Shell Programming and Scripting

Deleting files

Hi all, I have developed a shell script to copy the files from source to destination and simultaneously to delete the copied files in source. I can copy the files but the files cannot be deleted in source side. (3 Replies)
Discussion started by: Venkatesan
3 Replies

5. Shell Programming and Scripting

Matching 10 Million file records with 10 Million in other file

Dear All, I have two files both containing 10 Million records each separated by comma(csv fmt). One file is input.txt other is status.txt. Input.txt-> contains fields with one unique id field (primary key we can say) Status.txt -> contains two fields only:1. unique id and 2. status ... (8 Replies)
Discussion started by: vguleria
8 Replies

6. Shell Programming and Scripting

Fast processing(mv command) of 1 million+ files using find, mv and xargs

Hi, I'd like to ask if anybody can help improve my code to move 1 million+ files from a directory to another: find /source/dir -name file* -type f | xargs -I '{}' mv {} /destination/dir I learned this line of code from this forum as well and it works fine. However, file movement is kinda... (6 Replies)
Discussion started by: agentgrecko
6 Replies

7. UNIX for Dummies Questions & Answers

Pls. help with script to remove million files

Hi, one of the server, log directory was never cleaned up. We have so many files. I want to remove all the files that starts with dfr* but I get error message when I use the *. rm qfr* bash: /usr/bin/rm: Arg list too long I am trying to write this script but not working. ... (4 Replies)
Discussion started by: samnyc
4 Replies

8. Shell Programming and Scripting

Bash script deleting my files, and editing files in subdirectories question

#!/bin/bash # name=$1 type=$2 number=1 for file in ./** do if then filenumber=00$number elif then filenumber=0$number fi tempname="$name""$filenumber"."$type" if (4 Replies)
Discussion started by: TheGreatGizmo
4 Replies

9. UNIX for Advanced & Expert Users

Zip million files taking 12 hours or more

Hi I have task to zip files based on modified time but they are in millions and it is taking lot of time more than 12 hours and also eating up high cpu is there any other / better way to handle it quickly with less cpu consumptionfind . ! -name \"*.gz\" -mtime +7 -type f | grep -v '/.*/' |... (2 Replies)
Discussion started by: reldb
2 Replies
RM(1)							    BSD General Commands Manual 						     RM(1)

NAME
rm, unlink -- remove directory entries SYNOPSIS
rm [-dfiPRrvW] file ... unlink file DESCRIPTION
The rm utility attempts to remove the non-directory type files specified on the command line. If the permissions of the file do not permit writing, and the standard input device is a terminal, the user is prompted (on the standard error output) for confirmation. The options are as follows: -d Attempt to remove directories as well as other types of files. -f Attempt to remove the files without prompting for confirmation, regardless of the file's permissions. If the file does not exist, do not display a diagnostic message or modify the exit status to reflect an error. The -f option overrides any previous -i options. -i Request confirmation before attempting to remove each file, regardless of the file's permissions, or whether or not the standard input device is a terminal. The -i option overrides any previous -f options. -P Overwrite regular files before deleting them. Files are overwritten three times, first with the byte pattern 0xff, then 0x00, and then 0xff again, before they are deleted. -R Attempt to remove the file hierarchy rooted in each file argument. The -R option implies the -d option. If the -i option is specified, the user is prompted for confirmation before each directory's contents are processed (as well as before the attempt is made to remove the directory). If the user does not respond affirmatively, the file hierarchy rooted in that directory is skipped. -r Equivalent to -R. -v Be verbose when deleting files, showing them as they are removed. -W Attempt to undelete the named files. Currently, this option can only be used to recover files covered by whiteouts. The rm utility removes symbolic links, not the files referenced by the links. It is an error to attempt to remove the files ``.'' or ``..''. When the utility is called as unlink, only one argument, which must not be a directory, may be supplied. No options may be supplied in this simple mode of operation, which performs an unlink(2) operation on the passed argument. The rm utility exits 0 if all of the named files or file hierarchies were removed, or if the -f option was specified and all of the existing files or file hierarchies were removed. If an error occurs, rm exits with a value >0. NOTE
The rm command uses getopt(3) to parse its arguments, which allows it to accept the '--' option which will cause it to stop processing flag options at that point. This will allow the removal of file names that begin with a dash ('-'). For example: rm -- -filename The same behavior can be obtained by using an absolute or relative path reference. For example: rm /home/user/-filename rm ./-filename SEE ALSO
rmdir(1), undelete(2), unlink(2), fts(3), getopt(3), symlink(7) BUGS
The -P option assumes that the underlying file system is a fixed-block file system. In addition, only regular files are overwritten, other types of files are not. COMPATIBILITY
The rm utility differs from historical implementations in that the -f option only masks attempts to remove non-existent files instead of masking a large variety of errors. The -v option is non-standard and its use in scripts is not recommended. Also, historical BSD implementations prompted on the standard output, not the standard error output. STANDARDS
The rm command is almost IEEE Std 1003.2 (``POSIX.2'') compatible, except that POSIX requires rm to act like rmdir(1) when the file specified is a directory. This implementation requires the -d option if such behavior is desired. This follows the historical behavior of rm with respect to directories. The simplified unlink command conforms to Version 2 of the Single UNIX Specification (``SUSv2''). HISTORY
A rm command appeared in Version 1 AT&T UNIX. BSD
January 28, 1999 BSD
All times are GMT -4. The time now is 08:57 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy