04-03-2013
ls is guaranteed to perform badly here, because it must read the entire directory list and sort their names before it can print. It might bog for minutes or hours until it shows anything.
find doesn't have problems "dealing with" large numbers of files. In a sense find's job is rather simple -- opendir(), readdir(), print if match, loop until done. If it's struggling, that means it either has too much work to do -- finding 300 'good' files out of 1.2 million files you don't care about means scanning through all 1.2 million -- or the filesystem itself is responding slowly.
Small numbers of folders crammed full of millions of files generally perform rather badly, especially when already busy. The filesystem itself, rather than find, may be suffering here.
9 More Discussions You Might Find Interesting
1. AIX
my task : tar up large bunch of files(about 10,000 files) in the current directories that created more than 30 days ago
but it come with following error
find ./ -ctime +30 | xargs tar rvf test1.tar
tar: test1.tar: A file or directory in the path name does not exist. (3 Replies)
Discussion started by: darkrainbow
3 Replies
2. UNIX for Dummies Questions & Answers
Can anyone interpret and tell me the way the below command works?
find * -name "*${msgType}" -mtime +${archiveDays} -prune -type f -print 2>/dev/null | xargs rm -f 2> /dev/null
Please tell me the usage of prune and xargs in the above command?
Looking forward your reply.
Thanks in... (1 Reply)
Discussion started by: venkatesht
1 Replies
3. Shell Programming and Scripting
I believe what is happening is rm is executing in the script on every directory and on failure of the first it stops although returns status 0.
find $HOME -name /directory/filename | xargs -l rm
This is the code I use but file remains. I am using sun solaris system which has way limited... (4 Replies)
Discussion started by: Ebodee
4 Replies
4. Solaris
Hi
I need to delete more than 3 million files from /var/spool/clientmqueue. When I give the following command to delete the files, I get the error
# pwd
/var/spool/clientmqueue
# rm -f *
/usr/bin/rm: arg list too long
Please tell me how can I delete the files (5 Replies)
Discussion started by: sb200
5 Replies
5. Shell Programming and Scripting
I need to apply mp3gain (album mode) to all mp3 files in a given directory. Each album is in its own directory under /media/data/music/albums for example:
/media/data/music/albums/foo
/media/data/music/albums/bar
/media/data/music/albums/more
What needs to happen is:
cd... (4 Replies)
Discussion started by: audiophile
4 Replies
6. Shell Programming and Scripting
these are numeric ids..
222932017099186177
222932014385467392
222932017371820032
222932017409556480
I have text file having 300 millions of line as shown above. I want to find duplicates from this file. Please suggest the quicker way..
sort | uniq -d will... (3 Replies)
Discussion started by: pamu
3 Replies
7. UNIX for Dummies Questions & Answers
Hi,
Which way is faster rm -rf /path/ or find / -name -exec rm {} \; and why? (7 Replies)
Discussion started by: cain82
7 Replies
8. Shell Programming and Scripting
I have a script (ksh) which tries to run a function in parallel for performance gains. I am also trying to limit the number of parallel child processes to avoid overloading the system by using a variable to count triggered processes and waiting for completion e.g.
do_something ()
{
...
}
... (9 Replies)
Discussion started by: jawsnnn
9 Replies
9. UNIX for Advanced & Expert Users
Hi
I have task to zip files based on modified time but they are in millions and it is taking lot of time more than 12 hours and also eating up high cpu
is there any other / better way to handle it quickly with less cpu consumptionfind . ! -name \"*.gz\" -mtime +7 -type f | grep -v '/.*/' |... (2 Replies)
Discussion started by: reldb
2 Replies
LEARN ABOUT REDHAT
xfs_ncheck
xfs_ncheck(8) System Manager's Manual xfs_ncheck(8)
NAME
xfs_ncheck - generate pathnames from i-numbers for XFS
SYNOPSIS
xfs_ncheck [ -i ino ] ... [ -f ] [ -s ] [ -l logdev ] device
DESCRIPTION
xfs_ncheck with no -i arguments generates an inode number and pathname list of all files on the given filesystem. Names of directory files
are followed by /.. The output is not sorted in any particular order. The filesystem to be examined is specified by the device argument,
which should be the disk or volume device for the filesystem. Filesystems stored in files can also be checked, using the -f flag.
OPTIONS
-f Specifies that the filesystem image to be processed is stored in a regular file at device (see the mkfs.xfs -d file option). This
might happen if an image copy of a filesystem has been made into an ordinary file.
-l logdev
Specifies the device where the filesystem's external log resides. Only for those filesystems which use an external log. See the
mkfs.xfs -l option, and refer to xfs(5) for a detailed description of the XFS log.
-s Limits the report to special files and files with setuserid mode. This option may be used to detect violations of security pol-
icy.
-i ino Limits the report to only those files whose inode numbers follow. May be given multiple times to select multiple inode numbers.
If the filesystem is seriously corrupted, or very busy and looks like it is corrupt, a message of the form that would be generated by
xfs_check(8) may appear.
xfs_ncheck is only useful with XFS filesystems.
SEE ALSO
mkfs.xfs(8), xfs_check(8), xfs(5).
xfs_ncheck(8)