Fastest way to count big amount of files in sub directory


 
Thread Tools Search this Thread
Operating Systems AIX Fastest way to count big amount of files in sub directory
# 8  
Old 08-07-2010
I agree. The "find" command is much faster than the "ls" command because it does not sort the results.

It would certainly help to know how may files there are and whether the command is appreciably faster when issued a second time.

Some packages for specific filesystems (like vxfs) offer commands which are faster than even "find".

What Opeating System and version do you have and what type of filesystem do you use (please be very specific) ?
# 9  
Old 08-10-2010
Hey guys, I have done some testing on it at our test server. Indeed, the find command is faster than the perl script. Initially, I thought it is the other way until I really tested it.

---------- Post updated at 12:29 PM ---------- Previous update was at 12:09 PM ----------

Here are the test result for 127807 files inside 25 folders.
- find . -type f | wc -l (1 second)
- perl script using for loop and recurse (4 seconds)
- ls -RF | sed -e '/^$/d' -e '/.*[/@:=|>]$/d' |wc -l (10 seconds)
# 10  
Old 08-10-2010
Thank you for posting those results
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Fastest way calculating directory

Hi expert, Is there any fastest way to calculate recursive directory, and I have total 600 directories have 100000 files and 10 directory approximately 9000000 - 10000000 each files per directory. currently using this command "du -k --max-depth=0" to get the size but very slow it take 24 hours... (9 Replies)
Discussion started by: rufino
9 Replies

2. UNIX for Dummies Questions & Answers

[Solved] Count amount of times of appearing of character before a word?

Hello Is there a way to calculate how many times a particular symbol appeared in a string before a particular word. Desktop/Myfiles/pet/dog/puppy So, I want to count number of occurence of"/" in this directory before the word dog lets say. Cheers, Bob (3 Replies)
Discussion started by: FUTURE_EINSTEIN
3 Replies

3. Shell Programming and Scripting

Count files in every directory

Hi all, I'm looking over all the internet for finding how to make a count of the files for every directory apart.. not a total result for the xx directories. For example: -dir1 file1 file2 -subdir1 -subdir2 file1 -subdir3 file1 -dir2 -dir3 file4 ... (9 Replies)
Discussion started by: CODIII
9 Replies

4. UNIX for Dummies Questions & Answers

Delete X amount of similar files in a directory

Need a Unix command to save the last 20 versions of a file in a specific directory and delete everything else. Date is irrelevant. Anyone aware of such an animal? In my test, I came up with: ls -t1 /tmp/testfile* | tail -n +20 | xargs rm I don’t quite trust the author though! (1 Reply)
Discussion started by: rwsherman
1 Replies

5. Shell Programming and Scripting

What amount of store occupy only files in a directory?

Hello I'm trying to do an exercise programming in bash where I have to get only the store of files in a directory but NOT all capacity of the directory. I probe with: du -sh "$directory"*` but I get all the capacity and I probe with ls command but I couldnt. Are there any way to get only files... (2 Replies)
Discussion started by: adiegorpc
2 Replies

6. UNIX for Dummies Questions & Answers

Count number of files in directory excluding existing files

Hi, Please let me know how to find out number of files in a directory excluding existing files..The existing file format will be unknown..each time.. Thanks (3 Replies)
Discussion started by: ammu
3 Replies

7. UNIX for Advanced & Expert Users

mv OR cp many files from 1 big directory with sub folders all to 1 spot?

Hello All. I am trying to do this from a terminal prompt on my mac.... I have 100 folders all named different things. Those 100 folders are inside ~/Desktop/Pictures directory. Each of the 100 folders are uniquely named. The image files inside of each folder only have some similarities. ... (1 Reply)
Discussion started by: yoyoyo777
1 Replies

8. Shell Programming and Scripting

count number of files in a directory

what's the script to do that? i want to only count the number of files in that directory, not including any sub directories at all (5 Replies)
Discussion started by: finalight
5 Replies

9. UNIX for Dummies Questions & Answers

count amount of accounts?

Hi, I am relatively new to Unix and trying to understand as much as I can. I would like to know if it's possible to count the total number of Unix accounts? If so, can the count be done from any working directory or does it have to be specific to where the accounts are based? Thanks! (4 Replies)
Discussion started by: Trogan
4 Replies

10. Shell Programming and Scripting

how to delete/remove directory in fastest way

hello i need help to remove directory . The directory is not empty ., it contains several sub directories and files inside that.. total number of files in one directory is 12,24,446 . rm -rf doesnt work . it is prompting for every file .. i want to delete without prompting and... (6 Replies)
Discussion started by: getdpg
6 Replies
Login or Register to Ask a Question