#include <stdio.h>
#include <ftw.h>
static long file_count = 0L;
int ftw_callback( const char *path, const struct stat *sb, int flag )
{
if ( FTW_F == flag )
{
file_count++;
/* print every 1000 files */
if ( 0 == ( file_count % 1000 ) )
{
fprintf( stderr, "%ld\n", file_count );
}
}
return( 0 );
}
int main( int argc, char **argv )
{
int ii;
for ( ii = 1; ii < argc; ii++ )
{
ftw( argv[ ii ], ftw_callback, 256 );
}
fprintf( stderr, "Final count: %ld\n", file_count );
return( 0 );
}
That won't need to have another process reading the inode data and feeding it via a pipe. Compile it with the -m64 flag if you're on a 64-bit platform and you won't even need to worry if you have over 2 billion files....
At 200 lines per minute I doubt my application's the bottleneck. On my system it was able to process tens of thousands of lines per second... I wonder just how badly this disk's fragmented.
Jim's code "instantly" throws up the same values as df -i; After all I think I can use the inode count as an acceptable approximation.
achenle's code runs a little faster; I tried it first with a small directory like /opt with no issues:
Code:
[root@atlas ~]# time ./achenle_counter /opt
1000
2000
3000
4000
5000
6000
Final count: 6534
real 0m4.503s
user 0m0.010s
sys 0m0.161s
Now when I try to count the real directory it becomes slow again, I'm not sure why. E.g.:
Code:
[root@atlas ~]# time ./achenle_counter /export/archives/2010/storage
1000
2000
3000
4000
5000
Ctrl^C
real 2m52.076s
user 0m0.138s
sys 0m8.671s
I also found that the directory in question, besides having millions of files has also millions of directories (although I only want to count files). Could this be causing a slow counting?
Yes. find uses ftw() or nftw(), it opens every directory and stats every entry. When directories are large this takes a long time. One directory with 1M entries can take literally minutes to process.
So I have a loop that stated if a directory exists or not. If it does it prints the number of files within that directory. I use this code...
result=`(ls -l . | egrep -c '^-')`
However, no matter which directory I input, it outputs the number "2"
What is wrong here? (4 Replies)
I am trying to display the output of ls and also print the number of characters in EVERY file name. This is what I have so far:
#!/bin/sh
for x in `ls`; do
echo The number of characters in x | wc -m
done
Any help appreciated (1 Reply)
Help Needed !
Can we count number of folders of specific date in a directory, even if directory has folders of different dates.
Please reply as soon as possible. (1 Reply)
Hello, I'm writing a script in sh in which the first command line argument is a directory. from that, i'm suppose to count the number of readable, writable, and executable items in the directory. I know using $1 represents the directory, and ls would display all the items in the directory, and that... (4 Replies)
I've never toyed with awk, but it seems every time I present an elegant 2- to 8-line script, someone comes back with an awk 1-liner.
I just came up with this to count all the lines of source code in a directory. How would I do it in awk?
LINES=0
for n in $(wc -l *.cpp *.h | cut -b-7); do
... (2 Replies)
I have 20 files in a direcotry like BARE01_DLY_MKT_YYYYMMDD. The MKT differes for all these files but the remaining syntax remains the same for a particular day. If I am checking for today I need to make sure that there are 20 files that start with BARE01_DLY_MKT_20060720. How can I write a... (31 Replies)
I want to count the no of lines for files (.c and .h) present in a directory structure.
My code is:
#!/bin/bash
# Usage: linecount.sh directory_name
for file in $(find $1 -name ); do
wc -l "$file" >> filecount.txt
done
Problem is that the directory structure is really big... (3 Replies)
Some simple questions from a simple man.
If i wanted to count the number of files contained within a directory, say /tmp would ls -l /tmp ¦ wc -l suffice and will it be accurate?
second one: How would i check the number of files with a certain string in the filename, in the same directory.
... (2 Replies)
I am trying to write a script that will look for a file in a directory, then remove it. I need it to loop until it has removed a certain number of files. Is it better to do a repeat or to list each file in a pattern? Files will be numbered like RAF.01.*, RAF.02.*, etc. Thanks, James (6 Replies)
11-16-2010
