Counting total files with different file types in each folder
Trying to count total files with different file types with thousands of files in each folder.
Since some files do not have extensions I have to use below criteria.
Count Total Files starting with --> "^ERROR"
Count Total Files starting with --> "^Runtime"
Count Everything else or files without any extension
Since I have to count this in a 12TB root folder with 6800 sub folders with thousands of files in each, this should not get into buffer overflow or out of memory or too many files situations. It should be faster.
I think either perl or awk can do this implicitly with the help of xargs!,, but not entirely sure how..
Code:
# I wish something like this can print counts for each sub-folder.
for each targetDIR in $(6800 folders); do
find $targetDIR -type f | xargs -i awk -v file="{}" -v td="$targetDIR" \
'file ~ "./^ERROR" {CNT_ERROR += 1}; \
file ~ "./^Runtime" {CNT_Runtime += 1}; \
file !~ "./^ERROR|^./Runtime" {CNT_Others += 1}; \
END {print td "," CNT_ERROR "," CNT_Runtime "," CNT_Others}'
done
Then I can get overall counts myself.
Last edited by kchinnam; 06-27-2017 at 12:59 AM..
Reason: more verbose
So I'm trying to search for the top 10 or 15 items under a directory by file type. I want to run a command on a directory and get something like the following:
Example of expected output.. .PDF: 100, .txt: 95, .word: 80..
What would be the best way of going about this? I've searched around... (2 Replies)
Hi All,
When i give the ls -lrt to list out all files with total number of files , i get the output as
ls -lrt
total 72
-rw-r--r-- 1 hari staff 796 Jul 11 09:17 va.txt
-rw-r--r-- 1 hari staff 169 Jul 13 00:20 a.log
-rwxr-xr-x 1 hari staff 659 Aug... (9 Replies)
I'm trying to make this code below to work but I can't find the way to do the following: I want to make the script to touch only 10% of the total amount of files counted inside the given directory instead of all like it is now.
I would greatly appreciate it if someone can give me a direction on... (9 Replies)
Hi All,
I have written a script on this but it does not do the requisite job. My requirement is this:
1. I have two kinds of files each with different extensions. One set of files are *.dat (6000 unique DAT files all in one directory) and another set *.dic files (6000 unique DIC files in... (1 Reply)
I am trying to get the total size of the folder using the below command
but its not working. any ideas?
du -bc <foldername>/|grep total|tr -s " "|cut -d" " -f1
the output i am getting is
78996 total
but i just want it to be as
78996
please help (3 Replies)
wow, back again :)
I am trying to write a program that has many functions, however one of the functions will count the total number of users in the database.
I want to just use wc -l passwd (i've made a copy of passwd to play with, no worries lol)
but I am worried that if any of the... (4 Replies)
Hi,
I am trying to get the total file size for certain files per directory.
I am using
find /DirectoryPath -name '*.dta' -exec ls -l {} \; | awk '{ print $NF ": " $5 }' > /users/cergun/My\ Documents/dtafiles.txt
but this lists all the files in the directories.
I need the total... (9 Replies)
Hi all,
I have copied all my files to one folder.and i want to check how many files (count) in the folder recently moved or total files in my folder?
please send me the query asap. (3 Replies)
I'm trying to create a simple file inventory for a series of huge directories containing e-records. What I'm after is a list of all directories and sub-directories with just the number of each type of file in that directory/sub-directory. For example output would look like:
... (6 Replies)
Ok, another fun hiccup in my UNIX learning curve. I am trying to count the number of occurrences of an IP address across multiple files named example.hits. I can extract the number of occurrences from the files individually but when you use grep -c with multiple files you get the output similar to... (5 Replies)
06-27-2017
