Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Count the number of subset of files in a directory

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Count the number of subset of files in a directory
# 1  
Old 01-18-2013
Count the number of subset of files in a directory
hi I am trying to write a script to count the number of files, with slightly different subset name, in a directory

for example, in directory /data, there are a subset of files that are name as follow

Code:
 
/data/data_1_(1to however many).txt
/data/data_2_(1 to however many).txt
/data/data_3_(1to however many).txt
etc etc etc

now I want to write a script to count the number of files with each subset name (1 to 3 or however many there are)

Thanks for your help
Last edited by Scott; 01-18-2013 at 06:53 AM.. Reason: Code tags, please...
# 2  
Old 01-18-2013
Something like this?
Code:
cd /data
ls | awk -F_ '{A[$1 FS $2]++} END{for(i in A) print i, A[i]}'

# 3  
Old 01-18-2013
would you minding explaining the script so I could adapt it accordingly and test?
# 4  
Old 01-18-2013
Sure:
Code:
awk -F_ '                     # set field separator to underscore
  {
    A[$1 FS $2]++             # count the number of times $1 FS $2 occurs (field 1 and field2 separated by an underscore (for example "data_1")
  } 
END{
  for(i in A) print i, A[i]   # At the end print the results
}
'

Last edited by Scrutinizer; 01-20-2013 at 02:29 PM..
# 5  
Old 01-20-2013
Thanks for your reply. What is

Code:
A[$1 FS $2]++

?

is that an array?
Last edited by Scrutinizer; 01-20-2013 at 02:28 PM..
# 6  
Old 01-20-2013
That is an element in the associative array A with index $1 FS $2 (FS="_" , so this means $1_$2) that gets incremented by 1 ( ++ )
# 7  
Old 01-28-2013
The code has worked impressively, many thanks for that.

I want to write a line to determine that, if the number of files with any prefix is more than 5, then print out the prefix names in one line (separated by a single space), such as

data_1 data_2

I wrote this line

Code:
ls | awk -F_ '{A[$1 FS $2]++} END {for (j in A) {if (A[j] > 5) {printf j, " "}}}'

However the output from this line is

Code:
data_1data_2

It doesn't seem to recognise the single space I asked for between the prefix. Do you know what I may have done wrong?
Last edited by Scrutinizer; 01-28-2013 at 07:50 AM.. Reason: code tags
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Count number of pattern matches per line for all files in directory

I have a directory of files, each with a variable (though small) number of lines. I would like to go through each line in each file, and print the: -file name -line number -number of matches to the pattern /comp/ for each line. Two example files: cat... (4 Replies)
Discussion started by: pathunkathunk
4 Replies

2. Shell Programming and Scripting

Count number of files

Hi All! I need to have a script that counts the number of files arriving in a landing directory, them some app pick these files to be processed and load to a DB. But this process is so fast that I am not able to count all the files arriving on a landing directory. Please can you help? My... (6 Replies)
Discussion started by: fretagi
6 Replies

3. Shell Programming and Scripting

How to count number of files in directory and write to new file with number of files and their name?

Hi! I just want to count number of files in a directory, and write to new text file, with number of files and their name output should look like this,, assume that below one is a new file created by script Number of files in directory = 25 1. a.txt 2. abc.txt 3. asd.dat... (20 Replies)
Discussion started by: Akshay Hegde
20 Replies

4. Shell Programming and Scripting

How to count the number of files starting with a pattern in a Directory

Hi! In our current directory there are around 35000 files. Out of these a few thousands(around 20000) start with, "testfiles9842323879838". I want to count the number of files that have filenames starting with the above pattern. Please help me with the command i could use. Thank... (7 Replies)
Discussion started by: atechcorp
7 Replies

5. Shell Programming and Scripting

Count the number of words in some subset of file and disregard others

Hi All, I have some 6000 text files in a directory. My files are named like 1.txt, 2.txt 3.txt and so on until 6000.txt. I want to count the "number of words" in only first 3000 of them. Any suggestions? I know wc -w can count the number of words in a text file. I am using Red Hat Linux. (3 Replies)
Discussion started by: shoaibjameel123
3 Replies

6. UNIX for Dummies Questions & Answers

Count number of files in directory excluding existing files

Hi, Please let me know how to find out number of files in a directory excluding existing files..The existing file format will be unknown..each time.. Thanks (3 Replies)
Discussion started by: ammu
3 Replies

7. Shell Programming and Scripting

perl script on how to count the total number of lines of all the files under a directory

how to count the total number of lines of all the files under a directory using perl script.. I mean if I have 10 files under a directory then I want to count the total number of lines of all the 10 files contain. Please help me in writing a perl script on this. (5 Replies)
Discussion started by: adityam
5 Replies

8. UNIX for Dummies Questions & Answers

Read directory files and count number of lines

Hello, I'm trying to create a BASH file that can read all the files in my working directory and tell me how many words and lines are in that file. I wrote the following code: FILES="*" for f in "$FILES" do echo -e `wc -l -w $f` done My issue is that my file is outputting in one... (4 Replies)
Discussion started by: jl487
4 Replies

9. Shell Programming and Scripting

count number of files in a directory

what's the script to do that? i want to only count the number of files in that directory, not including any sub directories at all (5 Replies)
Discussion started by: finalight
5 Replies

10. Shell Programming and Scripting

Count the number of files in a directory

Hi All, How do i find out the number of files in a directory using unix command ? (14 Replies)
Discussion started by: Raynon
14 Replies
Login or Register to Ask a Question