Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

multiple files: counting

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting multiple files: counting
# 1  
Old 09-23-2010
multiple files: counting
In a directory, I have 5000 multiple files that contains around 4000 rows with 10 columns in each file containing a unique string 'AT' located at 4th column.

Code:
OM   3328   O     BT   268       5.800      7.500      4.700      0.000     1.400
OM   3329   O     BT   723       8.500      8.900      3.600      8.500     1.400
OM   3330   O     AT   231       6.700      5.500      7.600      0.000     1.400
OM   3331   O     AT   234       1.200      7.700      5.500      8.500     1.400
OM   3332   O     AT   256       3.800      5.800      5.200      0.000     1.400

I want to make a script to count how many 'AT' are found in each file, then print this count into a new file at second column. For example below, the 1st column represents the file number and the second column represents the counts. Since I have 5000 multiple files, I expect the first column to be 1....5000.

Code:
1    5
2    0
3    8
4    2
5    2
6    0
7    3
8    5
9    0
10   1

Thank you for helping.

-A
# 2  
Old 09-23-2010
Code:
ruby -e 'Dir["*"].each_with_index{|x,y| print "#{y+1} #{File.read(x).split.count("AT")}\n"}'

This User Gave Thanks to kurumi For This Post:
asanjuan
# 3  
Old 09-23-2010
Thanks kurumi.

The script works and thanks for the help.

In the directory, the files are numbered in order and should be printed also in order accordingly into the new file. So in this case, file 1 should be read in first followed by file 2 and so forth until file 5000. How to make it that way?

The small problem was that the files are being read in random order, which is not the way the script should do for the counting. So for example:

Directory:
Code:
1.txt
2.txt
3.txt
.
.
.
5000.txt


Newfile:
Code:
1    5
2    0
3    8
4    2
5    2
6    0
7    3
8    5
9    0
10   1

Column 1, row1 is the first file (1.txt), column 1 row 2 is the second file (2.txt) and goes on until last 5000.txt

-A
Last edited by asanjuan; 09-23-2010 at 08:21 AM..
# 4  
Old 09-23-2010
How about this,

Code:
ls [0-9]*.txt| xargs grep -wc "AT" | sort -t"." -n > newfile

Last edited by pravin27; 09-23-2010 at 08:58 AM..
# 5  
Old 09-23-2010
Bottleneck is to tell the script to read files inside directory by sorting it out numerically. Then, print this out accordingly into the newfile.

---------- Post updated at 08:03 PM ---------- Previous update was at 07:50 PM ----------

Hi pravin27, thanks.

The script gave the output:

Code:
1.txt  :5
2.txt  :0
3.txt  :8

The : symbol is not needed, as this cant be read in into making 2d graph. Also, file extension should not also appear in the first column.

The output expected is:

Code:
1  5
2  0
3  8

Thanks again for the time helping me.

-A

---------- Post updated at 08:13 PM ---------- Previous update was at 08:03 PM ----------

Since the expected newfile should be printed out without the suffix *.txt in 1st column, I modified the files in the directory such that:

Directory:
Code:
1
2
3
.
.
.5000

Then i issue this command at console:

Code:
ls * | xargs grep -wc "AT" | sort -t"." -n > newfile

Output looks:
Code:
1:4
2:5
3:0

How to tell script not have the symbol ":" in between columns 1 and 2. I want to see a space not this symbol ":" in between the two columns.

-A
# 6  
Old 09-23-2010
Code:
ruby -e 'Dir["*"].sort_by{|x| x.to_i}.each_with_index{|x,y| print "#{y+1} #{File.read(x).split.count("AT")}\n"}'

# 7  
Old 09-23-2010
try this,

Code:
ls [0-9]*.txt| xargs grep -wc "AT" | awk -F":" '{gsub(/.txt/,"");printf "%-4d %d\n", $1,$2}' | sort -n > newfile
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

[Solved] Counting The Number of Lines Between Values with Multiple Variables

Hey everyone, I have a bunch of lines with values in field 4 that I am interested in. If these values are between 1 and 3 I want it to count all these values to all be counted together and then have the computer print out LOW and the number of lines with those values in between 1 and 3,... (2 Replies)
Discussion started by: VagabondGold
2 Replies

2. Shell Programming and Scripting

Counting Multiple Fields with awk/nawk

I am trying to figure out a way in nawk to 1) get a count of the number of times a value appears in field 1 and 2) count each time the same value appears in field 2 for each value of field 1. So for example, if I have a text file with the following: grapes, purple apples, green squash, yellow... (2 Replies)
Discussion started by: he204035
2 Replies

3. Shell Programming and Scripting

Counting occurrences of all words in multiple files

Hey Unix gurus, I would like to count the number occurrences of all the words (regardless of case) across multiple files, preferably outputting them in descending order of occurrence. This is well beyond my paltry shell scripting ability. Researching, I can find many scripts/commands that... (4 Replies)
Discussion started by: twjolson
4 Replies

4. Shell Programming and Scripting

Counting multiple entries in a file using awk

Hi, I have a big file (~960MB) having epoch time values (~50 million entries) which looks like 897393601 897393601 897393601 897393601 897393602 897393602 897393602 897393602 897393602 897393603 897393603 897393603 897393603 and so on....each time stamp has more than one... (6 Replies)
Discussion started by: sajal.bhatia
6 Replies

5. Shell Programming and Scripting

Counting lines in multiple files

Hi, I have couple of .txt files (say 50 files) in a folder. For each file: I need to get the number of lines in each file and then that count -1 (I wanted to exclude the header. Then sum the counts of all files and output the total sum. Is there an efficient way to do this using shell... (7 Replies)
Discussion started by: Lucky Ali
7 Replies

6. Shell Programming and Scripting

pattern search for multiple log files and counting

I have 10 appservers and each appserver has 4 jvms . Each of these logs is archived and stored on a nfs directory . For example the files are /logs/200907/ap1-jvm1.server.log.20090715.gz /logs/200907/ap2-jvm2.server.log.20090714.gz /logs/200908/ap1-jvm1.server.log.20090812.gz I want to... (3 Replies)
Discussion started by: gubbu
3 Replies

7. Solaris

Counting up files

Hi, I have a load of if statements that look for files in a directory, I want to be able to count them up and the total files confirmed in an email? I ahve tried expr but i this does not work and it only reads in the first if and ignores the rest. Please see script, #!/bin/ksh ... (2 Replies)
Discussion started by: Pablo_beezo
2 Replies

8. Shell Programming and Scripting

Help with counting files please

Hi all. If I have a unix directory with multiple files, lets say, I have some with .dat extensions, some with .txt extensions, etc etc. How in a script would I provide a count of all the different file types (so, the different extensions, I guess) in the directory?? So if I had: test.dat... (6 Replies)
Discussion started by: gerard1
6 Replies

9. UNIX for Dummies Questions & Answers

counting files

Which one line command can count and print to the screen the number of files containing "a" or "A" in their name (5 Replies)
Discussion started by: Edy
5 Replies
Login or Register to Ask a Question