Total count in each category for given file list


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Total count in each category for given file list
# 1  
Old 09-10-2014
Total count in each category for given file list

I have list of file names in filename.txt
below is file format
>>File1
_________________________
Code:
01~12345~Y~YES~aaaaa~can
02~23456~N~NO~bbbbb~can
.
.
.
99~23

__________________________

Need to find total count from each file depending on specific string and add them to have total count in each category.

Code:
1. directory/${LINE}|tail -1|grep -i "99"|cut -d  '~' -f 2
2. directory/${LINE}|grep '~YES~'| wc -l
3. directory/${LINE}|grep '~No~'| wc -l

I need output as :
Code:
1.total count : 23  #addition of trailer values for each file '99' suggests it's a trailer
2. YES : 10  # total occurrences of '~YES~' 
3. NO : 5     # total ocurrences of '~NO~'


Last edited by santoshdrkr; 09-10-2014 at 11:45 PM.. Reason: Please use [code][/code] tags.
# 2  
Old 09-10-2014
Not sure what you want your output to look like but this might get you started:

Code:
while read file
do
  awk -F~ '
    {if($1=="99")T=$2; else T=x}
    /~YES~/{Y++}
    /~NO~/{N++}
    END{
       print FILENAME
       print "\t99:" T
       print "\tYES:" Y
       print "\tNO:" N
    }' $file
done< filename.txt


Last edited by Chubler_XL; 09-10-2014 at 09:23 PM..
This User Gave Thanks to Chubler_XL For This Post:
# 3  
Old 09-10-2014
Quote:
Originally Posted by Chubler_XL
Not sure what you want your output to look like but this might get you started:

Code:
while read file
do
  awk -F~ '
    {if($1=="99")T=$2; else T=x}
    /~YES~/{Y++}
    /~NO~/{N++}
    END{
       print FILENAME
       print "\t99:" T
       print "\tYES:" Y
       print "\tNO:" N
    }' $file
done< filename.txt

Thanks Chubler_XL,
Can i call two different loops inside one ksh script one for total count and other for YES /No?
# 4  
Old 09-11-2014
Yes but it's less efficient as filename.txt and the individual files are read twice:

Code:
while read file
do
    printf "$file\n\t99: %s\n" $(sed -n '$s/^99~//p' $file)
done < filename.txt

printf "\n\n"

while read file
do
  awk -F~ '
    {if($1=="99")T=$2; else T=x}
    /~YES~/ {Y++}
    /~NO~/{N++}
    END{
       print FILENAME
       print "\tYES:" Y
       print "\tNO:" N
    }' $file
done< filename.txt

This User Gave Thanks to Chubler_XL For This Post:
# 5  
Old 09-11-2014
Thanks dear,

Actually efficiency will not be considered in this case as filename.txt will have maximum 10 record(file names).

Can you please tell me if second loop can find YES / No as addition from all files in filename.txt?
# 6  
Old 09-11-2014
Yes 2nd loop is counting records with ~YES~ and ~NO~

I left a line from the previous solution in that can be removed:

Code:
while read file
do
    printf "$file\n\t99: %s\n" $(sed -n '$s/^99~//p' $file)
done < filename.txt

printf "\n\n"

while read file
do
  awk -F~ '
    /~YES~/ {Y++}
    /~NO~/{N++}
    END{
       print FILENAME
       print "\tYES:" Y
       print "\tNO:" N
    }' $file
done< filename.txt

This User Gave Thanks to Chubler_XL For This Post:
# 7  
Old 09-11-2014
what will be the scripts if we go for merging files and then counting individual count?
I need output in below format
Code:
Total Records : 263
YES : 143
NO : 8
No Response : 102

Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Total record count of all the file present in a directory

Hi All , We need one help on the below requirement.We have multiple pipe delimited .txt file(around 100 .txt files) present on one directory.We need the total record count of all the files present in that directory without header.File format as below : ... (8 Replies)
Discussion started by: STCET22
8 Replies

2. Shell Programming and Scripting

Category and count with awk

I want to categorize and count the as below: Input file: A1 G1 C1 F1 A2 G1 C1 F1 A3 G1 C1 F2 A4 G1 C2 F2 A7 G1 C2 F2 A8 G1 C2 F3 A11 G1 C2 F3 A23 G1 C2 F3 B4 G1 C2 F3 AC4 G2 C3 F4 B6 G2 C4 F4 BB5 G2 C4 F4 A25 G2 C5 F4 B13 G2 C5 F5 D12 G2 C5 F5 D2 G2 C5 F5 (3 Replies)
Discussion started by: aydj
3 Replies

3. UNIX for Dummies Questions & Answers

To count total of specific character in a file and save its value to a variable

Hi all, I have a file that contains characters. How do I get total of spesific character from that file and save the count to a variable for doing for calculation. data.txt 1 2 2 2 2 3 3 4 5 6 7 8 5 4 3 4 (5 Replies)
Discussion started by: weslyarfan
5 Replies

4. Shell Programming and Scripting

Count total duplicates

Hi all, I have found another post threads talking about count duplicate lines, but I am interested in obtain the total number of duplicates. For example: #file.txt a1 a2 a1 a3 a1 a2 a4 a5 #out 3 (lines are duplicates) Thank you! (12 Replies)
Discussion started by: mikloz
12 Replies

5. UNIX for Dummies Questions & Answers

How to find count total number of pattern in a file …?

How to find count total number of pattern in a file … File contains : a.txt ------------- aaa bbb nnn ccc aaa bbb aaa ddd aaa aaa aaa aaa grep -c aaa a.txt Op: 4 ( But my requirement is should count the total no of patterns as 7 ) (4 Replies)
Discussion started by: Jitten
4 Replies

6. UNIX Desktop Questions & Answers

What is the way to get a total count of students and with highest marks from a file?

I have different things that I was trying to do but am kind of struggling with this since I'm a Linux noob. I have a files with student names ,marks,year school the . What is the most efficient way to get a total count of students and student with highest marks Initially I tried to get a count... (1 Reply)
Discussion started by: anil2103
1 Replies

7. UNIX for Dummies Questions & Answers

Grep bunch of gzip files to count based on category

Started using unix commands recently. I have 50 gzip files. I want to grep each of these files for a line count based particular category in column 3. How can I do that? For example Sr.No Date City Description Code Address 1 06/09 NY living here 0909 10st st nyc 2 ... (5 Replies)
Discussion started by: jinxx
5 Replies

8. UNIX for Advanced & Expert Users

Count total file downloaded using FTP

Hi All, I'm developing a FTP script as below: ftp -v -n <IP_ADDRESS> << EOF user avery jSqaqUU2 lcd /directory/folder/ ascii prompt mget * bye EOF I would like to enhance the script to count the total file downloaded. For example, once the script run i want the message "Total <n>... (1 Reply)
Discussion started by: cas553
1 Replies

9. UNIX for Dummies Questions & Answers

Total file size of a subset list

Hello! I'm trying to find out the total file size of a subset list in a directory. For example, I do not need to know the total file size of all the files in a directory, but I need to know what the total size is of say, "ls -l *FEB08*" in a directory. Is there any easy way of doing this? ... (3 Replies)
Discussion started by: tekster757
3 Replies
Login or Register to Ask a Question