Visit Our UNIX and Linux User Community


awk or sed script to count number of occurrences and creating an average


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers awk or sed script to count number of occurrences and creating an average
# 1  
Old 09-26-2019
awk or sed script to count number of occurrences and creating an average

Hi Friends ,

I am having one problem as stated file .
Having an input CSV file as shown in the code

Code:
U_TOP_LOGIC/U_HPB2/U_HBRIDGE2/i_core/i_paddr_reg_2_/Q,1,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,0,0
U_TOP_LOGIC/U_HPB2/U_HBRIDGE2/i_core/i_paddr_reg_4_/Q,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,0,0
U_TOP_LOGIC/U_HPB2/U_HBRIDGE2/i_core/i_paddr_reg_5_/Q,0,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0
U_TOP_LOGIC/U_HPB2/U_HBRIDGE2/i_core/i_paddr_reg_3_/Q,1,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,0,0

I need to find the number of times "1" occurs in each line and it should go as the last column in each line
Similarly I need to find the number of times "0" occurs in each line and it should go as the last column in each line
I need to create the average of [ number of times "1" occurs / number of "1" or "0" occurs] in each line
I need to create the average of [ number of times "0" occurs / number of "1" or "0" occurs] in each line

Output file should be something like this
Code:
U_TOP_LOGIC/U_HPB2/U_HBRIDGE2/i_core/i_paddr_reg_2_/Q,1,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,0,0,{Number of 1 occurence}, {Number of 0 occurence} , {Number of 1 / Number of 1 and 0 } , {Number of 0 / Number of 1 and Number of 0}
U_TOP_LOGIC/U_HPB2/U_HBRIDGE2/i_core/i_paddr_reg_4_/Q,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,0,0,{Number of 1 occurrence}, {Number of 0 occurrence} , {Number of 1 / Number of 1 and 0 } , {Number of 0 / Number of 1 and Number of 0}
U_TOP_LOGIC/U_HPB2/U_HBRIDGE2/i_core/i_paddr_reg_5_/Q,0,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,{Number of 1 occurrence}, {Number of 0 occurrence} , {Number of 1 / Number of 1 and 0 } , {Number of 0 / Number of 1 and Number of 0}
U_TOP_LOGIC/U_HPB2/U_HBRIDGE2/i_core/i_paddr_reg_3_/Q,1,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,0,0,{Number of 1 occurrence}, {Number of 0 occurrence} , {Number of 1 / Number of 1 and 0 } , {Number of 0 / Number of 1 and Number of 0}

Could you help me out ?

Thanks
Kshitij Smilie
# 2  
Old 09-26-2019
Are those always ones and zeroes?Always starting from field 2?
# 3  
Old 09-26-2019
Awk or sed script

Yes Rudic , yes all 1's and 0's starting from field 2

Thanks
Kshitij
# 4  
Old 09-26-2019
Try
Code:
awk -F, -vOFS=, '{SUM = 0; C = NF - 1; for (i=2; i<=NF; i++) SUM += $i; print $0, SUM, C - SUM, SUM / C, (C - SUM) / C}' file
U_TOP_LOGIC/U_HPB2/U_HBRIDGE2/i_core/i_paddr_reg_2_/Q,1,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,0,0,33,34,0.492537,0.507463
U_TOP_LOGIC/U_HPB2/U_HBRIDGE2/i_core/i_paddr_reg_4_/Q,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,0,0,33,34,0.492537,0.507463
U_TOP_LOGIC/U_HPB2/U_HBRIDGE2/i_core/i_paddr_reg_5_/Q,0,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,32,35,0.477612,0.522388
U_TOP_LOGIC/U_HPB2/U_HBRIDGE2/i_core/i_paddr_reg_3_/Q,1,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,0,0,33,34,0.492537,0.507463

# 5  
Old 10-03-2019
Thanks a lot !

Previous Thread | Next Thread
Test Your Knowledge in Computers #993
Difficulty: Medium
Illumination is a free and open-source Unix operating system based on OpenSolaris.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Count the number of string occurrences to display 0 entries in output

Hello Friends, Can somebody assist an issue I am having? I have a separate file with a list of account ids XXX200B02Y01 XXX200B03Y01 XXX200B05Y01 XXX200B07Y01 XXX200B08Y01 I call the file, and run an egrep against a directory and logfiles AccountID=$(cat... (2 Replies)
Discussion started by: liketheshell
2 Replies

2. Shell Programming and Scripting

awk Group By and count string occurrences

Hi Gurus, I'm scratching my head over and over and couldn't find the the right way to compose this AWK properly - PLEASE HELP :confused: Input: c,d,e,CLICK a,b,c,CLICK a,b,c,CONV c,d,e,CLICK a,b,c,CLICK a,b,c,CLICK a,b,c,CONV b,c,d,CLICK c,d,e,CLICK c,d,e,CLICK b,c,d,CONV... (6 Replies)
Discussion started by: Royi
6 Replies

3. Shell Programming and Scripting

Script to count word occurrences, but exclude some?

I am trying to count the occurrences of ALL words in a file. However, I want to exclude certain words: short words (i.e. <3 chars), and words contained in an blacklist file. There is also a desire to count words that are capitalized (e.g. proper names). I am not 100% sure where the line on... (5 Replies)
Discussion started by: Cronk
5 Replies

4. Shell Programming and Scripting

Count occurrences in awk

Hello, I have an output from GDB with many entries that looks like this 0x00007ffff7dece94 39 in dl-fini.c 0x00007ffff7dece97 39 in dl-fini.c 0x00007ffff7ab356c 50 in exit.c 0x00007ffff7aed9db in _IO_cleanup () at genops.c:1022 115 in dl-fini.c 0x00007ffff7decf7b in _dl_sort_fini (l=0x0,... (6 Replies)
Discussion started by: ikke008
6 Replies

5. Shell Programming and Scripting

Help with Unix and Awk to count number of occurrences

Hi, I have a file (movies.sh), this file contains list of movies such as I want to redirect the movies from movies.sh to file_to_process to allow me process the file with out losing anything. I have tried Movies.sh >> file_to_process But I want to add the row number to the data... (2 Replies)
Discussion started by: INHF
2 Replies

6. Shell Programming and Scripting

count and number instances of a character in sed or awk

I currently use LaTeX together with a sed script to set cloze test papers for my students. I currently pepend and equals sign to the front of the words I want to leave out in the finished test, =perpendicular, for example. I am able to number the blanks using a variable in LaTeX. I would like to... (8 Replies)
Discussion started by: maouinin
8 Replies

7. Shell Programming and Scripting

Count the number of occurrences of a pattern between each occurrence of a different pattern

I need to count the number of occurrences of a pattern, say 'key', between each occurrence of a different pattern, say 'lu'. Here's a portion of the text I'm trying to parse: lu S1234L_149_m1_vg.6, part-att 1, vdp-att 1 p-reserver IID 0xdb registrations: key 4156 4353 0000 0000 ... (3 Replies)
Discussion started by: slipstream
3 Replies

8. Shell Programming and Scripting

Count the number of occurrences of the word

I am a newbie in UNIX shell script and seeking help on this UNIX function. Please give me a hand. Thanks. I have a large file. Named as 'MyFile'. It was tab-delmited. I am told to write a shell function that counts the number of occurrences of the ord “mysring” in the file 'MyFile'. (1 Reply)
Discussion started by: duke0001
1 Replies

9. UNIX for Dummies Questions & Answers

count number of fields not using SED or AWK

hi forums i need help with a little problem i am having. i need to count the number of fields that are in a saved variable so i can use that number to make a different function work properly. is there a way of doing this without using SED/AWK? anything would be greatly appreciated (4 Replies)
Discussion started by: strasner
4 Replies

10. UNIX for Dummies Questions & Answers

How to count number of occurrences of a "|" from a variable?

I have a variable, var="some1|some2|some3" I want to know how many "|" are in $var. When I say echo $var | grep -c '|' I am getting only 1 :confused: :confused: :confused: ? (4 Replies)
Discussion started by: jingi1234
4 Replies

Featured Tech Videos