Sponsored Content
Top Forums Shell Programming and Scripting Count number of unique values in each column of array Post 303011216 by Don Cragun on Monday 15th of January 2018 09:30:41 AM
Old 01-15-2018
When I slightly modify the code suggested by Chubler_XL to be:
Code:
awk '
{ 
   mc = NF > mc ? NF : mc
   for(i=NF; i; i--) {
      T[$i]
      C[i FS $i]++
   }
}
END {
  printf "Value"
  for(i=1; i<=mc;i++) printf "\tCOL%d",i

  for(v in T) {
     printf "\n%s", v
     for(i=1; i<=mc;i++) printf "\t%d",C[i FS v]
  }
  printf "\n"
}' a2.txt

and store this in a file named Chubler_XL, make it executable and run the command:
Code:
./Chubler_XL > Chubler_XL.out

and I slightly modify the code suggested by Yoda to be:
Code:
awk '
        BEGIN {
                n = split ( "./. 0/0 0/1 1/1", T )
        }
        {
                for ( i = 1; i <= NF; i++ )
                        R[i FS $i] += 1
        }
        END {
                printf "VAL\t"
                for ( i = 1; i <= NF; i++ )
                        printf "COL%d\t", i
                printf "\n"

                for ( j = 1; j <= n; j++ )
                {
                        printf "%s\t", T[j]
                        for ( i = 1; i <= NF; i++ )
                                printf "%d\t", R[i FS T[j]]
                        printf "\n"
                }
        }
' a2.txt

and store this in a file named Yoda, make it executable and run the command:
Code:
./Yoda > Yoda.out

and I write the code:
Code:
awk -v line_count="$(wc -l < a2.txt)" '
function check() {
	printf("Checking fields 2 through %d in file: %s\n", NF, f)
	for(i = 2; i <= NF; i++)
		if(c[i] != line_count)
			printf("file %s: field %d count %d\n", f, i, c[i])
	split("", c)
}
FNR == 1 {
	line_count += 0
	if(f == "")
		printf("Evaluating output produced from %d lines in a2.txt\n",
		    line_count)
	else
		check()
	f = FILENAME
	next
}
{	for(i = 2; i <= NF; i++)
		c[i] += $i
}
END {	check()
}' *.out

and store that in a file named counter, make it executable, and run it, I get the output:
Code:
Evaluating output produced from 83 lines in a2.txt
Checking fields 2 through 305 in file: Chubler_XL.out
Checking fields 2 through 305 in file: Yoda.out

which shows that the sums of the values for each of the 304 fields does indeed equal the number of lines found in the file you attached in post #6.

I see no indication that either of these suggestions is producing results that are incorrect although neither of them produce output that is at all close to the output you showed us in post #4. I do note that the output you showed us in post #4 only shows output for the three values "0/0", "0/1", and "1/1"; but the data in a2.txt also includes some entries with the value "./." which is included in the output produced by the code Chubler_XL suggested and in the output produced by the code Yoda suggested (after changing it to look for those four values instead of the values, "A", "B", "C", and "D" that you said were included as values in your statements in post #1.

If you'd like to show us the code you used to produce the output for the 1st 26 columns you showed us in post #4, maybe we can help you explain why that code failed to correctly interpret the output produced by Chubler_XL's code or Yoda's code.
This User Gave Thanks to Don Cragun For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Not able to read unique values in array

Hi Friends, I am having some trouble reading into an array. Basically, I am trying to grep for a pattern and extract it's value and store the same into an array. For eg., if my input is: <L:RECORD>name=faisel farooq,age=21, company=TCS,project=BT</L:RECORD> <L:RECORD>name=abc... (1 Reply)
Discussion started by: faiz1985
1 Replies

2. Shell Programming and Scripting

print unique values of a column and sum up the corresponding values in next column

Hi All, I have a file which is having 3 columns as (string string integer) a b 1 x y 2 p k 5 y y 4 ..... ..... Question: I want get the unique value of column 2 in a sorted way(on column 2) and the sum of the 3rd column of the corresponding rows. e.g the above file should return the... (6 Replies)
Discussion started by: amigarus
6 Replies

3. UNIX for Dummies Questions & Answers

count number of rows based on other column values

Could anybody help with this? I have input below ..... david,39 david,39 emelie,40 clarissa,22 bob,42 bob,42 tim,32 bob,39 david,38 emelie,47 what i want to do is count how many names there are with different ages, so output would be like this .... david,2 emelie,2 clarissa,1... (3 Replies)
Discussion started by: itsme999
3 Replies

4. Shell Programming and Scripting

How to count Unique Values from a file.

Hi I have the following info in a file - <Cell id="25D"/> <Cell id="26A"/> <Cell id="26B"/> <Cell id="26C"/> <Cell id="27A"/> <Cell id="27B"/> <Cell id="27C"/> <Cell id="28A"/> I would like to know how would you go about counting all... (4 Replies)
Discussion started by: Prega
4 Replies

5. UNIX for Dummies Questions & Answers

count number of distinct values in each column with awk

Hi ! input: A|B|C|D A|F|C|E A|B|I|C A|T|I|B As the title of the thread says, I would need to get: 1|3|2|4 I tried different variants of this command, but I don't manage to obtain what I need: gawk 'BEGIN{FS=OFS="|"}{for(i=1; i<=NF; i++) a++} END {for (b in a) print b}' input ... (2 Replies)
Discussion started by: beca123456
2 Replies

6. Shell Programming and Scripting

Count frequency of unique values in specific column

Hi, I have tab-deliminated data similar to the following: dot is-big 2 dot is-round 3 dot is-gray 4 cat is-big 3 hot in-summer 5 I want to count the frequency of each individual "unique" value in the 1st column. Thus, the desired output would be as follows: dot 3 cat 1 hot 1 is... (5 Replies)
Discussion started by: owwow14
5 Replies

7. Shell Programming and Scripting

Count occurrence of column one unique value having unique second column value

Hello Team, I need your help on the following: My input file a.txt is as below: 3330690|373846|108471 3330690|373846|108471 0640829|459725|100001 0640829|459725|100001 3330690|373847|108471 Here row 1 and row 2 of column 1 are identical but corresponding column 2 value are... (4 Replies)
Discussion started by: angshuman
4 Replies

8. Shell Programming and Scripting

Print count of unique values

Hello experts, I am converting a number into its binary output as : read n echo "obase=2;$n" | bc I wish to count the maximum continuous occurrences of the digit 1. Example : 1. The binary equivalent of 5 = 101. Hence the output must be 1. 2. The binary... (3 Replies)
Discussion started by: H squared
3 Replies

9. UNIX for Beginners Questions & Answers

Awk: count unique element of array

Hi, tab-separated input: blabla_1 A,B,C,C blabla_2 A,E,G blabla_3 R,Q,A,B,C,R,Q output: blabla_1 3 blabla_2 3 blabla_3 5 After splitting $2 in an array, I am trying to store the number of unique elements in a variable, but have some difficulties resetting the variable to 0 before... (6 Replies)
Discussion started by: beca123456
6 Replies

10. UNIX for Beginners Questions & Answers

Count unique column

Hello, I am trying to count unique rows in my file based on 4 columns (2-5) and to output its frequency in a sixth column. My file is tab delimited My input file looks like this: Colum1 Colum2 Colum3 Colum4 Coulmn5 1.1 100 100 a b 1.1 100 100 a c 1.2 200 205 a d 1.3 300 301 a y 1.3 300... (6 Replies)
Discussion started by: nans
6 Replies
All times are GMT -4. The time now is 11:10 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy