Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Count number of unique values in each column of array

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Count number of unique values in each column of array
# 8  
Old 01-15-2018
Note that the output produced by the code suggested by Yoda and Chubler_XL in posts #2 and #3 is in the format specified in post #1. To get expanded output (with totals for each column and for each row as a sanity check (with *** at the end of any line where the checks fail) in a format closer to that shown in post #4, one could try the following:
Code:
awk '
{	for(i = 1; i <= NF; i++) {
		c[i, $i]++
		v[$i]
	}
}
END {	printf("Value")
	for(i in v)
		printf("\t%s", i)
	printf("\ttotal\n")
	for(i = 1; i <= NF; i++) {
		printf("COL%d", i)
		t = 0
		for(j in v) {
			printf("\t%d", c[i, j])
			t += c[i, j]
			C[j] += c[i, j]
		}
		printf("\t%d%s\n", t, (t == NR) ? "" : "***")
	}
	printf("Total")
	t = 0
	for(i in v) {
		printf("\t%d", C[i])
		t += C[i]
	}
	printf("\t%d%s\n", t, (t == NR * NF) ? "" : "***")
}' a2.txt

which with the sample input file attached to post #6 produces the output:
Code:
Value	./.	0/0	0/1	1/1	total
COL1	0	61	18	4	83
COL2	0	63	14	6	83
COL3	0	59	22	2	83
COL4	0	64	16	3	83
COL5	0	60	21	2	83
COL6	0	61	20	2	83
COL7	0	64	17	2	83
COL8	0	60	23	0	83
COL9	0	56	25	2	83
COL10	0	66	15	2	83
COL11	0	63	18	2	83
COL12	0	62	20	1	83
COL13	0	63	15	5	83
COL14	0	54	26	3	83
COL15	0	63	18	2	83
COL16	1	66	15	1	83
COL17	0	65	16	2	83
COL18	0	63	16	4	83
COL19	2	59	20	2	83
COL20	0	55	22	6	83
COL21	0	63	18	2	83
COL22	0	67	16	0	83
COL23	0	60	17	6	83
COL24	0	57	22	4	83
COL25	0	55	25	3	83
COL26	0	62	18	3	83
COL27	0	55	23	5	83
COL28	0	57	23	3	83
COL29	1	49	28	5	83
COL30	0	52	28	3	83
COL31	0	61	21	1	83
COL32	0	59	22	2	83
COL33	0	56	23	4	83
COL34	0	55	24	4	83
COL35	0	63	19	1	83
COL36	0	57	23	3	83
COL37	0	62	18	3	83
COL38	0	61	21	1	83
COL39	0	59	21	3	83
COL40	0	56	21	6	83
COL41	0	56	24	3	83
COL42	0	60	21	2	83
COL43	0	60	19	4	83
COL44	0	56	23	4	83
COL45	0	63	20	0	83
COL46	0	61	20	2	83
COL47	0	55	27	1	83
COL48	0	60	17	6	83
COL49	0	59	22	2	83
COL50	0	63	16	4	83
COL51	0	64	15	4	83
COL52	0	61	17	5	83
COL53	0	63	17	3	83
COL54	0	61	16	6	83
COL55	1	64	17	1	83
COL56	0	59	20	4	83
COL57	0	58	16	9	83
COL58	0	55	25	3	83
COL59	0	58	18	7	83
COL60	0	57	19	7	83
COL61	0	57	21	5	83
COL62	0	52	23	8	83
COL63	0	55	22	6	83
COL64	0	58	19	6	83
COL65	0	57	19	7	83
COL66	0	59	21	3	83
COL67	0	63	18	2	83
COL68	0	57	22	4	83
COL69	0	55	26	2	83
COL70	0	62	16	5	83
COL71	1	47	29	6	83
COL72	1	58	16	8	83
COL73	0	60	17	6	83
COL74	0	61	19	3	83
COL75	0	60	21	2	83
COL76	0	53	28	2	83
COL77	0	54	22	7	83
COL78	0	57	22	4	83
COL79	0	57	20	6	83
COL80	0	58	21	4	83
COL81	0	54	20	9	83
COL82	0	57	19	7	83
COL83	0	52	21	10	83
COL84	0	61	18	4	83
COL85	0	57	24	2	83
COL86	0	56	22	5	83
COL87	0	54	21	8	83
COL88	0	58	18	7	83
COL89	0	56	22	5	83
COL90	0	60	18	5	83
COL91	0	64	18	1	83
COL92	0	61	21	1	83
COL93	0	58	18	7	83
COL94	0	58	23	2	83
COL95	0	62	20	1	83
COL96	0	63	18	2	83
COL97	0	60	20	3	83
COL98	0	65	18	0	83
COL99	0	57	23	3	83
COL100	0	61	19	3	83
COL101	0	60	19	4	83
COL102	0	55	27	1	83
COL103	0	64	17	2	83
COL104	1	63	17	2	83
COL105	0	61	22	0	83
COL106	0	54	29	0	83
COL107	0	60	22	1	83
COL108	0	61	20	2	83
COL109	0	61	19	3	83
COL110	0	57	23	3	83
COL111	0	58	19	6	83
COL112	0	61	19	3	83
COL113	0	61	20	2	83
COL114	0	58	21	4	83
COL115	0	61	19	3	83
COL116	0	64	17	2	83
COL117	0	60	19	4	83
COL118	0	62	20	1	83
COL119	0	63	18	2	83
COL120	0	60	21	2	83
COL121	0	63	18	2	83
COL122	0	67	15	1	83
COL123	0	61	16	6	83
COL124	0	67	12	4	83
COL125	0	58	21	4	83
COL126	0	58	23	2	83
COL127	0	60	21	2	83
COL128	0	62	20	1	83
COL129	0	60	21	2	83
COL130	0	63	19	1	83
COL131	0	65	13	5	83
COL132	0	62	19	2	83
COL133	0	57	23	3	83
COL134	0	60	22	1	83
COL135	0	68	13	2	83
COL136	0	62	17	4	83
COL137	0	58	20	5	83
COL138	0	65	12	6	83
COL139	0	54	27	2	83
COL140	0	69	12	2	83
COL141	0	58	23	2	83
COL142	0	62	20	1	83
COL143	0	67	14	2	83
COL144	0	62	19	2	83
COL145	0	59	19	5	83
COL146	0	63	17	3	83
COL147	0	60	22	1	83
COL148	0	65	15	3	83
COL149	1	54	25	3	83
COL150	2	55	20	6	83
COL151	0	71	10	2	83
COL152	2	61	16	4	83
COL153	0	64	19	0	83
COL154	0	60	18	5	83
COL155	0	61	19	3	83
COL156	0	63	15	5	83
COL157	1	69	13	0	83
COL158	1	58	20	4	83
COL159	1	60	19	3	83
COL160	1	57	22	3	83
COL161	0	63	20	0	83
COL162	1	56	22	4	83
COL163	0	56	23	4	83
COL164	1	56	21	5	83
COL165	0	55	27	1	83
COL166	0	63	15	5	83
COL167	0	56	23	4	83
COL168	0	62	18	3	83
COL169	0	54	25	4	83
COL170	0	56	22	5	83
COL171	0	63	16	4	83
COL172	0	53	23	7	83
COL173	0	65	16	2	83
COL174	0	60	20	3	83
COL175	0	54	21	8	83
COL176	0	65	13	5	83
COL177	0	66	14	3	83
COL178	0	59	21	3	83
COL179	0	83	0	0	83
COL180	0	53	27	3	83
COL181	0	64	16	3	83
COL182	0	56	23	4	83
COL183	0	56	22	5	83
COL184	0	55	25	3	83
COL185	0	64	19	0	83
COL186	0	64	17	2	83
COL187	0	53	26	4	83
COL188	0	61	20	2	83
COL189	0	59	23	1	83
COL190	0	64	16	3	83
COL191	0	55	22	6	83
COL192	0	68	14	1	83
COL193	0	59	20	4	83
COL194	0	64	16	3	83
COL195	0	64	16	3	83
COL196	0	69	13	1	83
COL197	0	64	16	3	83
COL198	0	58	23	2	83
COL199	0	61	20	2	83
COL200	0	55	24	4	83
COL201	0	59	20	4	83
COL202	0	60	21	2	83
COL203	0	61	20	2	83
COL204	0	58	24	1	83
COL205	0	59	18	6	83
COL206	0	59	22	2	83
COL207	0	58	22	3	83
COL208	0	56	26	1	83
COL209	0	58	21	4	83
COL210	0	66	16	1	83
COL211	0	62	20	1	83
COL212	0	61	20	2	83
COL213	0	60	21	2	83
COL214	0	62	20	1	83
COL215	0	58	22	3	83
COL216	0	62	21	0	83
COL217	0	60	19	4	83
COL218	0	62	20	1	83
COL219	0	60	20	3	83
COL220	0	62	20	1	83
COL221	0	58	23	2	83
COL222	0	55	28	0	83
COL223	0	61	20	2	83
COL224	0	64	17	2	83
COL225	0	55	23	5	83
COL226	0	64	16	3	83
COL227	0	60	16	7	83
COL228	0	59	20	4	83
COL229	0	59	22	2	83
COL230	0	51	32	0	83
COL231	0	60	16	7	83
COL232	0	65	15	3	83
COL233	0	61	21	1	83
COL234	0	66	14	3	83
COL235	0	59	22	2	83
COL236	0	67	13	3	83
COL237	0	58	24	1	83
COL238	0	59	20	4	83
COL239	0	60	22	1	83
COL240	1	64	16	2	83
COL241	0	65	16	2	83
COL242	1	53	27	2	83
COL243	0	65	17	1	83
COL244	0	58	24	1	83
COL245	0	60	18	5	83
COL246	0	61	18	4	83
COL247	0	63	16	4	83
COL248	0	53	26	4	83
COL249	1	59	20	3	83
COL250	0	58	22	3	83
COL251	0	62	20	1	83
COL252	0	62	19	2	83
COL253	0	52	25	6	83
COL254	0	54	25	4	83
COL255	0	68	9	6	83
COL256	0	56	21	6	83
COL257	0	56	22	5	83
COL258	0	63	18	2	83
COL259	0	54	29	0	83
COL260	1	53	16	13	83
COL261	0	55	23	5	83
COL262	1	52	24	6	83
COL263	0	55	20	8	83
COL264	0	48	26	9	83
COL265	0	59	17	7	83
COL266	0	59	18	6	83
COL267	0	56	24	3	83
COL268	0	53	21	9	83
COL269	0	62	17	4	83
COL270	0	55	23	5	83
COL271	0	55	23	5	83
COL272	0	58	20	5	83
COL273	0	56	22	5	83
COL274	0	54	25	4	83
COL275	0	56	22	5	83
COL276	0	50	28	5	83
COL277	2	50	28	3	83
COL278	0	58	20	5	83
COL279	0	59	19	5	83
COL280	0	56	17	10	83
COL281	0	64	13	6	83
COL282	0	60	16	7	83
COL283	0	60	17	6	83
COL284	0	61	14	8	83
COL285	0	54	24	5	83
COL286	0	55	27	1	83
COL287	0	57	21	5	83
COL288	0	56	26	1	83
COL289	0	57	21	5	83
COL290	0	55	24	4	83
COL291	1	52	29	1	83
COL292	1	59	21	2	83
COL293	0	52	30	1	83
COL294	0	60	20	3	83
COL295	0	55	24	4	83
COL296	2	63	16	2	83
COL297	0	57	23	3	83
COL298	0	57	23	3	83
COL299	0	62	20	1	83
COL300	0	57	20	6	83
COL301	0	64	16	3	83
COL302	0	55	23	5	83
COL303	0	64	17	2	83
COL304	0	0	0	83	83
Total	30	18008	6078	1116	25232

The order of the middle columns of the above output may vary with different implementations of awk, but contents of the columns should be consistent.
This User Gave Thanks to Don Cragun For This Post:
Geneanalyst
# 9  
Old 01-15-2018
Double post
# 10  
Old 01-15-2018
Quote:
Originally Posted by Don Cragun
When I slightly modify the code suggested by Chubler_XL to be:
Code:
awk '
{ 
   mc = NF > mc ? NF : mc
   for(i=NF; i; i--) {
      T[$i]
      C[i FS $i]++
   }
}
END {
  printf "Value"
  for(i=1; i<=mc;i++) printf "\tCOL%d",i

  for(v in T) {
     printf "\n%s", v
     for(i=1; i<=mc;i++) printf "\t%d",C[i FS v]
  }
  printf "\n"
}' a2.txt

and store this in a file named Chubler_XL, make it executable and run the command:
Code:
./Chubler_XL > Chubler_XL.out

and I slightly modify the code suggested by Yoda to be:
Code:
awk '
        BEGIN {
                n = split ( "./. 0/0 0/1 1/1", T )
        }
        {
                for ( i = 1; i <= NF; i++ )
                        R[i FS $i] += 1
        }
        END {
                printf "VAL\t"
                for ( i = 1; i <= NF; i++ )
                        printf "COL%d\t", i
                printf "\n"

                for ( j = 1; j <= n; j++ )
                {
                        printf "%s\t", T[j]
                        for ( i = 1; i <= NF; i++ )
                                printf "%d\t", R[i FS T[j]]
                        printf "\n"
                }
        }
' a2.txt

and store this in a file named Yoda, make it executable and run the command:
Code:
./Yoda > Yoda.out

and I write the code:
Code:
awk -v line_count="$(wc -l < a2.txt)" '
function check() {
    printf("Checking fields 2 through %d in file: %s\n", NF, f)
    for(i = 2; i <= NF; i++)
        if(c[i] != line_count)
            printf("file %s: field %d count %d\n", f, i, c[i])
    split("", c)
}
FNR == 1 {
    line_count += 0
    if(f == "")
        printf("Evaluating output produced from %d lines in a2.txt\n",
            line_count)
    else
        check()
    f = FILENAME
    next
}
{    for(i = 2; i <= NF; i++)
        c[i] += $i
}
END {    check()
}' *.out

and store that in a file named counter, make it executable, and run it, I get the output:
Code:
Evaluating output produced from 83 lines in a2.txt
Checking fields 2 through 305 in file: Chubler_XL.out
Checking fields 2 through 305 in file: Yoda.out

which shows that the sums of the values for each of the 304 fields does indeed equal the number of lines found in the file you attached in post #6.

I see no indication that either of these suggestions is producing results that are incorrect although neither of them produce output that is at all close to the output you showed us in post #4. I do note that the output you showed us in post #4 only shows output for the three values "0/0", "0/1", and "1/1"; but the data in a2.txt also includes some entries with the value "./." which is included in the output produced by the code Chubler_XL suggested and in the output produced by the code Yoda suggested (after changing it to look for those four values instead of the values, "A", "B", "C", and "D" that you said were included as values in your statements in post #1.

If you'd like to show us the code you used to produce the output for the 1st 26 columns you showed us in post #4, maybe we can help you explain why that code failed to correctly interpret the output produced by Chubler_XL's code or Yoda's code.


Don, thanks for your time and for Chubler XL's and Yoda's time, great work and impressive expertise! it works when I copy the code into a file and make it executable, but I do have a dumb question; why does it give weird results when the code is simply copy pasted into terminal (guessing that some parts of the code are not executed)
Chubler_XL.txt (5.9 KB) 
Last edited by Geneanalyst; 01-15-2018 at 04:24 PM.. Reason: ran as executable
# 11  
Old 01-15-2018
Copying text containing <tab> characters into a bash shell with filename completion enabled can cause all sorts of strange things to happen. It may fail for other reasons as well. (I usually use ksh instead of bash.)

Have you tried running the code I suggested in post #8 which seems to provide output in a format closer to what you now seem to want? The code Yoda and Chubler_XL suggested in posts #2 and post #3 is in the format you specified in post #1 (which is totally different from your latest output format).
This User Gave Thanks to Don Cragun For This Post:
Geneanalyst
# 12  
Old 01-15-2018
Quote:
Originally Posted by Don Cragun
Copying text containing <tab> characters into a bash shell with filename completion enabled can cause all sorts of strange things to happen. It may fail for other reasons as well. (I usually use ksh instead of bash.)

Have you tried running the code I suggested in post #8 which seems to provide output in a format closer to what you now seem to want? The code Yoda and Chubler_XL suggested in posts #2 and post #3 is in the format you specified in post #1 (which is totally different from your latest output format).
Again thanks for all you do to help out all the folks out there like myself. Great forum BTW.

I am sure many people in addition to geneticists will find this code useful.

Yes, I like your latest code, however, I need the actual sample names printed instead of the header consisting of COL1, COL2, ....

To do this I am grabbing the header with sample names (columns 10 thru N) from a different file. I use:

Code:
awk 'FNR==28 {for(i=9;i<=NF;i++){printf "%s ", $i}; printf "\n"}' a.txt

in the shell right before your code to print a header containing actual sample names which are contained in a.txt. So this is the last half of my code:

Code:
# Strip columns 1-9 and write to a2.txt
awk '{for(i=10;i<=NF;i++){printf "%s ", $i}; printf "\n"}' a1.txt > a2.txt
# PRINT HEADER
awk 'FNR==28 {for(i=9;i<=NF;i++){printf "%s ", $i}; printf "\n"}' a.txt
# Print count of 0/0 0/1 1/1 ./. for each sample in the run
awk '
{ 
   mc = NF > mc ? NF : mc
   for(i=NF; i; i--) {
      T[$i]
      C[i FS $i]++
   }
}
END {
  for(v in T) {
     printf "\n%s", v
     for(i=1; i<=mc;i++) printf "\t%d",C[i FS v]
  }
  printf "\n"
}' a2.txt


If I use your latest code which transposes the output, I would have to modify my code (highlighted above) to transpose the header containing the sample names also. Any ideas.
# 13  
Old 01-15-2018
Quote:
Originally Posted by Geneanalyst
Again thanks for all you do to help out all the folks out there like myself. Great forum BTW.

I am sure many people in addition to geneticists will find this code useful.

Yes, I like your latest code, however, I need the actual sample names printed instead of the header consisting of COL1, COL2, ....

To do this I am grabbing the header with sample names (columns 10 thru N) from a different file. I use:

Code:
awk 'FNR==28 {for(i=9;i<=NF;i++){printf "%s ", $i}; printf "\n"}' a.txt

in the shell right before your code to print a header containing actual sample names which are contained in a.txt. So this is the last half of my code:

Code:
# Strip columns 1-9 and write to a2.txt
awk '{for(i=10;i<=NF;i++){printf "%s ", $i}; printf "\n"}' a1.txt > a2.txt
# PRINT HEADER
awk 'FNR==28 {for(i=9;i<=NF;i++){printf "%s ", $i}; printf "\n"}' a.txt
# Print count of 0/0 0/1 1/1 ./. for each sample in the run
awk '
{ 
   mc = NF > mc ? NF : mc
   for(i=NF; i; i--) {
      T[$i]
      C[i FS $i]++
   }
}
END {
  for(v in T) {
     printf "\n%s", v
     for(i=1; i<=mc;i++) printf "\t%d",C[i FS v]
  }
  printf "\n"
}' a2.txt


If I use your latest code which transposes the output, I would have to modify my code (highlighted above) to transpose the header containing the sample names also. Any ideas.
I am glad that you find this forum useful. We all hope to help each other learn how to better use the tools that are available on our systems.

Can we see the first half of your code, samples of your a.txt and a1.txt, and the output that you are hoping to produce from those input files?

Note that you say you're printing headers from fields 10 through N from line 28th line of a.txt, but your code is printing headers starting in field 9 (not 10). What is the field separator in that file?

Note also that the three awk commands you have shown us from the last half of your code could more efficiently be done with a single awk command without needing to take filesystem space for the file a2.txt (unless there is some other reason why you need to have that as a separate file).
These 2 Users Gave Thanks to Don Cragun For This Post:
Chubler_XL Geneanalyst
# 14  
Old 01-15-2018
Quote:
Originally Posted by Don Cragun
I am glad that you find this forum useful. We all hope to help each other learn how to better use the tools that are available on our systems.

Can we see the first half of your code, samples of your a.txt and a1.txt, and the output that you are hoping to produce from those input files?

Note that you say you're printing headers from fields 10 through N from line 28th line of a.txt, but your code is printing headers starting in field 9 (not 10). What is the field separator in that file?

Note also that the three awk commands you have shown us from the last half of your code could more efficiently be done with a single awk command without needing to take filesystem space for the file a2.txt (unless there is some other reason why you need to have that as a separate file).

Thanks Don. Here is the my whole code:

Code:
# Identify target derived alleles to the exculsion of outgroups
awk -F "\t" '(NR>28) { if(($313 == "0/0") && ($314 != "0/0") && ($315 != "0/0") && ($316 != "0/0") && ($317 != "0/0") && ($318 != "0/0") && ($319 != "0/0")) {print $0} }' a.txt > a1.txt
#
# Strip columns 1-9 and write to a2.txt
awk '{for(i=10;i<=NF;i++){printf "%s ", $i}; printf "\n"}' a1.vcf > a2.txt
# PRINT HEADER
awk 'FNR==28 {for(i=9;i<=NF;i++){printf "%s ", $i}; printf "\n"}' a.txt
# Print count of 0/0 0/1 1/1 ./. for each sample in the run
awk '
{ 
   mc = NF > mc ? NF : mc
   for(i=NF; i; i--) {
      T[$i]
      C[i FS $i]++
   }
}
END {
  for(v in T) {
     printf "\n%s", v
     for(i=1; i<=mc;i++) printf "\t%d",C[i FS v]
  }
  printf "\n"
}' a2.txt

In the 1st part, the input data a.txt (attached file. I only copied the 1st 100 rows to include the header which has the sample names) is queried. Columns 1-9 do not contain relevant information. Columns 313-319 contain the target samples against which all the test samples are compared ( columns 10-312).

The rows that survive the comparison operation are written to a1.txt ( columns 1-9 don't contain relevant information).

Next the header containing the sample names is extracted from a.txt, and your code is executed for counting the number of unique values.

Next I manually add the values in the 0/1 and 1/1 columns, and create a totals column. I then sort the total column from high to low. The sample with the highest total indicates the most similarity to target sample (column 313).

I like the way you transposed the result and would like to also transpose the header with sample names. So instead of COL1. COL2, ..., I would like the sample names from row 28 ( columns 10-319), such as shown below.

Code:
FORMAT    1/1    0/0    0/1    TOTAL 0/0 & 0/1
.Kurd_C3_ID001    78    183    201    384
Balochi_HGDP00052    86    175    201    376
Balochi_HGDP00054    71    166    225    391
Balochi_HGDP00056    71    158    233    391
Balochi_HGDP00058    90    168    204    372
Balochi_HGDP00062    91    148    223    371
Balochi_HGDP00064    85    183    194    377
Balochi_HGDP00066    79    185    198    383
Balochi_HGDP00068    95    163    202    365
Balochi_HGDP00072    75    168    217    385
Balochi_HGDP00074    80    198    183    381
Balochi_HGDP00078    89    171    199    370
Balochi_HGDP00080    88    149    222    371
Balochi_HGDP00082    85    179    195    374
Balochi_HGDP00086    102    162    198    360
Balochi_HGDP00088    89    175    194    369
Balochi_HGDP00090    87    177    197    374
Balochi_HGDP00092    87    191    184    375
Balochi_HGDP00096    87    166    207    373
Balochi_HGDP00098    95    190    175    365
GujaratiD_NA20847    74    168    220    388
GujaratiD_NA20899    86    183    193    376

Moderator's Comments:
Mod Comment Please use CODE tags around sample input and output as well as around code segments.
a.txt (57.9 KB) 
a1.txt (575.5 KB) 
Last edited by Don Cragun; 01-16-2018 at 05:14 AM.. Reason: Add additional CODE tags.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Count unique column

Hello, I am trying to count unique rows in my file based on 4 columns (2-5) and to output its frequency in a sixth column. My file is tab delimited My input file looks like this: Colum1 Colum2 Colum3 Colum4 Coulmn5 1.1 100 100 a b 1.1 100 100 a c 1.2 200 205 a d 1.3 300 301 a y 1.3 300... (6 Replies)
Discussion started by: nans
6 Replies

2. UNIX for Beginners Questions & Answers

Awk: count unique element of array

Hi, tab-separated input: blabla_1 A,B,C,C blabla_2 A,E,G blabla_3 R,Q,A,B,C,R,Q output: blabla_1 3 blabla_2 3 blabla_3 5 After splitting $2 in an array, I am trying to store the number of unique elements in a variable, but have some difficulties resetting the variable to 0 before... (6 Replies)
Discussion started by: beca123456
6 Replies

3. Shell Programming and Scripting

Print count of unique values

Hello experts, I am converting a number into its binary output as : read n echo "obase=2;$n" | bc I wish to count the maximum continuous occurrences of the digit 1. Example : 1. The binary equivalent of 5 = 101. Hence the output must be 1. 2. The binary... (3 Replies)
Discussion started by: H squared
3 Replies

4. Shell Programming and Scripting

Count occurrence of column one unique value having unique second column value

Hello Team, I need your help on the following: My input file a.txt is as below: 3330690|373846|108471 3330690|373846|108471 0640829|459725|100001 0640829|459725|100001 3330690|373847|108471 Here row 1 and row 2 of column 1 are identical but corresponding column 2 value are... (4 Replies)
Discussion started by: angshuman
4 Replies

5. Shell Programming and Scripting

Count frequency of unique values in specific column

Hi, I have tab-deliminated data similar to the following: dot is-big 2 dot is-round 3 dot is-gray 4 cat is-big 3 hot in-summer 5 I want to count the frequency of each individual "unique" value in the 1st column. Thus, the desired output would be as follows: dot 3 cat 1 hot 1 is... (5 Replies)
Discussion started by: owwow14
5 Replies

6. UNIX for Dummies Questions & Answers

count number of distinct values in each column with awk

Hi ! input: A|B|C|D A|F|C|E A|B|I|C A|T|I|B As the title of the thread says, I would need to get: 1|3|2|4 I tried different variants of this command, but I don't manage to obtain what I need: gawk 'BEGIN{FS=OFS="|"}{for(i=1; i<=NF; i++) a++} END {for (b in a) print b}' input ... (2 Replies)
Discussion started by: beca123456
2 Replies

7. Shell Programming and Scripting

How to count Unique Values from a file.

Hi I have the following info in a file - <Cell id="25D"/> <Cell id="26A"/> <Cell id="26B"/> <Cell id="26C"/> <Cell id="27A"/> <Cell id="27B"/> <Cell id="27C"/> <Cell id="28A"/> I would like to know how would you go about counting all... (4 Replies)
Discussion started by: Prega
4 Replies

8. UNIX for Dummies Questions & Answers

count number of rows based on other column values

Could anybody help with this? I have input below ..... david,39 david,39 emelie,40 clarissa,22 bob,42 bob,42 tim,32 bob,39 david,38 emelie,47 what i want to do is count how many names there are with different ages, so output would be like this .... david,2 emelie,2 clarissa,1... (3 Replies)
Discussion started by: itsme999
3 Replies

9. Shell Programming and Scripting

print unique values of a column and sum up the corresponding values in next column

Hi All, I have a file which is having 3 columns as (string string integer) a b 1 x y 2 p k 5 y y 4 ..... ..... Question: I want get the unique value of column 2 in a sorted way(on column 2) and the sum of the 3rd column of the corresponding rows. e.g the above file should return the... (6 Replies)
Discussion started by: amigarus
6 Replies

10. Shell Programming and Scripting

Not able to read unique values in array

Hi Friends, I am having some trouble reading into an array. Basically, I am trying to grep for a pattern and extract it's value and store the same into an array. For eg., if my input is: <L:RECORD>name=faisel farooq,age=21, company=TCS,project=BT</L:RECORD> <L:RECORD>name=abc... (1 Reply)
Discussion started by: faiz1985
1 Replies
Login or Register to Ask a Question