Column manipulation and counter


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Column manipulation and counter
# 1  
Old 03-28-2013
Column manipulation and counter

Hello,

I was wondering how I could change this input:

Code:
cat f1

Quote:
col1 col2
DOG5 go
DOG5 go
DOG24 stop
DOG76 stop
DOG24 yield
DOG24 halt

to produce this output:
Quote:
col1 col2 go stop yield halt
2 DOG5 2
1 DOG76 1
3 DOG24 1 1 1

As you can tell, I have several unsorted repeating strings in col1 of the input file. Each of these strings has an associated word in col2. Col2 has a total of 4 possible words (go, stop, yield, halt).

The output should have different columns. It should produce the number of times each string repeats in col1. In col2 the output should have the name of the string that is repeating (for ex: DOG5 repeated twice in the input so the output is now displaying the number "2" in col1 and the name "DOG5" in col2).

The other columns in the output should have a column for each of the words that the input col2 had (go, stop, yield, halt). For each string that is now only being shown once in col2, the output should mark how many "go" "stop" "yield" and "halt" each of the col2 strings had. For ex: DOG5 had a total of 2 "go" from the input. DOG5 is also blank for the remaining words because there were no DOG5's in the input that had stop or yield or halt. DOG24 had 1 stop 1 yield and 1 halt from the input and that is now being shown in different columns in the output.

So far, I have this.

Code:
 cat f1 | awk '{print $1}'|sort -k 1,1 | uniq -c | sort -k 1,1 -n

Quote:
1 col1
1 DOG76
2 DOG5
3 DOG24
Any ideas?

---------- Post updated at 11:58 AM ---------- Previous update was at 11:55 AM ----------

Apologizes for the formatting issue. The output below may be more readable.

Quote:
col1 col2 go stop yield halt
2 DOG5 2 0 0 0
1 DOG76 0 1 0 0
3 DOG24 0 1 1 1
# 2  
Old 03-28-2013
Code:
awk '
NR == 1 {
        print "col1","col2","go","stop","yield","halt"
} NR > 1 {
        $1 = $1
        F[$1]++
        if ( $2 ~ "go" )
                go[$1]++
        if ( $2 ~ "stop" )
                stop[$1]++
        if ( $2 ~ "yield" )
                yield[$1]++
        if ( $2 ~ "halt" )
                halt[$1]++
} END {
        for ( c in F )
                print F[c], c, (go[c]==""?0:go[c]), (stop[c]==""?0:stop[c]), (yield[c]==""?0:yield[c]), (halt[c]==""?0:halt[c])
} ' OFS='\t'  file

# 3  
Old 03-28-2013
Thank you. This works nicely. Can you explain what the For c in F section means?
# 4  
Old 03-28-2013
That is how we scan all elements of an Array

Check GNU AWK Scanning an Array for further reference.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to lowercase the values in a column in awk and include a dynamic counter?

Hi, I am trying to incorporate 2 functions into my `awk` command. I want to lower case Column 2 (which is essentially the same information in Col1, except in Col1 I want to maintain the capitalization) and I want to count from 0-N that begins and ends with the start of certain markers that I... (6 Replies)
Discussion started by: owwow14
6 Replies

2. UNIX for Dummies Questions & Answers

Column manipulation

Hi, I's like to ask for help with the following question: this is an example file: 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 I need to have it re-arranged in the following manner: first 2 columns, zero, zero, last two columns, columns 3 to... (6 Replies)
Discussion started by: zajtat
6 Replies

3. UNIX for Advanced & Expert Users

Column manipulation of a file

In a space delimited file how would you go about removing 2 and 3 column of a file and add "word" to the first column of the file in both awk and vi? (2 Replies)
Discussion started by: cokedude
2 Replies

4. Shell Programming and Scripting

Column manipulation

Hallo Team, I have a .csv file. I would like to have another column called Prefix. Under column Prefix i would like to have the first 3 numbers from the theird column. Other Licenced Operator Alpha Code Numbering Per minute Interconnect Fee per rate group (R Excluding VAT) ... (4 Replies)
Discussion started by: kekanap
4 Replies

5. Shell Programming and Scripting

Column manipulation

Hi, I have an input file in which I need to add a column with the value of part of another column. I.e. I have the following line (example, not the actual data): field1,field2,field3,test12,field5,field6I need to get the last two characters of the 4th field, and add them at the end as a... (5 Replies)
Discussion started by: Subbeh
5 Replies

6. UNIX for Dummies Questions & Answers

Adding a column to a text file based on mathematical manipulation

Hi, I have a tab delimited text file with three different columns. I want to add an extra column to the text file. The extra column will be the second column and it will equal third column - 1. How do I go about doing that? Thanks! Input: chr1 788822 rs11240777 chr1 1008567 rs9442372... (2 Replies)
Discussion started by: evelibertine
2 Replies

7. UNIX for Dummies Questions & Answers

[Solved] Column manipulation

Hi Everyone, I was wondering if someone could help me to transform my data into a format I need. Here is an example of what my data looks like E F G H A 1 2 3 4 B 5 6 7 8 C 9 1 2 3 D 4 5 6 7 and this is what I would need it to look like: AE 1 BE 5 CE 9 DE 4 AF 2 BF 6 CF 1 (6 Replies)
Discussion started by: zajtat
6 Replies

8. Shell Programming and Scripting

counter

Hi, I need some help. Shell script counter. i need to add condition to check if counter is more than 10 and longer than 3 hours? it runs every 5 mins. it only check count and send email right now. it runs in cron as below gregcount.ksh gregdb 10 > /tmp/gregcount.out 2> /tmp/gregcount.err ... (1 Reply)
Discussion started by: pega
1 Replies

9. Shell Programming and Scripting

Flat File column manipulation

Hi All, I have a tab delimited input file with say 4 fields (columns) as below : 0000443 1AGPR061 2006 Daiml 0002198 1B3XG0K2 1989 Chdds 0002199 1Bd64J0L 1990 Ch34s 0002275 1B3s4J0K 1989 Chadys 0002276 1B465302 2002 Dageml 0002290 1B45430K 1989 Cays I want the 2nd column in file to... (5 Replies)
Discussion started by: net
5 Replies

10. Shell Programming and Scripting

File Manipulation (Move Column Position)

Hi All, I have a comma separated value (.CSV) file like the one below. The file contains about 20000 lines. FileName EmpNo,Name,Age,Sex,Band,Spouse,Children,Salary, 1000,Arnold,24,M,B,N.A.,No,10000, 1001,Jenny,27,F,C,John,2,20000, ................................... What i need is to... (1 Reply)
Discussion started by: ultimate
1 Replies
Login or Register to Ask a Question