Group by and string concatenation


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Group by and string concatenation
# 1  
Old 04-08-2013
Group by and string concatenation

Hi,

I was trying to work on a file which had the following data format
Code:
1  hi 
1  this
1  is
1  john
2  hello
3  test
3  case

the expected output file is the below
Code:
     1  hi, this, is, john 
     2  hello 
     3  test, case

I tried using awk or while read, but I couldnt go much far, Can anybody help me in writing a command ?
# 2  
Old 04-08-2013
Code:
awk '{A[$1]=(A[$1]==""?$2:A[$1]","$2)}END{for(i in A) print i, A[i]}' file

This User Gave Thanks to Yoda For This Post:
# 3  
Old 04-08-2013
Thanks Yoda, that works great, can you also explain the command, so that i know what to do in future.

Thanks,
Karthik
# 4  
Old 04-08-2013
Code:
awk '
{
        # Check if array indexed by 1st field (A[$1]) is null.
        # If yes, assign value = 2nd field ($2)
        # If no, assign value = previous value (A[$1]) comma (",") 2nd field ($2)
        A[$1] = A[$1] == "" ? $2 : A[$1] "," $2
}
        # END Block
END {
        # For each element in array
        # Print
        for (i in A) {
                print i, A[i]
        }
} ' file

This User Gave Thanks to Yoda For This Post:
# 5  
Old 04-09-2013
Hi Yoda,

Thanks for the explanation, This does solve my purpose, but to understand the script a little better, I was trying to modify the script to include a third field, instead of just 2, and I wasnt able to get it running, can you please let me know where I could add a new field.

Thanks,
Karthik
# 6  
Old 04-09-2013
Can you post sample input and desired output?
# 7  
Old 04-09-2013
You mean like this sort of input file?
Code:
[root@Imperfecto_1 test]# cat infile
1  hi a
1  this b
1  is
1  john
2  hello c d
3  test r
3  case


Try this - for any # of columns
Code:
awk '{for(i=2;i<=NF;i++){A[$1]=(A[$1]?A[$1]","$i:$i)}}END{for(i in A){print i, A[i]}}' infile

--ahamed
This User Gave Thanks to ahamed101 For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with String concatenation

I have a script which is migrated from AIX to Linux & now while running it is no able to concatenate string values The string concatenation step under while loop is not displaying desired result Please find below the piece of code: while read EXT_FILE ; do EXT_FILE=$EXT_FILE.ext.sent echo... (7 Replies)
Discussion started by: PreetArul
7 Replies

2. Shell Programming and Scripting

String concatenation problem

Hi there, I'm writing a basic script where I want to make a string of 2 numeric fields from a file, which I have done, but the behavior is rather confusing. I have a file of random values such as: 1 2 3 4 5 6 7 8 9 10 and my awk code is: BEGIN { FS = " " } { str = str $1 $2 } END {... (7 Replies)
Discussion started by: HMChadwick
7 Replies

3. Shell Programming and Scripting

String concatenation

Hi, I have two files. cat file.txt a b c d cat file1.txt j k l m I need the output as a:j (12 Replies)
Discussion started by: nareshkumar522
12 Replies

4. Shell Programming and Scripting

String / Variable Concatenation

Hi all, I'm trying to build a variable name automatically through a for loop for a script I'm working on, basically I want to build the variables named: $JVM_HOME0 or $JVM_HOME1 so that I can loop through some file copy/deletes and a server restart once completed. With the code below, I get this... (3 Replies)
Discussion started by: hydroponx
3 Replies

5. Shell Programming and Scripting

String concatenation problems

#! /bin/csh set tt=12345_UMR_BH452_3_2.txt set rr=`echo $tt | cut -d_ -f1` set rr1=welcome set ff=$rr $rr1 echo $ff why $ff returned only 12345 and not 12345welcome? thanks (2 Replies)
Discussion started by: jdsignature88
2 Replies

6. Shell Programming and Scripting

String/Variable Concatenation

Hello, Trying to concatenate the following using bourne shell: # !/bin/bash # this works in bash shell e.g. get the results I am expecting fnTmp=C$cindex.$station_0.$station_1.$station_3.$ts.tmp # # under !/bin/sh # the results are not the same Any assistance would be... (8 Replies)
Discussion started by: LAVco
8 Replies

7. Shell Programming and Scripting

String Concatenation

Hi All, I need to concatenate the values in the array into a variable. Currently the code is : for (( i=1 ; i <= $minCount ; i++ )) do var="${var}""${sample_file}" done The output is : /tmp/1/tmp/2/tmp/3/tmp/4/tmp/5/tmp/6/tmp/7/tmp/8/tmp/9/tmp/10 I need a space between... (1 Reply)
Discussion started by: sh_kk
1 Replies

8. Shell Programming and Scripting

String concatenation with spaces

Hi, I have a variable $ID=40 and I need to build a string like 40 40 40 40 40 40 so repeating ID 'n' times separated by spaces. Any help? Thanks Sarah (2 Replies)
Discussion started by: f_o_555
2 Replies

9. Shell Programming and Scripting

Help concatenation string and variable

Hello, in my script i have this lines of code in a while cycle: .. let j=i+1 t_prod_$i = `cat myfile.csv | grep world | cut -d ";" -f$j` let i+=1 ... So if i try an echo $t_prod_$i at the end of the cycle i cannot see the right value obtained by `cat myfile.csv | grep world |... (5 Replies)
Discussion started by: drain
5 Replies

10. UNIX for Dummies Questions & Answers

string concatenation

my input file contains thousands of lines like below 234A dept of education 9788 dept of commerce 8677 dept of engineering How do i add a delimeter ':' after FIRST 4 CHARACTERS in a line 234A:dept of education 9788:dept of commerce 8677:dept of engineering (7 Replies)
Discussion started by: systemsb
7 Replies
Login or Register to Ask a Question