Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Implementing Listagg like function in shell

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Implementing Listagg like function in shell
# 8  
Old 06-28-2016
This is how far I can get right now:

Code:
awk '
NR == 1         {KN = split (KEYCOL, KC, ",")
                 AN = split (AGGCOL, AC, ",")
                }
                {KEY = ""
                 for (i=1; i<=KN; i++)  {KEY = KEY $KC[i] OFS
                                         $KC[i] = ""
                                        }
                 for (i=1; i<=AN; i++)  {F[KEY,i] = F[KEY,i] DL[KEY,i] $AC[i]
                                         DL[KEY,i] = ","
                                         $AC[i] = "\001" i
                                        }
                 LINE[KEY] = $0
                }
END             {for (l in LINE)        {for (i=1; i<=AN; i++)  sub ("\001" i, F[l,i], LINE[l])
                                         print l, LINE[l]
                                        }
                }
' OFS="\t" KEYCOL="1" AGGCOL="2,4" file
B00000			1	...	abc
B00001			1,2,2	...	abc,def,ghi,jkl
B00002			1,2	...	abc,def
B00003			1	...	xyz

Result with KEYCOL="1,2" AGGCOL="4" :
Code:
B00002	2				...	def
B00003	1				...	xyz
B00000	1				...	abc
B00001	1				...	abc,def
B00001	2				...	ghi,jkl
B00002	1				...	abc

This User Gave Thanks to RudiC For This Post:
prohank
# 9  
Old 06-28-2016
Prohank,

Suppose file contains 5 columns, key fields are 1, 2 and agg field is 5.
What about field no 3 and 4. Do we suppose to agg or ignore ?

Thanks
Pravin
# 10  
Old 06-28-2016
This is brilliant. Thanks @RudiC. This has reduced a lot of manual work.

@pravin27 I think for the current situation its ok for me to mention all the columns as either a key or an agg field as there are not too many columns.

It also avoids confusion if others use it.
# 11  
Old 06-28-2016
Next version: duplicate field contents removed; multiple OFS reduced to single:
Code:
awk '
NR == 1         {KN = split (KEYCOL, KC, ",")
                 AN = split (AGGCOL, AC, ",")
                }
                {KEY = ""
                 for (i=1; i<=KN; i++)  {KEY = KEY $KC[i] OFS
                                         $KC[i] = ""
                                        }
                 for (i=1; i<=AN; i++)  {if (F[KEY,i] !~ "(^|,)" $AC[i] "(,|$)")        {F[KEY,i] = F[KEY,i] DL[KEY,i] $AC[i]
                                                                                         DL[KEY,i] = ","
                                                                                        }
                                         $AC[i] = "\001" i
                                        }
                 LINE[KEY] = $0
                }
END             {for (l in LINE)        {for (i=1; i<=AN; i++)  sub ("\001" i, F[l,i], LINE[l])
                                         OP = sprintf ("%s%s%s", l, OFS, LINE[l])
                                         gsub (OFS OFS "*", OFS, OP)
                                         print OP
                                        }
                }
' OFS="\t" KEYCOL="1" AGGCOL="2,4" file
B00000	1	...	abc
B00001	1,2,3	...	abc,def,ghi,jkl,ybc,xab
B00002	1,2	...	abc,def
B00003	1	...	xyz

(tested with an extended input file)
This User Gave Thanks to RudiC For This Post:
prohank
# 12  
Old 06-30-2016
Thanks RudiC,

Your solution works great with tab as delimiter but I am using | and it is failing with the below error:

Code:
awk: 0602-521 There is a regular expression error.
        *?+ not preceded by valid expression
 The input line number is 24. The file is /tmp/abc.del.
 The source line number is 1.

# 13  
Old 07-01-2016
The script I posted has 23 lines, so it is difficult to track an error in line 24 without seeing a) the script you ran b) the input file (or a representative extraction of it).

You could run it with <TAB>s and then tr '\t' '|' the result.
# 14  
Old 07-01-2016
To clarify a little bit, 24 is the last line of my input file and source line is 1 because I ran your script by eliminating all the new lines and replacing them with semi colons.

When I run it the you have given the error is on line 18 which is:
Code:
gsub (OFS OFS "*", OFS, OP)

Actually I changed the input file to test the code, it comes with | by default.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

DB2 Query modification to remove duplicate values using LISTAGG function

I am using DB2 v9 and trying to get country values in comma seperated format using below query SELECT distinct LISTAGG(COUNTRIES, ',') WITHIN GROUP(ORDER BY EMPLOYEE) FROM LOCATION ; Output Achieved MEXICO,UNITED STATES,INDIA,JAPAN,UNITED KINGDOM,MEXICO,UNITED STATES The table... (4 Replies)
Discussion started by: Perlbaby
4 Replies

2. Shell Programming and Scripting

Implementing linked list in shell scripting

Hello Experts, Is it possible to implement linked list in shell scripting? is yes then how can we do it? Any working example is highly appreciated. Thanks in advance. (4 Replies)
Discussion started by: mukulverma2408
4 Replies

3. Shell Programming and Scripting

Need help implementing a timout in my Shell Script for RHEL6

Hey Guys, My problem: I have a script that will be querying the database every minute to see if it gets a response, the response its querying for is "UP" in a table i made called dbup in the database. Now, I am trying to add the component to implement a timeout if the script does not get a... (2 Replies)
Discussion started by: mo_VERTICASQL
2 Replies

4. Programming

Implementing function outside struct

I have this code where I have declared a struct with some functions. Trying to write the function implementation outside the struct declaration and do not know how to proceed. #ifndef ParseEl_hh #define ParseEl_hh #include <iostream> #include <fstream> #include "DynBaseObj.hh"... (7 Replies)
Discussion started by: kristinu
7 Replies

5. Programming

Problem with implementing the times() function in C (struct tms times return zero/negative values)

Hello, i'm trying to implement the times() function and i'm programming in C. I'm using the "struct tms" structure which consists of the fields: The tms_utime structure member is the CPU time charged for the execution of user instructions of the calling process. The tms_stime structure... (1 Reply)
Discussion started by: g_p
1 Replies

6. Shell Programming and Scripting

Implementing Queue Using Shell scripts

HI I want to implement a control mechanism using Shell scripts .The intention is to have controlled number of jobs running in parallel External process will kickstart 40 jobs in parallel .All the 40 jobs will call the same generic script with different parameter values .But at a... (4 Replies)
Discussion started by: police
4 Replies

7. Homework & Coursework Questions

implementing mkdir, chdir, mv, pwd inside a shell !

1. The problem statement, all variables and given/known data: need to implement mkdir, chdir, mv, pwd given a shell.cpp directory.cpp and some other files this shell missing these commands, and i need to implement them inside the shell 2. Relevant commands, code, scripts,... (0 Replies)
Discussion started by: evantheking
0 Replies

8. Programming

Implementing a shell in C

Hi, I am implementing a shell in C, with the following problem... Suppose the shell is invoked from the command line as >> myshell < test.in > test.out 2>&1 I have to execute the commands in test.in and redirect them to test.out How does one detect in the main function that the shell... (1 Reply)
Discussion started by: jacques83
1 Replies

9. Programming

need help in implementing simple interactive shell in C

hello all, i hv attached herewith my program to implement a simple interactive shell in C. no matter hw hard I try, I keep getting some errors. i need help - urgently !! proj1test7.c: In function `parseCommand': proj1test7.c:102: warning: assignment makes pointer from integer without a cast... (2 Replies)
Discussion started by: nix1209
2 Replies

10. Programming

Implementing a shell

I'm implementing a shell in C that supports piping, output redirection, and background processing, and a few other commands. I was wondering how I'd go about implementing the output redirection. So, I'd open a file and I'd fork and execute the command. But how would I get stdout into the file? Any... (10 Replies)
Discussion started by: ununium
10 Replies
Login or Register to Ask a Question