group by for lines


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting group by for lines
# 1  
Old 06-21-2005
group by for lines

I post my question here also because i have a difficult script to create that do the following:

The examples i got dont support it.
Lets say the input file is:
ABC^1^~^data^x
ABC^2^~^data^x
ABC^3^~^data^x
DEF^4^~^data^y
DEF^4^~^data^z
DEF^5^~^data^y
DEF^5^~^data^z
DEF^6^~^data^z
DEF^6^~^data^y
XYZ^7^~^data^t

Now I would like to group couple of fields(field 2 and field 5) and concatenate them with comma according to my key which is the first field.
My output should be:

ABC^1,2,3^~^data^x
DEF^4,5,6^~^data^y,z
XYZ^7^~^data^t

Notice that I would like to print all the line's fields.

Is it possible?
# 2  
Old 06-21-2005
My current Script:

This is the start but i have problem on duplicate data, does anyone have any idea regarding it

BEGIN { FS=OFS="^" }
{
arr[$1] = ($1 in arr) ? arr[$1] "," $2 : $2
arr2[$1] = ($1 in arr2) ? arr2[$1] "," $5 : $5
#In order not to repeat the value, how can i validate it
}
END {
if ( length(arr2) > length(arr1) ) {

}
if ( length(arr1) > length(arr2) ) {

}
if(length(arr1) == length(arr2) {

}

for (i in arr)
# I need to print all
}
# 3  
Old 06-22-2005
Very similar question to https://www.unix.com/shell-programming-and-scripting/19562-shell-script-comparing-files-file.html
You should be easily able to mod the python code presented there.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Group Multiple Lines on SINGLE line matching pattern

Hi Guys, I am trying to format my csv file. When I spool the file using sqlplus the single row output is wrapped on three lines. Somehow I managed to format that file and finally i am trying to make the multiple line on single line. The below command is working fine but I need to pass the... (3 Replies)
Discussion started by: RJSKR28
3 Replies

2. Shell Programming and Scripting

Summing columns over group of lines

I have an input file that looks like: ID1 V1 ID2 V2 P1 P2 P3 P4 ..... n no. of columns 1 1 1 1 1.0000 1.0000 1.0000 1.0000 1 1 1 2 0.9999 0.8888 0.7777 0.6666 1 2 1 1 0.8888 0.7777 0.6666 0.5555 1 2 1 2 0.7777 0.6666 0.5555 0.4444 2 1 1 1 0.6666 0.5555 0.4444 0.3333 2 1 1 2 0.5555 0.4444... (4 Replies)
Discussion started by: sdp
4 Replies

3. Shell Programming and Scripting

Deleting lines based on a condition for a group of files

hi i have a set of similar files. i want to delete lines until certain pattern appears in those files. for a single file the following command can be used but i want to do it for all the files at a time since the number is in thousands. awk '/PATTERN/{i++}i' file (6 Replies)
Discussion started by: anurupa777
6 Replies

4. Shell Programming and Scripting

Print lines containing same values in a group

Hi, I have a table like this: Name A1 A2 A3 B1 B2 B3 111 abc abc abc cbc cbc cbc 222 acv acv n_n bbc bbc bbc 333 bvv bvv bvv cBx ccx ccx 444 ttk ttk ttk kke kke kke 555 mcr mcr mcr mcr mcr mcr The 1st column is just names... (3 Replies)
Discussion started by: polsum
3 Replies

5. Shell Programming and Scripting

need to delete all lines from a group of files except the 1st 2 lines

Hello, I have a group of text files with many lines in each file. I need to delete all the lines in each and only leave 2 lines in each file. (3 Replies)
Discussion started by: script_op2a
3 Replies

6. UNIX for Advanced & Expert Users

Merge a group of lines into single line

Hi Everybody, Below are the contents of the a text file .., SN = 8 MSI = 405027002277133 IKVALUE = DE6AA6A11D42B69DF6398D44B17BC6F2 K4SNO = 2 CARDTYPE = SIM ALG = COMP128_3 SN = 8 MSI = 405027002546734 IKVALUE = 1D9F8BAA73973D8FBF8CBFB01436D822 K4SNO = 2 CARDTYPE = SIM ALG =... (8 Replies)
Discussion started by: prasanth_babu
8 Replies

7. AIX

Delete group of lines from file

I have a rather large file that needs lines deleted periodically. How may I accomplish this task? Should I run into an error message "not enough disk space" AIX 5.3 Thanks (2 Replies)
Discussion started by: tfort73
2 Replies

8. UNIX for Dummies Questions & Answers

sort group of n lines base on pattern in first line

I have record having 10 fields and each field being printed on a new line, first line cotains name of exchange, 2nd line stock name, third line stock price, etc etc... now i want to retrieve data only for a particular exchanged and that too only 2nd and 3rd row info... NSE RNRL 70 12 1... (1 Reply)
Discussion started by: manishma71
1 Replies

9. Shell Programming and Scripting

Get the total of a field in all the lines of a group

Hi I have a Fixed format data file where I need to to get the total of the field at certain position in a file for a group of lines. In this data file I need the total of all the field ats position 30:39 for each line starting with 6 and for each group startign with 5. Which means for... (27 Replies)
Discussion started by: appsguy616
27 Replies

10. UNIX for Dummies Questions & Answers

How to group lines into batches

Hi all I need to group all the lines that start with 2. For every 3 lines (that start with 2) i need to group them in one group and add 1 for each group. I need your help to batch the files content.The file look like this: 040171011140820070000000009650244002933170003000000075272... (2 Replies)
Discussion started by: ashikin_8119
2 Replies
Login or Register to Ask a Question