Concatenate last field values for all occurences


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Concatenate last field values for all occurences
# 1  
Old 04-10-2013
Concatenate last field values for all occurences

Hello all,

Maybe you can help me with an awk script to get what I need.

I have the input file with format below:

Code:
[2013-04-02 00:00:00.014] REQUEST|79023787741690|738227864597|985
[2013-04-02 00:00:00.215] REQUEST|79024002151717|738229423534|985
[2013-04-02 00:00:00.215] REQUEST|79024002151717|738229423534|*985
[2013-04-02 00:00:00.215] NDS-REQUEST|79024002151717|738229423534
[2013-04-02 00:00:00.215] REQUEST|79023364221647|738227504615|985
[2013-04-02 00:00:00.297] EXT-RESPONSE|79023998911716|738225664496|*702|Active|0|233
[2013-04-02 00:00:00.299] RESPONSE|79023998911716|738225664496|*702|233
[2013-04-02 00:00:00.481] NDS-RESPONSE|79024002151717|738229423534|1|AMB
[2013-04-02 00:00:00.482] RESPONSE|79024002151717|738229423534|*985|233

- Fields are separated with "|" and values in field 2 may have more than one occurence.
- I want to print only one time each value that is in field 2, where field 1 contains exactly "REQUEST" or "RESPONSE/EXT-RESPONSE",
and concatenate all the values in last fields in each ocurrence.

For example,
- 79023787741690 only appears once, then only has one "last field" that is 985
- 79024002151717 appears 5 times, but only 4 times, the line where it's present contains exactly "REQUEST" or "RESPONSE/EXT-RESPONSE".
and the last field (in order of appearence) for each occurence are 985, *985, AMB and 233

So the output for this input would be:

Code:
79023787741690|985
79024002151717|985|*985|AMB|233
79023364221647|985
79023998911716|233|233

I hope make ssense. Thanks in advance.
# 2  
Old 04-10-2013
Try:
Code:
awk -F'[ \t]*|\\|' '$3~/^REQUEST$|RESPONSE/ {A[$4]=A[$4] OFS $NF} END{for(i in A) print i A[i]}' OFS=\| file

This User Gave Thanks to Scrutinizer For This Post:
# 3  
Old 04-10-2013
Hello Scrutinizer,

Thank you for your help. It works just fine.

May you explain how is made the regexp for field separator. I understand that
this [ \t] means space or tab, but the next part, why not only one "|"?

Thanks again
# 4  
Old 04-10-2013
Hi Ophiuchus, the pipe symbol is a special character (alternation operator) that needs to be escaped if intended literally. Behind the -F is a string that contains a regex, which requires the escape character ( \ ) to be escaped itself, hence the double \\
# 5  
Old 04-11-2013
Hello Scrutinizer,

Many thatnks for your help. You've helped me a lot.

Regards.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Concatenate column values when header is Matching from multiple files

there can be n number of columns but the number of columns and header name will remain same in all 3 files. Files are tab Delimited. a.txt Name 9/1 9/2 X 1 7 y 2 8 z 3 9 a 4 10 b 5 11 c 6 12 b.xt Name 9/1 9/2 X 13 19 y 14 20 z 15 21 a 16 22 b 17 23 c 18 24 c.txt Name 9/1 9/2... (14 Replies)
Discussion started by: Nina2910
14 Replies

2. Shell Programming and Scripting

Display combination of 4 field uniqe record and along with concatenate 5th and 6th field.

Table ACN|NAME|CITY|CTY|NO1|NO2 115|AKKK|ASH|IND|10|15 115|AKKK|ASH|IND|20|20 115|AKKK|ASH|IND|30|35 115|AKKK|ASH|IND|30|35 112|ABC|FL|USA|15|15 112|ABC|FL|USA|25|20 112|ABC|FL|USA|25|45 i have written shell script using cut command and awk programming getting error correct it and add... (5 Replies)
Discussion started by: udhal
5 Replies

3. Shell Programming and Scripting

Quoting the values in second field

Hi, I have got a file comp_data containing the below data : 38232836|9302392|49 39203827|8203203,3933203|52 72832788|567,3245,2434324|100 This file can have many rows like shown above. I want the values separated by "," in second column(taking "|" as delimiter) to be in quotes. These... (2 Replies)
Discussion started by: msabhi
2 Replies

4. Shell Programming and Scripting

concatenate consecutive field values

Hi, I have a file like this A Bob A Sam A John B David C Paul C Sandra If the consecutive field values in column one is same, then concatenate the corresponding strings. So, I need an output like this, A Bob_Sam_John B David C Paul_Sandra I usually work with excel but... (3 Replies)
Discussion started by: polsum
3 Replies

5. Shell Programming and Scripting

adding field values if field matches

hi i have file as below , i want to add duplicate records like bell_bb to one record with valuve as 15 ( addition of both ) any oneline awk script to achive this ? header 0 CAMPAIGN_NAME 1 Bell_BB 14 Bell_MONTHLY 803 SOLO_UNBEATABLE 644 Bell_BB 1 Bell_MONTHLY 25 SOLO_UNBEATABLE... (4 Replies)
Discussion started by: raghavendra.cse
4 Replies

6. Shell Programming and Scripting

Count number of occurences of a character in a field defined by the character in another field

Hello, I have a text file with n lines in the following format (9 column fields): Example: contig00012 149606 G C 49 68 60 18 c$cccccacccccccccc^c I need to count the number of lower-case and upper-case occurences in column 9, respectively, of the... (3 Replies)
Discussion started by: s052866
3 Replies

7. Shell Programming and Scripting

Subtract field values

I've got a long logfile of the form network1:123:45:6789:01:234:56 network2:12:34:556:778:900:12 network3:... I've got a similar logfile from a week later with different values for each of the fields eg network1:130:50:6800:10:334:66 network2:18:40:600:800:999:20 network3:... ... (5 Replies)
Discussion started by: Yorkie99
5 Replies

8. Shell Programming and Scripting

Get 4 character each from 2 different fields concatenate and add as a new field

Hi, I have a huge text file. It looks like abcde bangalo country 12345 lastfield i want to get first 3 characters from field1 and first 3 characters from field 2 and insert the result as a new field. example the result should be: abcde bangalo abcban country 12345 lastfield Please... (4 Replies)
Discussion started by: ajithshankar@ho
4 Replies

9. Shell Programming and Scripting

how to concatenate values of two variables with an underscore(_) in between

Hi, I'm new to shell programming. I have two variables a and b a=val1 b=val2 could anyone kindly post the shell script to concatenate the values of variable a and b with an underscore(_) in between? The final output should be val1_val2. (8 Replies)
Discussion started by: badrimohanty
8 Replies

10. Shell Programming and Scripting

Find top N values for field X based on field Y's value

I want to find the top N entries for a certain field based on the values of another field. For example if N=3, we want the 3 best values for each entry: Entry1 ||| 100 Entry1 ||| 95 Entry1 ||| 30 Entry1 ||| 80 Entry1 ||| 50 Entry2 ||| 40 Entry2 ||| 20 Entry2 ||| 10 Entry2 ||| 50... (1 Reply)
Discussion started by: FrancoisCN
1 Replies
Login or Register to Ask a Question