Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk to combine all matching dates and remove non-matching

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk to combine all matching dates and remove non-matching
# 1  
Old 11-12-2016
awk to combine all matching dates and remove non-matching
Using the awk below I am able to combine all the matching dates in $1, but I can not seem to remove the non-matching from the file. Thank you Smilie.

file
Code:
20161109104500.0+0000,x,5631
20161109104500.0+0000,y,2
20161109104500.0+0000,z,2
20161109104500.0+0000,a,4117
20161109104500.0+0000,b,6182
20161109104500.0+0000,c,1556
618
225
41
4
20161109104500.0+0000,k,2689
20161109104500.0+0000,l,6182
20161109104500.0+0000,d,3241
20161109104500.0+0000,e,2418
20161109104500.0+0000,f,26
20161109104500.0+0000,g,285166.843750
20161109104500.0+0000,h,67216

awk with current output
Code:
awk -F',' '{if(a!=$1) {a=$1; printf "\n%s%s",$0,FS} else {a=$1;$1="";printf $0 }} END {printf "\n" }' file

20161109104500.0+0000,x,5631, y 2 z 2 a 4117 b 6182 c 1556
618,
225,
41,
4,
20161109104500.0+0000,k,2689, l 6182 d 3241 e 2418 f 26 g 285166.843750 h 67216

desired output
Code:
20161109104500.0+0000,x,5631, y 2 z 2 a 4117 b 6182 c 1556
20161109104500.0+0000,k,2689, l 6182 d 3241 e 2418 f 26 g 285166.843750 h 67216

# 2  
Old 11-12-2016
Hello cmccabe,

If your Input_file is exactly the same as sample shown into your post then following may help you in same.
Code:
awk -F, '{if(A!=$1 && A && VAL ~ /,/){print VAL;VAL=""};VAL=A==$1?VAL OFS $2 OFS $3:$0;A=$1} END{print VAL}'   Input_file

Output will be as follows.
Code:
20161109104500.0+0000,x,5631 y 2 z 2 a 4117 b 6182 c 1556
20161109104500.0+0000,k,2689 l 6182 d 3241 e 2418 f 26 g 285166.843750 h 67216

NOTE: Above code considers that the 1st fields we have to match are on the basis of there next line not the whole Input_file. Input_file's non-matching lines will not have comma(,) in them.

If you have mor requirements in this, request you to be more clear into your post so that could try to help you more on same. I hope this helps.

Thanks,
R. Singh
Last edited by RavinderSingh13; 11-12-2016 at 02:01 PM..
This User Gave Thanks to RavinderSingh13 For This Post:
cmccabe
# 3  
Old 11-13-2016
Shouldn't the output be:
Code:
20161109104500.0+0000,x,5631,y,2,z,2,a,4117,b,6182,c,1556,k,2689,l,6182,d,3241,e,2418,f,26,g,285166.843750,h,67216

Code:
awk 'NF==1{next} p!=$1{if(s) print s; p=$1; s=$0; next} {$1=x; s=s $0} END{if(s) print s}' FS=, OFS=, file

This User Gave Thanks to Scrutinizer For This Post:
cmccabe
# 4  
Old 11-16-2016
Thank you both Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove lines with Pattren Matching

Hi , I need to remove the lines that matches the pattern TABLEEXCLUDE *.AQ$_*_F ; * is wildcard, it can be any word. For example, I have following file: TABLEEXCLUDE THOT.AQ$_PT_ADDR_CLEANUP_QTAB2_F ; TABLEEXCLUDE THOT.AQ$_MICRO_SERVICE_QT_F ; TEST TABLEEXCLUDE... (1 Reply)
Discussion started by: rcc50886
1 Replies

2. Shell Programming and Scripting

awk to combine all matching fields in input but only print line with largest value in specific field

In the below I am trying to use awk to match all the $13 values in input, which is tab-delimited, that are in $1 of gene which is just a single column of text. However only the line with the greatest $9 value in input needs to be printed. So in the example below all the MECP2 and LTBP1... (0 Replies)
Discussion started by: cmccabe
0 Replies

3. Shell Programming and Scripting

Remove lines matching a substring in a specific column

Dear group, I have following input text file: Brit 2016 11 18 12 00 10 1.485,00 EUR Brit 2016 11 18 12 00 10 142,64 EUR Brit 2016 11 18 12 00 10 19,80 EUR Brit 2016 11 18 12 00 10 545,00 EUR Brit 2016 11 18 12 00 10 6.450,00 EUR... (3 Replies)
Discussion started by: gfhsd
3 Replies

4. Shell Programming and Scripting

awk to combine matching lines in file

I am trying to combine all matching lines in the tab-delimited using awk. The below runs but no output results. Thank you :). input chrX 110925349 110925532 ALG13 chrX 110925349 110925532 ALG13 chrX 110925349 110925532 ALG13 chrX 47433390 47433999 SYN1... (3 Replies)
Discussion started by: cmccabe
3 Replies

5. Linux

Remove matching files from a list

hi, i have a cache file with below file list more gtda_11.cache GTDA_Dly_Pmix_GB_6_20130624.20130624070610.psv GTDA_Dly_Pmix_CH_006_20130624.20130624140018.psv GTDA_Dly_Pmix_GB_6_20130624.20130624070620.psv GTDA_Dly_Pmix_BE_6_20130624.20130624070620.psv... (2 Replies)
Discussion started by: renuk
2 Replies

6. Shell Programming and Scripting

Compare file1 for matching line in file2 and print the difference in matching lines

Hello, I have two files file 1 and file 2 each having result of a query on certain database tables and need to compare for Col1 in file1 with Col3 in file2, compare Col2 with Col4 and output the value of Col1 from File1 which is a) not present in Col3 of File2 b) value of Col2 is different from... (2 Replies)
Discussion started by: RasB15
2 Replies

7. Shell Programming and Scripting

Insert lines above matching line with content from matching

Hi, I have text file: Name: xyz Gender: M Address: "120_B_C; ksilskdj; lsudlfw" Zip: 20392 Name: KLM Gender: F Address: "65_D_F; wnmlsi;lsuod;,...." Zip:90233I want to insert 2 new lines before the 'Address: ' line deriving value from this Address line value The Address value in quotes... (1 Reply)
Discussion started by: ysrini
1 Replies

8. Shell Programming and Scripting

awk: matching and not matching

Hello all, simple matching and if not match problem that i can't figure out. file1 hostname: 30 10 * * * /home/toto/start PROD instance_name1 -p 00 9 * * * /home/toto/start PROD instance_name2 -p 15 8 * * * /home/toto/start PROD instance_name3 -p hostname2: 00 8 * * *... (5 Replies)
Discussion started by: maverick72
5 Replies

9. Shell Programming and Scripting

How to remove all words from a matching word in a line?

Hi Guys, :p I have a file like this: 2010-04-25 00:00:30,095 INFO - ]- start process U100M4 2010-04-25 00:00:30,096 DEBUG - ] -- call EJB 2010-04-25 00:00:30,709 INFO - - end processU100M4 2010-04-25 00:00:30,710 DEBUG - got message=Sorry I want to out put format. 2010-04-25... (5 Replies)
Discussion started by: ooilinlove
5 Replies

10. Shell Programming and Scripting

Remove matching lines with list of strings

Hi, HP-UX gxxxxxxxc B.11.23 U ia64 3717505098 unlimited-user license I have a file with below pipe separated field values: xxx|xxx|abcd|xxx|xxx|xx xxx|xxx|abcd#123|xxx|xxx|xx xxx|xxx|abcd#345|xxx|xxx|xx xxx|xxx|pqrs|xxx|xxx|xx xxx|xxx|pqrs#123|xxx|xxx|xx The third field has values like... (6 Replies)
Discussion started by: Nanu_Manju
6 Replies
Login or Register to Ask a Question