Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Delete row if column matches

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Delete row if column matches
# 1  
Old 01-26-2015
Delete row if column matches
Hi, I have a long file in the format below. I want to delete the consecutive lines that contain the same value in column 1.
I have tried awk '!x[$1]++' FS="," filename
This has not worked.

Code:
14,MM709_BHP_DM,BHP,BHP_MC709_DM	
19,OFFLINE,CHE,CHEV_MC773_DM	 
20,M33,BP,BP_MIM775_NS_DM	
20,M84,BP,BP_MIM775_NS_DM
20,MM709_BHP_DM,BHP,BHP_MC709_DM	
24,OFFLINE,BP,BP_MC775_EW_DM

I want my output to be:

Code:
14,MM709_BHP_DM,BHP,BHP_MC709_DM	
19,OFFLINE,CHE,CHEV_MC773_DM	 
20,M33,BP,BP_MIM775_NS_DM	
24,OFFLINE,BP,BP_MC775_EW_DM

Last edited by Don Cragun; 01-26-2015 at 06:38 PM.. Reason: Add CODE and ICODE tags.
# 2  
Old 01-26-2015
Try

Code:
$ cat tmp
14,MM709_BHP_DM,BHP,BHP_MC709_DM
19,OFFLINE,CHE,CHEV_MC773_DM
20,M33,BP,BP_MIM775_NS_DM
20,M84,BP,BP_MIM775_NS_DM
20,MM709_BHP_DM,BHP,BHP_MC709_DM
24,OFFLINE,BP,BP_MC775_EW_DM


$ awk -F, '$1!=b{b=$1;print}' tmp
14,MM709_BHP_DM,BHP,BHP_MC709_DM
19,OFFLINE,CHE,CHEV_MC773_DM
20,M33,BP,BP_MIM775_NS_DM
24,OFFLINE,BP,BP_MC775_EW_DM

# 3  
Old 01-26-2015
For your sample input, your script seems to work just fine. Does it not work for you with that sample input?

What operating system and shell are you using?

To produce output for different input still meeting the output you say you want, you might try:
Code:
awk -F, 'x!=$1;{x=$1}' filename

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

If pattern in column 3 matches pattern in column 2 (any row), print value in column 1

Hi all, I have searched and searched, but I have not found a solution that quite fits what I am trying to do. I have a long list of data in three columns. Below is a sample: 1,10,8 2,12,10 3,13,12 4,14,14 5,15,16 6,16,18 Please use code tags What I need to do is as follows: If a... (4 Replies)
Discussion started by: bleedingturnip
4 Replies

2. Shell Programming and Scripting

Bash - delete from csv all the row if the first column is length >

Hi guys, i have a csv file like: USERID;COG;DESCR;FIL;OFF user001;user;test1;001;A01 user002;user;test2;002;A02 user0003;user;test3;003;A03 user004;user;test4;004;A04 user0005;user;test5;005;A05 etc.. I need to read line for line and, if value of first column is > 7 char (in this example... (4 Replies)
Discussion started by: kamose
4 Replies

3. Shell Programming and Scripting

Delete a row if either of column value is zero

Hi, My input file is this way 1.1 0.0 2.4 3.5 7.9 1.8 22.3 4.7 8.9 0.9 1.3 0.0 3.4 5.6 0.0 1.1 2.2 0.0 0.0 1.1 0.0 0.0 3.4 5.6 I would like to delete the entire row, if either of 2nd and 3rd columns are 0.0. Please note that my values are all decimal values. So, my output would... (4 Replies)
Discussion started by: jacobs.smith
4 Replies

4. Shell Programming and Scripting

How to merge multiple rows into single row if first column matches ?

Hi, Can anyone suggest quick way to get desired output? Sample input file content: A 12 9 A -0.3 2.3 B 1.0 -4 C 34 1000 C -111 900 C 99 0.09 Output required: A 12 9 -0.3 2.3 B 1.0 -4 C 34 1000 -111 900 99 0.09 Thanks (3 Replies)
Discussion started by: cbm_000
3 Replies

5. Shell Programming and Scripting

delete a row with a specific value at a certain column

Hi, I want to delete rows that have 0 at column 4. The file looks like this: chr01 13 61 2 chr01 65 153 0 chr01 157 309 1 chr01 313 309 0 chr01 317 469 1 chr01 473 557 0 I want to delete all rows with a 0 at column 4 chr01 13 61 2 chr01 157 309 1 chr01 ... (3 Replies)
Discussion started by: kylle345
3 Replies

6. Shell Programming and Scripting

Delete row if a a particular column has more then three characters in it

Hi i have a data like hw:dsfnsmdf:39843 chr2 76219829 51M atatata 51 872389 hw:dsfnsmdf:39853 chr2 76219839 51M65T atatata 51 872389 hw:dsfnsmdf:39863 chr2 76219849 51M atatata 51 872389 hw:dsfnsmdf:39873 chr2 ... (3 Replies)
Discussion started by: bhargavpbk88
3 Replies

7. Shell Programming and Scripting

search column and delete row if greater than value

Hi, as the title states i need to find a way to search a column for values great than 1000, and if it is, then delete that row. An example 1 7.021 6.967 116.019 4 U 6.980E+07 0.000E+00 e 0 0 0 0 2 8.292 7.908 118.063 3 U 1.440E+07 0.000E+00 e 0 821 814 ... (3 Replies)
Discussion started by: olifu02
3 Replies

8. UNIX for Dummies Questions & Answers

How do you delete cells from a space delimited text file given row and column number?

How do you delete cells from a space delimited text file given row and column number? Letś say the row number is r and the column number is c. Thanks! (5 Replies)
Discussion started by: evelibertine
5 Replies

9. Shell Programming and Scripting

Delete first row last column

Hi All, I am having following file and I want to delete 1 row last column. Current File Content: ================ procedure test421 put_line procedure test321 test421 procedure test521 test321 procedure test621 test521 Expected File Content: =========================== procedure... (3 Replies)
Discussion started by: susau_79
3 Replies

10. Shell Programming and Scripting

Delete a row that has a duplicate column

I'm trying to remove lines of data that contain duplicate data in a specific column. For example. apple 12345 apple 54321 apple 14234 orange 55656 orange 88989 orange 99898 I only want to see apple 12345 orange 55656 How would i go about doing this? (5 Replies)
Discussion started by: spartan22
5 Replies
Login or Register to Ask a Question