Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

deleting rows under a certain condition

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting deleting rows under a certain condition
# 1  
Old 06-06-2012
deleting rows under a certain condition
there are 20 variables and I would like to delete the rows if 13th-20th columns are all NA.

Thank you!


Code:
FID IID aspirpre statihos fibrahos ocholhos arbhos betabhos alphbhos cacbhos diurehos numbcig.x toast1 toast2 toast3 toast4 ischoth1 ischoth2 ischoth3 ischoth4
101 101 1 1 1 1 1 2 1 2 1 15 1 0 0 0 0 0 0 0
102 102 2 0 0 0 0 0 0 0 0 NA NA NA NA NA NA NA NA NA
103 103 2 2 1 1 1 2 1 1 1 NA 0 0 0 0 0 0 0 0
104 104 1 0 0 0 0 0 0 0 0 15 NA NA NA NA NA NA NA NA
105 105 1 2 1 1 1 1 1 1 1 NA 0 0 1 0 0 0 0 0
106 106 2 1 1 1 1 1 1 1 1 12 NA NA NA NA NA NA NA NA
107 107 1 2 1 1 1 1 1 1 1 15 0 0 1 0 0 0 0 0
108 108 1 1 1 1 2 1 1 2 2 NA 0 0 1 0 0 0 0 0
109 109 2 2 1 1 2 1 1 1 2 25 0 0 0 1 0 0 0 0
110 110 1 0 0 0 0 0 0 0 0 5 NA NA NA NA NA NA NA NA

Last edited by pludi; 06-06-2012 at 06:06 PM..
# 2  
Old 06-06-2012
Code:
awk '{ P=0 ; for(N=13; (!P)&&(N<=20); N++) if($N != "NA") P=1 } P' filename

This User Gave Thanks to Corona688 For This Post:
johnkim0806
# 3  
Old 06-06-2012
This is an example using 'sed':
Code:
$ cat test.txt
FID IID aspirpre statihos fibrahos ocholhos arbhos betabhos alphbhos cacbhos diurehos numbcig.x toast1 toast2 toast3 toast4 ischoth1 ischoth2 ischoth3 ischoth4
101 101 1 1 1 1 1 2 1 2 1 15 1 0 0 0 0 0 0 0
102 102 2 0 0 0 0 0 0 0 0 NA NA NA NA NA NA NA NA NA
103 103 2 2 1 1 1 2 1 1 1 NA 0 0 0 0 0 0 0 0
104 104 1 0 0 0 0 0 0 0 0 15 NA NA NA NA NA NA NA NA
105 105 1 2 1 1 1 1 1 1 1 NA 0 0 1 0 0 0 0 0
106 106 2 1 1 1 1 1 1 1 1 12 NA NA NA NA NA NA NA NA
107 107 1 2 1 1 1 1 1 1 1 15 0 0 1 0 0 0 0 0
108 108 1 1 1 1 2 1 1 2 2 NA 0 0 1 0 0 0 0 0
109 109 2 2 1 1 2 1 1 1 2 25 0 0 0 1 0 0 0 0
110 110 1 0 0 0 0 0 0 0 0 5 NA NA NA NA NA NA NA NA


$ sed '/NA NA NA NA NA NA NA NA$/d' test.txt
FID IID aspirpre statihos fibrahos ocholhos arbhos betabhos alphbhos cacbhos diurehos numbcig.x toast1 toast2 toast3 toast4 ischoth1 ischoth2 ischoth3 ischoth4
101 101 1 1 1 1 1 2 1 2 1 15 1 0 0 0 0 0 0 0
103 103 2 2 1 1 1 2 1 1 1 NA 0 0 0 0 0 0 0 0
105 105 1 2 1 1 1 1 1 1 1 NA 0 0 1 0 0 0 0 0
107 107 1 2 1 1 1 1 1 1 1 15 0 0 1 0 0 0 0 0
108 108 1 1 1 1 2 1 1 2 2 NA 0 0 1 0 0 0 0 0
109 109 2 2 1 1 2 1 1 1 2 25 0 0 0 1 0 0 0 0



if those are tabs(x'09') and not spaces between the columns you will need to use a '\t':
Code:
$ sed '/NA\tNA\tNA\tNA\tNA\tNA\tNA\tNA$/d' test.txt
FID     IID     aspirpre        statihos        fibrahos        ocholhos        arbhos  betabhos        alphbhos        cacbhos diurehos        numbcig.x   toast1   toast2  toast3  toast4  ischoth1        ischoth2        ischoth3        ischoth4
101     101     1       1       1       1       1       2       1       2       1       15      1       0       0       0       0       0       0       0
103     103     2       2       1       1       1       2       1       1       1       NA      0       0       0       0       0       0       0       0
105     105     1       2       1       1       1       1       1       1       1       NA      0       0       1       0       0       0       0       0
107     107     1       2       1       1       1       1       1       1       1       15      0       0       1       0       0       0       0       0
108     108     1       1       1       1       2       1       1       2       2       NA      0       0       1       0       0       0       0       0
109     109     2       2       1       1       2       1       1       1       2       25      0       0       0       1       0       0       0       0

This User Gave Thanks to spacebar For This Post:
johnkim0806
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Average across rows with a condition

Hi Friends, My input file Gene1 10 20 0 Gene2 5 0 15 Gene3 10 10 10 Gene4 5 0 0 If there is a zero for any gene in any column, I don't want that column to be considered which reduces the denominator value during average. Here is my output Gene1 10 20 0 10 Gene2 5 0 15 10 Gene3... (5 Replies)
Discussion started by: jacobs.smith
5 Replies

2. Shell Programming and Scripting

Deleting the records based on the condition

Hi, Can any one help me, in deleting the records from the database table based on the following condition: script should take a configurable parameter as input. The input is nothing but “no. of years”. For example, if I enter 2 as input parameter, then the 2 year old records should get... (2 Replies)
Discussion started by: zxcjggu708
2 Replies

3. UNIX for Dummies Questions & Answers

Deleting the unwanted data based on condition

hi i have my input data like this aaa bbb ccc asa dff nmj mnj saa dff oik aax cdx saa oik asq sdf dssi want my output file to be like this mnj saa dff oik aax cdx saa oiki want to retain only those lines which will have oik just below them and i want oik to be as next column to those... (1 Reply)
Discussion started by: anurupa777
1 Replies

4. Shell Programming and Scripting

selecting and deleting specific lines with condition

I have a set of data as below: The first field, $1 represent "|". The $3 (3rd field) and $6 (6th field) in my data file represent "number-molecule" which has arrangement as below: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 ... (9 Replies)
Discussion started by: vjramana
9 Replies

5. Shell Programming and Scripting

Parsing a CSV file and deleting all rows on condition

Hello list, I am working on a csv file which contains two fields per record which contain IP addresses. What I am trying to do is find records which have identical fields(IP addresses) which occur 4(four) times, and if they do, delete all records with that specific identical field(ip address). ... (4 Replies)
Discussion started by: landossa
4 Replies

6. Shell Programming and Scripting

deleting rows that have certain characters

Hi, I want to delete rows whenever column one has the letters 'rpa'. The file is tab seperated. e.g. years 1 bears 1 cats 2 rpat 3 rpa99 4 rpa011 5 then removing 'rpa' containing rows based on the first column years 1 bears 1 cats 2 thanks (7 Replies)
Discussion started by: phil_heath
7 Replies

7. Shell Programming and Scripting

Deleting of Specific Rows.

Fruit : Price : Quantity apple : 20 : 40 chiku : 40 :30 Hey guys, i have written a code using sed to delete a specific char which is being typed in. But the problem i am having is , how can i expand my coding to actually allow it do delete the whole row. For example,... (21 Replies)
Discussion started by: gregarion
21 Replies

8. Shell Programming and Scripting

Deleting specific rows in large files having rows greater than 100000

Hi Guys, I need help in modifying a large text file containing more than 1-2 lakh rows of data using unix commands. I am quite new to the unix language the text file contains data in a pipe delimited format sdfsdfs sdfsdfsd START_ROW sdfsd|sdfsdfsd|sdfsdfasdf|sdfsadf|sdfasdf... (9 Replies)
Discussion started by: manish2009
9 Replies

9. Shell Programming and Scripting

deleting rows that dont have ....

Hi I posted earlier. This is sorta similar but I want to delete rows that dont have R, T, Y or U. Nam1 RTYU Nam2 RRTT Nam3 RYTU Nam4 IRTT So the output would look like this? Nam1 RTYU Nam2 RRTT Nam3 RYTU too many problems thanks (3 Replies)
Discussion started by: kylle345
3 Replies

10. Shell Programming and Scripting

Deleting rows that begin with #

Hi, I have a file that has rows that start with # and ends with #. For example.. # hi text JK NM JK NM JK K JK NM # no # yes So I want to remove the #'s and put them into another file. so the output will be two files.. File 1: JK NM JK NM JK K JK NM (3 Replies)
Discussion started by: phil_heath
3 Replies
Login or Register to Ask a Question