Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Delete lines

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Delete lines
# 1  
Old 05-13-2010
Delete lines
Hello.

I've a file with 17 column. I need to filter this file. For example, if the data in column number 2, is smaller than 4, or it's a word, the programa must delete the line.

Input:
Code:
abc            523              2              5       .....
dfghy         0                 54             26     .......
poir            aca             12             -5    ......... 
locid           52             158             -23    .........

Output:

Code:
abc            523              2              5       .....
locid           52             158             -23    .........

Thanks
# 2  
Old 05-13-2010
Code:
 
awk '$2 <4 || $2 ~/[a-z]/ { next } 1'  input_file

This User Gave Thanks to panyam For This Post:
kekaes
# 3  
Old 05-13-2010
Ok. Thanks, it is just what I needed . Can you explain to me the significance of "{ next } 1"?
# 4  
Old 05-13-2010
Quote:
Originally Posted by kekaes
Ok. Thanks, it is just what I needed . Can you explain to me the significance of "{ next } 1"?
Code:
 
 if the data in column number 2, is smaller than 4, or it's a word, go to  "next" line ( ignore the current line )

Last edited by panyam; 05-13-2010 at 09:39 AM..
This User Gave Thanks to panyam For This Post:
kekaes
# 5  
Old 05-13-2010
Shorter Smilie
Code:
awk '$2>3 && $2==int($2)' file

This User Gave Thanks to danmero For This Post:
kekaes
# 6  
Old 05-13-2010
Or using Perl -

Code:
$
$ cat f5
abc            523            2              5
dfghy          0              54             26
poir           aca            12             -5
locid          52             158            -23
xyzw           pq9s           82             95
$
$
$ perl -ane '$F[1]>=4 && print' f5
abc            523            2              5
locid          52             158            -23
$

tyler_durden
This User Gave Thanks to durden_tyler For This Post:
kekaes
# 7  
Old 05-13-2010
The following may be irrelevant, but I point it out as a heads up, just in case.

Quote:
Originally Posted by panyam
Code:
awk '$2 <4 || $2 ~/[a-z]/ { next } 1'  input_file

That expression will match non-numeric data that's not alphabetical. If that is undesirable, I would recommend either using a complemented decimal digit class, [^0-9], or going with danmero's approach.

Code:
$ cat input
abc            523              2              5       .....
dfghy         0                 54             26     .......
poir            4abc             12             -5    ......... 
foo            4&^%$#@             12             -5    ......... 
locid           52             158             -23    .........
$ awk '$2 <4 || $2 ~/[a-z]/ { next } 1' input
abc            523              2              5       .....
foo            4&^%$#@             12             -5    ......... 
locid           52             158             -23    .........

Also, durden_tyler's perl solution is a bit more accepting:

Code:
$ perl -ane '$F[1]>=4 && print' input
abc            523              2              5       .....
poir            4abc             12             -5    ......... 
foo            4&^%$#@             12             -5    ......... 
locid           52             158             -23    .........

Regards,
Alister

---------- Post updated at 12:03 PM ---------- Previous update was at 11:49 AM ----------

I do not recommend using the following in production, as it's not immediately obvious what it does, but I share it in the spirit of AWK golf.

Dedicated to danmero Smilie:
Code:
awk '($2==$2+0)*$2>3' file

For AWK novices, the parenthetical tests whether or not the value in $2 is a non-numeric string. $2+0 forces a conversion to a numeric value. When the result is compared to $2 with ==, if $2 is a numeric value, then two idential numbers are compared and the result of the comparision is 1. Multiplying $2 by 1 doesn't alter the result of the comparison with 3.

However, if $2 is a non-numeric string, the result of $2+0 (a number) will not be identical to $2 (a string), because the conversion process will have discarded the non-numeric portions of the string. When AWK compares the two with ==, it will not be comparing two numbers. When one value in a comparison is a non-numeric string, the other is converted (if necessary) to a string before the comparison. In this case, the result is a comparison between two unidentical strings, which yields the number 0. 0>$3 is false and the line is skipped.

This approach allows floating point values in $2, while the int() approach does not. I only mention this for the sake of being thorough in this description, not to tout it as an advantage.

If nothing else, that AWK snippet makes for a good lesson in AWK type conversion Smilie

Cheers,
Alister
Last edited by alister; 05-13-2010 at 01:24 PM..
This User Gave Thanks to alister For This Post:
kekaes
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Delete multiple lines between blank lines containing two patterns

Hi all, I'm looking for a way (sed or awk) to delete multiple lines between blank lines containing two patterns ex: user: alpha parameter_1 = 15 parameter_2 = 1 parameter_3 = 0 user: alpha parameter_1 = 15 parameter_2 = 1 parameter_3 = 0 user: alpha parameter_1 = 16... (3 Replies)
Discussion started by: ce9888
3 Replies

2. Shell Programming and Scripting

Sed/awk to delete single lines that aren't touching other lines

Hello, I'm trying to figure out how to use sed or awk to delete single lines in a file. By single, I mean lines that are not touching any other lines (just one line with white space above and below). Example: one two three four five six seven eight I want it to look like: (6 Replies)
Discussion started by: slimjbe
6 Replies

3. Shell Programming and Scripting

search and replace, when found, delete multiple lines, add new set of lines?

hey guys, I tried searching but most 'search and replace' questions are related to one liners. Say I have a file to be replaced that has the following: $ cat testing.txt TESTING AAA BBB CCC DDD EEE FFF GGG HHH ENDTESTING This is the input file: (3 Replies)
Discussion started by: DeuceLee
3 Replies

4. UNIX for Advanced & Expert Users

In a huge file, Delete duplicate lines leaving unique lines

Hi All, I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space. I dont know if this works : I want to read each line of the File in a For Loop, and want to... (16 Replies)
Discussion started by: krishnix
16 Replies

5. Shell Programming and Scripting

need to delete all lines from a group of files except the 1st 2 lines

Hello, I have a group of text files with many lines in each file. I need to delete all the lines in each and only leave 2 lines in each file. (3 Replies)
Discussion started by: script_op2a
3 Replies

6. UNIX for Dummies Questions & Answers

How get only required lines & delete the rest of the lines in file

Hiiii I have a file which contains huge data as a.dat: PDE 1990 1 9 18 51 28.90 24.7500 95.2800 118.0 6.1 0.0 BURMA event name: 010990D time shift: 7.3000 half duration: 5.0000 latitude: 24.4200 longitude: 94.9500 depth: 129.6000 Mrr: ... (7 Replies)
Discussion started by: reva
7 Replies

7. Shell Programming and Scripting

sed problem - delete all lines until a match on 2 lines

First of all, I know this can be more eassily done with perl or other scripting languages but, that's not the issue. I need this in sed. (or wander if it's possible ) I got a file (trace file to recreate the control file from oracle for the dba boys) which contains some lines another line... (11 Replies)
Discussion started by: plelie2
11 Replies

8. Shell Programming and Scripting

How to delete lines in a file that have duplicates or derive the lines that aper once

Input: a b b c d d I need: a c I know how to get this (the lines that have duplicates) : b d sort file | uniq -d But i need opossite of this. I have searched the forum and other places as well, but have found solution for everything except this variant of the problem. (3 Replies)
Discussion started by: necroman08
3 Replies

9. Shell Programming and Scripting

Grep and delete lines except the lines with strings

Hi I am writing a script which should read a file and search for certain strings 'approved' or 'removed' and retain only those lines that contain the above strings. Ex: file name 'test' test: approved package waiting for approval package disapproved package removed package approved... (14 Replies)
Discussion started by: vj8436
14 Replies

10. Shell Programming and Scripting

How to delete first 5 lines and last five lines in all text files

Hi I want to delete first five and last five lines in text files without opening the file and also i want to keep the same file name for all the files. Thanks in advance!!! Ragav (10 Replies)
Discussion started by: ragavendran31
10 Replies
Login or Register to Ask a Question