Help required deleting specific lines from file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help required deleting specific lines from file
# 1  
Old 02-07-2014
Wrench Help required deleting specific lines from file

Hi,

I have a file with 20 columns of data and hundreds of lines of the same format.

Here is an example line. The data repeats underneath with the same format.

Code:
  15 1 4 GLY - 1 65 LYSH 23 N - 24 H - 634 O 0.188 157.552 487 48.70

I have been sorting this data by hand but I was wondering if I can use awk, grep, sed or the like to simplify this task.

Some things I want to do to the data:
1) Remove all lines in which column 20 has a value less than 5.00 (This line is a percentage with 2 decimal places, and i would like to remove values with <5%.)

2) Remove all lines in which column 3 IS NOT one of the following values: 3-4, 18-22, 30-38, 42-49, 52-59, 63-65 (ie. 3, 4, 18, 19, 20, 21, 22, 30, 31, 32, 33, 34, 35, 36, 37, 38, etc) (so removing all lines in which column 3 has the value 1, 2, 5 etc)

I realise this is quite a complicated request, but if anyone has any ideas as to how I could achieve this it would be gratefully received.

Last edited by livbaddeley; 02-07-2014 at 09:01 AM.. Reason: changed numbers
# 2  
Old 02-07-2014
A few questions in response:-
  • What have you tried so far?
  • Where are you stuck?
  • What OS and version are you using?
  • What tools would you like to work with?
Most importantly,
  • What have you tried so far?


Regards,
Robin
# 3  
Old 02-07-2014
Hi,

I'm running Linux and so using bash shell.

I have to admit I haven't tried anything yet as I am quite new to Linux (I am a new researcher) and I can only do very simple manipulations to files like basic awk commands.

---------- Post updated at 01:32 PM ---------- Previous update was at 01:07 PM ----------

Hi again!

I've managed to find a solution to point 1) using awk.

Code:
awk '$20 > 5.00 {print $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20}' filename > filename2

Is this the simplest way of achieving point 1?

I still need help with point 2.

Many thanks!
Liv
This User Gave Thanks to livbaddeley For This Post:
# 4  
Old 02-07-2014
I'm not great on awk, but I think you could shorten it to:-
Code:
awk '$20 > 5.00 {print $0}' infile > outfile



Robin
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Deleting specific lines from text file via scripting

Hi, I'm trying to search for some number and from that line, i need to delete the 5th line exactly. Eg: Consider below as text file data: 10000 a b c d e . . . 10000 w q t (8 Replies)
Discussion started by: Gautham
8 Replies

2. Shell Programming and Scripting

Issue deleting all lines (having a specific string) in a file

I'm trying to create a script. There are 2 files - fileA.log & fileB.log fileA.log has the below data : aaaa cccc eeee fileB.log has the below data : cjahdskjah aaaa xyz jhaskjdhas bbbb abc ajdhjkh cccc abc cjahdskjah ... (7 Replies)
Discussion started by: Pandee
7 Replies

3. Shell Programming and Scripting

Deleting lines in a fixed length file where there is a word at specific location

I have a big file having 100 K lines. I have to read each line and see at 356 character position whethere there is a word "W" in it. If it is their then don't delete the line otherwise delete it. There are two lines as one Header and one trailer which should remain same. Can somebody... (5 Replies)
Discussion started by: mohit kanoongo
5 Replies

4. UNIX for Dummies Questions & Answers

Deleting lines that contain a specific string from a space delimited text file?

Hi, I have a space delimited text file that looks like the following: 250 rs10000056 0.04 0.0888 4 189321617 250 rs10000062 0.05 0.0435 4 5254744 250 rs10000064 0.02 0.2403 4 127809621 250 rs10000068 0.01 NA 250 rs1000007 0.00 0.9531 2 237752054 250 rs10000081 0.03 0.1400 4 17348363... (5 Replies)
Discussion started by: evelibertine
5 Replies

5. Shell Programming and Scripting

Deleting specific lines in a file

Hello, I have a file filled with dates, such as: 04-08-2011 message 04-08-2011 message 03-08-2011 message 01-08-2011 message 31-07-2011 message 24-07-2011 message 15-07-2011 message 13-12-2008 message 26-11-2007 message And I want to delete those lines whose date is older than 10... (5 Replies)
Discussion started by: asanchez
5 Replies

6. Shell Programming and Scripting

deleting specific lines in a file

Hello, I have a file like: 26-07-2011 sunz02 message1 26-07-2011 sunz02 message2 26-07-2011 sunz02 message3 15-07-2011 sunz02 message1 15-07-2011 sunz02 message2 15-07-2011 sunz02 message3... (5 Replies)
Discussion started by: asanchez
5 Replies

7. Shell Programming and Scripting

Deleting specific lines in a file

Hello, I have a file like this one: 03-07-2011 sunz02 message1 03-07-2011 sunz02 message2 03-07-2011 sunz02 message3 01-07-2011 sunz02 message1 01-07-2011 sunz02 message2 01-07-2011 sunz02 ... (1 Reply)
Discussion started by: asanchez
1 Replies

8. Shell Programming and Scripting

deleting specific lines in a file

I want to delete all lines from a file (orig_file) that contain the regex values (bad_inv_list) I tried a for each loop with sed but it isn't working for file in `cat bad_inv_list`; do sed '/$file/d' orig_file > pared_down_file.1 mv pared_down_file.1 orig_file done I've added... (2 Replies)
Discussion started by: verge
2 Replies

9. Shell Programming and Scripting

Shell deleting specific lines

Hi, I'am working under unix solaris I have a text file with set of lines, each set of lines (BLOCK) have three fixed lines : Between SECND line and THEND we have N lines, N differ from a block to another sample : i have to make a script wich delete each 3 fixed lines if N=0... (3 Replies)
Discussion started by: salbanito
3 Replies

10. Shell Programming and Scripting

Deleting specific lines in a file

I have a file which has some lines starting with a particular word. I would like to delete 5 lines before each such line containing that particular word. eg: line1 line2 line3 line4 line5 line6 "particular word"... I would like to delete line2-line6 and all such occurences in that... (4 Replies)
Discussion started by: ramu_1980
4 Replies
Login or Register to Ask a Question