Deleting specific rows in large files having rows greater than 100000


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Deleting specific rows in large files having rows greater than 100000
# 8  
Old 12-11-2009
Code:
awk '/END/{f=0}/START/{f=1;next}f' file

# 9  
Old 12-11-2009
The below code is working fine.

sed '1,/START-OF-DATA/d;/END-OF-DATA/,$d' corp_pfd_asia.out > corp_pfd_asiaout.txt

However there is a slight problem. The header of the data ends with "# PRODUCT=Corp/Pfd" after START-OF-DATA. when i input "# PRODUCT=Corp/Pfd" instead of "START-OF-DATA" it gives error think coz of "/"

sed '1,/PRODUCT=Corp/Pfd/d;/END-OF-DATA/,$d' corp_pfd_asia.out > corp_pfd_asiaout.txt


Thank you very much
Manish

---------- Post updated at 04:18 PM ---------- Previous update was at 03:42 PM ----------

The below code is working fine.

sed '1,/START-OF-DATA/d;/END-OF-DATA/,$d' corp_pfd_asia.out > corp_pfd_asiaout.txt

However there is a slight problem. The header of the data ends with "# PRODUCT=Corp/Pfd" after START-OF-DATA.

when i input "# PRODUCT=Corp/Pfd" instead of "START-OF-DATA" it gives error think coz of "/"

sed '1,/# PRODUCT=Corp/Pfd/d;/END-OF-DATA/,$d' corp_pfd_asia.out > corp_pfd_asiaout.txt


Thank you very much
Manish
# 10  
Old 12-11-2009
Hi Manish 2009:
You will have to escape forward / like this: \/ :
Code:
sed '1,/# PRODUCT=Corp\/Pfd/d;/END-OF-DATA/,$d' corp_pfd_asia.out > corp_pfd_asiaout.txt

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extracting data from specific rows and columns from multiple csv files

I have a series of csv files in the following format eg file1 Experiment Name,XYZ_07/28/15, Specimen Name,Specimen_001, Tube Name, Control, Record Date,7/28/2015 14:50, $OP,XYZYZ, GUID,abc, Population,#Events,%Parent All Events,10500, P1,10071,95.9 Early Apoptosis,1113,11.1 Late... (6 Replies)
Discussion started by: pawannoel
6 Replies

2. UNIX for Dummies Questions & Answers

Deleting rows where the value in a specific column match

Hi, I have a tab delimited text file where I want to delete all rows that have the same string for column 1. How do I go about doing that? Thanks! Example Input: aa 1 aa 2 aa 3 bb 4 bc 5 bb 6 cd 8 Output: bc 5 cd 8 (4 Replies)
Discussion started by: evelibertine
4 Replies

3. Shell Programming and Scripting

deleting rows under a certain condition

there are 20 variables and I would like to delete the rows if 13th-20th columns are all NA. Thank you! FID IID aspirpre statihos fibrahos ocholhos arbhos betabhos alphbhos cacbhos diurehos numbcig.x toast1 toast2 toast3 toast4 ischoth1 ischoth2 ischoth3 ischoth4 101 101 1 1 1 1 1 2 1 2... (2 Replies)
Discussion started by: johnkim0806
2 Replies

4. UNIX for Dummies Questions & Answers

Deleting specific rows from a text file

How do I go about deleting specific rows from a text file (given row number)? (5 Replies)
Discussion started by: evelibertine
5 Replies

5. UNIX for Dummies Questions & Answers

Help with deleting specific rows from a text file

I know this is a complicated question but I will try to illustrate it with some data. I have a data file that looks like the following: 1341 NA06985 0 0 2 46.6432798439 1341 NA06991 NA06993 NA06985 2 48.8478948517 1341 NA06993 0 0 1 45.8022601455 1340 NA06994 0 0 1 48.780669145 1340... (1 Reply)
Discussion started by: evelibertine
1 Replies

6. Shell Programming and Scripting

Large file - columns into rows etc

I have done a couple of searches on this and have found many threads but I don't think I've found one that is useful to me - probably because I have very basic comprehension of perl and beginners shell so trying to manipulate a script already posted maybe beyond my capabilities.... Anyway - I... (26 Replies)
Discussion started by: Myrona
26 Replies

7. Shell Programming and Scripting

deleting rows that have certain characters

Hi, I want to delete rows whenever column one has the letters 'rpa'. The file is tab seperated. e.g. years 1 bears 1 cats 2 rpat 3 rpa99 4 rpa011 5 then removing 'rpa' containing rows based on the first column years 1 bears 1 cats 2 thanks (7 Replies)
Discussion started by: phil_heath
7 Replies

8. Shell Programming and Scripting

Deleting of Specific Rows.

Fruit : Price : Quantity apple : 20 : 40 chiku : 40 :30 Hey guys, i have written a code using sed to delete a specific char which is being typed in. But the problem i am having is , how can i expand my coding to actually allow it do delete the whole row. For example,... (21 Replies)
Discussion started by: gregarion
21 Replies

9. Shell Programming and Scripting

Deleting rows that begin with #

Hi, I have a file that has rows that start with # and ends with #. For example.. # hi text JK NM JK NM JK K JK NM # no # yes So I want to remove the #'s and put them into another file. so the output will be two files.. File 1: JK NM JK NM JK K JK NM (3 Replies)
Discussion started by: phil_heath
3 Replies

10. Shell Programming and Scripting

Deleting specific files greater then 90 days

I have a directory that contains a number of history files for a process. Every file starts with the string "EVACK". FOr example EVACK0101.000001 EVACK0102.095940 EVACKmmdd.hhmiss I want to erase in the specific directory ONLY(no sub-directories) all EVACK files older then 90 days. I... (2 Replies)
Discussion started by: beilstwh
2 Replies
Login or Register to Ask a Question