Visit Our UNIX and Linux User Community


Deleting rows from csv file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Deleting rows from csv file
# 1  
Old 09-30-2009
Deleting rows from csv file

Hello,

I am supposed to process about 100 csv files. But these files have some extra lines at the bottom of the file. these extra lines start with a header for each column and then some values below. These lines are actually a summary of the actual data and not supposed to be processed. These unncessary lines may be different for each files. Data To be deleted are as under:


Yr, Month, Prod
2009, 09, XYX
2008, 04, YYY

Is there any way I can delete these unnecessary lines of each files without opening them individually and manually deleting it?

I Appreciate a help in this.

Regards
# 2  
Old 09-30-2009
sed -e '/Yr, Month, Prod/d' -e '/2009, 09, XYX/d' -e '/2008, 04, YYY/d'
# 3  
Old 09-30-2009
need -i option in above sed command.
# 4  
Old 10-01-2009
Quote:
Originally Posted by rdcwayx
need -i option in above sed command.
True. But this option should be use with care! No try and error type of thing. Better use the option -i with a suffix like -i.bak which will create an copy of original file, just in case... Smilie

theshashi's solution will only work on the sample file given by OP. I understand that the lines following the Yr, Month, Prod line can have different values.

Here is a possible awk solution:
Code:
awk '/Yr, Month, Prod/{f=1}!f' file

Which is shorthand for:
Code:
awk '/Yr, Month, Prod/{f=1}!f{print}' file

# 5  
Old 10-01-2009
Code:
 
sed '/Yr, Month, Prod/', $d' file


Last edited by edidataguy; 10-01-2009 at 03:53 AM..
# 6  
Old 10-01-2009
deleting lines after a specific line

Basically my requirement is to delete all the lines that follows the below mentioned line. ( delete should include the below line also)

Please help

"Yr, Month, Prod"
# 7  
Old 10-01-2009
Did you tried the awk and sed solutions given above?

---------- Post updated at 05:46 PM ---------- Previous update was at 05:42 PM ----------

There is a small typo in edidataguy's solution:

Code:
sed '/Yr, Month, Prod/', $d' file
                      ^
# should be

sed '/Yr, Month, Prod/,$d' file


Previous Thread | Next Thread
Test Your Knowledge in Computers #900
Difficulty: Medium
KornShell (ksh) is a Unix shell which was developed by David Korn at UC Berkeley.
True or False?

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Get duplicate rows from a csv file

How can i get the duplicates rows from a file using unix, for example i have data like a,1 b,2 c,3 d,4 a,1 c,3 e,5 i want output to be like a,1 c,3 (4 Replies)
Discussion started by: ggupta
4 Replies

2. Shell Programming and Scripting

Deleting all the fields(columns) from a .csv file if all rows in that columns are blanks

Hi Friends, I have come across some files where some of the columns don not have data. Key, Data1,Data2,Data3,Data4,Data5 A,5,6,,10,, A,3,4,,3,, B,1,,4,5,, B,2,,3,4,, If we see the above data on Data5 column do not have any row got filled. So remove only that column(Here Data5) and... (4 Replies)
Discussion started by: ks_reddy
4 Replies

3. UNIX for Dummies Questions & Answers

Deleting specific rows from a text file

How do I go about deleting specific rows from a text file (given row number)? (5 Replies)
Discussion started by: evelibertine
5 Replies

4. UNIX for Dummies Questions & Answers

Help with deleting specific rows from a text file

I know this is a complicated question but I will try to illustrate it with some data. I have a data file that looks like the following: 1341 NA06985 0 0 2 46.6432798439 1341 NA06991 NA06993 NA06985 2 48.8478948517 1341 NA06993 0 0 1 45.8022601455 1340 NA06994 0 0 1 48.780669145 1340... (1 Reply)
Discussion started by: evelibertine
1 Replies

5. Shell Programming and Scripting

Parsing a CSV file and deleting all rows on condition

Hello list, I am working on a csv file which contains two fields per record which contain IP addresses. What I am trying to do is find records which have identical fields(IP addresses) which occur 4(four) times, and if they do, delete all records with that specific identical field(ip address). ... (4 Replies)
Discussion started by: landossa
4 Replies

6. Shell Programming and Scripting

Deleting specific rows in large files having rows greater than 100000

Hi Guys, I need help in modifying a large text file containing more than 1-2 lakh rows of data using unix commands. I am quite new to the unix language the text file contains data in a pipe delimited format sdfsdfs sdfsdfsd START_ROW sdfsd|sdfsdfsd|sdfsdfasdf|sdfsadf|sdfasdf... (9 Replies)
Discussion started by: manish2009
9 Replies

7. Shell Programming and Scripting

Deleting columns from CSV file

Hi All, Am working on perl script which should delete columns in existing CSV file. If my file is : AA,BB,CC,DD 00,11,22,33 00,55,66,77 00,99,88,21 AA,BB... are all my headers can come in any order (e.g AA,CC,BB...) and rest are values. I want to delete column CC... Can anybody help... (2 Replies)
Discussion started by: darshakraut
2 Replies

8. Shell Programming and Scripting

deleting rows & columns form a csv file

Hi , I want to delete some rows & columns from file. can someone please help me on this? Regards. (2 Replies)
Discussion started by: code19
2 Replies

9. Shell Programming and Scripting

Deleting Lines from .csv file

Hello All, I have a .csv file and I have to delete the selcted records stored in a vairable e.g echo $lname 7 88 91 94 97 100 103 106 I dont know how to pass the variable name to "sed" for deleting the $lname from a file can any one help as this is very urgent. $lname is changing the... (3 Replies)
Discussion started by: 009satya
3 Replies

10. Shell Programming and Scripting

Deleting the emty rows in a file

I am getting some spaces between the two lines(rows) in file.i want delete that empty rows in the file example 1 abc xyz 2 def jkl like i am having lots of rows in a file i want to delete the spce between the two rows give any... (7 Replies)
Discussion started by: srivsn
7 Replies

Featured Tech Videos