deleting lines using awk-best way?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting deleting lines using awk-best way?
# 1  
Old 04-24-2006
Power deleting lines using awk-best way?

Hi all....I'm using awk to validate a csv file, but now I've been told to delete any invalid lines from the file.. Im not sure what the best way to do this is?

Would it be to create a temp file say "csv_temp.tmp" file and print all the valid records to that temp file. Then delete the old file "csv_file.csv" and rename the temp file to "csv_file.csv"?

is this an efficient way of doing this? any code snippets of kind of file manipulation would be greatly appreciated?

Kind Regards

Satnam
# 2  
Old 04-24-2006
If this is real data, then consider writing the data into two files: good.csv bad.csv
Then when someone asks 'What did you exclude?' - bad.csv has your answers.
# 3  
Old 04-24-2006
Bug cheers..

Hi jim, thanks for the feedback always useful stuff!

cheers
# 4  
Old 08-29-2007
can i know how i can delete a particular record from a 1 million record file using awk?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Mainain the files after deleting lines with awk

hello I want to delete the lines of many files (e.g. file1, file2, ..., file1111) in my directory (e.g. dir1) that they have the 2nd column grater than a value (e.g. 40) placing them to a different directory (e.g. newdir) and keeping the originals. I wrote the code which is functional awk '{... (3 Replies)
Discussion started by: phaethon
3 Replies

2. Shell Programming and Scripting

Deleting all lines containing numbers

Hi guys I have a text file in the following format what i would like ot do is iterate through the file deleting the lines containing only numbers. I have googled this and have been unable to find any help ( maybe its my search terms) so if any one an give me a heads up i would... (3 Replies)
Discussion started by: dunryc
3 Replies

3. Shell Programming and Scripting

Deleting lines above the patterns

I want to delete 1 line above the paatern and 3 line below the pattern and the pattern line itself, on the whole 5 lines. If there are three patterns what to do and the final text file to be captured in a new file. (3 Replies)
Discussion started by: razen
3 Replies

4. Shell Programming and Scripting

deleting lines between patterns using sed or awk

hi, Here is excerpt from my xml file <!-- The custom module to do the authentication for LDAP --> </login-module> <login-module code="com.nlayers.seneca.security.LdapLogin" flag="sufficient"> <module-option... (1 Reply)
Discussion started by: sunrexstar
1 Replies

5. Shell Programming and Scripting

awk : deleting specific incorrect lines

Hello friends, I searched in forums for similar threads but what I want is to have a single awk code to perform followings; I have a big log file going like this; ... 7450494 1724465 -47 003A98B710C0 7450492 1724461 -69 003A98B710C0 7450488 1724459 001DA1915B70 trafo_14:3 7450482... (5 Replies)
Discussion started by: enes71
5 Replies

6. Shell Programming and Scripting

deleting lines in ex

Hello, im using ex to manipulate some text. Im trying to delete all the lines except those on which a certain regex can be found. the important part of the script: ex temp << 'HERE' g/regex/p HERE this command prints the lines I want to end up with, but it doesnt delete the others.... (2 Replies)
Discussion started by: drareeg
2 Replies

7. Shell Programming and Scripting

Deleting particular lines.

hi all, i have got a scenario in which i need to delete all the lines that ends with file names. e.g. input can be cms/images/services_icons/callback.png cms/cms/images/services_icons/sync.php cms/cms/images/services_icons and output should be cms/cms/images/services_icons ... (13 Replies)
Discussion started by: kashifv
13 Replies

8. Shell Programming and Scripting

deleting last n lines from a output

Friends, I am executing this command in solaris sar -d 3 3 | awk 'NR > 2 { if ($1 !~ /,.+/) print }' | egrep -v "nfs|device" . Now i want to delete the last two lines of my output as they are records of average which i don't want. can some one pls give me some idea on how to proceed. (7 Replies)
Discussion started by: achak01
7 Replies

9. Shell Programming and Scripting

deleting lines

I am trying deleting lines from a text file using sed.. sed '/OBJECT="ABC/{N;N;N;d; }' will do if i have to delete lines starting with Object and next 3 lines but I was looking for a way to delet lines starting with OBJECT and all the lines till it reaches a blank lines ..or it reaches a... (8 Replies)
Discussion started by: ajnabi
8 Replies

10. Programming

deleting lines

I am spooling a file from oracle and trying to delete the last line of the spooled file which I am unable to do. Problem is that this file can have multiple records each time and I have no way of knowing how many because the amount can vary. I had an idea of using a while loop to read the... (1 Reply)
Discussion started by: supercbw
1 Replies
Login or Register to Ask a Question