Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Remove Corrupted Records in the file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Remove Corrupted Records in the file
# 1  
Old 09-14-2011
Remove Corrupted Records in the file
Hi All,

Please help me with the below query. I need to delete the bad records in the file and have to save the file after deleting them. Thanks in advance.

Ex:
-----
I have a file called ABC and the data in the file is as follows.

Code:
08XS021642525520110830BBSBI              99.9375                 100           .28521415                 .26                   020110830075937                   0
08XS021811173920110830BBSBI             99.96875           100.03125                                                           020110830075937                   0
08XS021811319820110830BBSBI             99.96875           100.03125                                                           020110830075937                   0
08USG3603GAD4620110830BBSBI                  100                 100                                                           020110830075937                   0
08XS022341103320110830BBSBI             99.96875           100.03125                                                           020110830075937                   0
08XS022341154620110830BBSBI             99.96875           100.03125                                                           020110830075937                   0
08XS022341189220110830BBSBI             99.96875           100.03125                                                           020110830075937                   0
08XS022456431920110830BBSBI             99.96875           100.03125                                                           020110830075937                   0
08XS022307088820110830BBSBI             99.96875           100.03125                                                           020110830075937                   0

So here I have to go through this file and search for Interpreter and if any results for that then I have to delete the entire line. So my output should be like this.

Output:
-----------
Code:
08XS021642525520110830BBSBI              99.9375                 100           .28521415                 .26                   020110830075937                   0
08XS021811173920110830BBSBI             99.96875           100.03125                                                           020110830075937                   0
08XS021811319820110830BBSBI             99.96875           100.03125                                                           020110830075937                   0
08XS022341103320110830BBSBI             99.96875           100.03125                                                           020110830075937                   0
08XS022341154620110830BBSBI             99.96875           100.03125                                                           020110830075937                   0
08XS022341189220110830BBSBI             99.96875           100.03125                                                           020110830075937                   0
08XS022456431920110830BBSBI             99.96875           100.03125                                                           020110830075937                   0
08XS022307088820110830BBSBI             99.96875           100.03125                                                           020110830075937                   0


Moderator's Comments:
Mod Comment
Please use code tags when posting data and code samples!
Last edited by vgersh99; 09-14-2011 at 10:23 AM.. Reason: code tags, please
# 2  
Old 09-14-2011
Hi,

I don't understand what 'search for Interpreter' means? Can you describe it?

How to know which records are corrupt?

Regards,
Birei
# 3  
Old 09-14-2011
same here....
does "corrupt" mean $2==$3==100?
# 4  
Old 09-14-2011
Hi Birei,

Thanks. Bad records will always contain the word "Interpreter" in it. So I came to a conclusion if we can grep for this like grep Interpreter, then we can remove that entire line from the file. If you any other better alternative also, please suggest the same.
# 5  
Old 09-14-2011
Quote:
Originally Posted by rajeshamathi

Bad records will always contain the word "Interpreter" in it.
man,in your example, there is no Interpreter at all.

if you want to filter out the lines containing certain word, e.g. "interpreter" you can try
Code:
grep -v

# 6  
Old 09-14-2011
Code:
printf "/ 100 100 /d\nw\nq\n" | ed -s yourfile

... but would require a loop

or
Code:
printf ",s/^.*[[:blank:]]*100[[:blank:]]*100.*//g\nw\nq\n" | ed -s yourfile

but would leave empty lines
Last edited by ctsgnb; 09-14-2011 at 11:23 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove bad records from file and move them into a file then send those via email

Hi my requirement is that i want pull the bad records from input file and move those records in to a seperate file. that file has to be sent via email.. any suggentions please (1 Reply)
Discussion started by: sxk4999
1 Replies

2. UNIX for Dummies Questions & Answers

Need to remove certain records off a file.

New to unix. I have a couple files of 5 million records. I have a key field on those records. I have about 300 keys that I need to remove off the file, and I don't want to write a program to do it. I have used grep -v in the past and that works great for a few records, but I can't see myself... (2 Replies)
Discussion started by: jclanc8
2 Replies

3. Shell Programming and Scripting

Deleting duplicate records from file 1 if records from file 2 match

I have 2 files "File 1" is delimited by ";" and "File 2" is delimited by "|". File 1 below (3 record shown): Doc1;03/01/2012;New York;6 Main Street;Mr. Smith 1;Mr. Jones Doc2;03/01/2012;Syracuse;876 Broadway;John Davis;Barbara Lull Doc3;03/01/2012;Buffalo;779 Old Windy Road;Charles... (2 Replies)
Discussion started by: vestport
2 Replies

4. Shell Programming and Scripting

how to remove particular records from a file???

I need to remove header(H) and trailer(T) from a file keeping other records as such. The source file will look as below I have to remove H|20120203_000500|20120203_000500 and T| 10111246 from the above file. Please let me know how to do... (6 Replies)
Discussion started by: siteregsam
6 Replies

5. Shell Programming and Scripting

Remove somewhat Duplicate records from a flat file

I have a flat file that contains records similar to the following two lines; 1984/11/08 7 700000 123456789 2 1984/11/08 1941/05/19 7 700000 123456789 2 The 123456789 2 represents an account number, this is how I identify the duplicate record. The ### signs represent... (4 Replies)
Discussion started by: jolney
4 Replies

6. UNIX for Dummies Questions & Answers

Grep specific records from a file of records that are separated by an empty line

Hi everyone. I am a newbie to Linux stuff. I have this kind of problem which couldn't solve alone. I have a text file with records separated by empty lines like this: ID: 20 Name: X Age: 19 ID: 21 Name: Z ID: 22 Email: xxx@yahoo.com Name: Y Age: 19 I want to grep records that... (4 Replies)
Discussion started by: Atrisa
4 Replies

7. Shell Programming and Scripting

Remove Duplicate Records

Hi frinds, Need your help. item , color ,desc ==== ======= ==== 1,red ,abc 1,red , a b c 2,blue,x 3,black,y 4,brown,xv 4,brown,x v 4,brown, x v I have to elemnet the duplicate rows on the basis of item. the final out put will be 1,red ,abc (6 Replies)
Discussion started by: imipsita.rath
6 Replies

8. Shell Programming and Scripting

Remove duplicate records

I want to remove the records based on duplicate. I want to remove if two or more records exists with combination fields. Those records should not come once also file abc.txt ABC;123;XYB;HELLO; ABC;123;HKL;HELLO; CDE;123;LLKJ;HELLO; ABC;123;LSDK;HELLO; CDF;344;SLK;TEST key fields are... (7 Replies)
Discussion started by: svenkatareddy
7 Replies

9. Shell Programming and Scripting

Remove all instances of duplicate records from the file

Hi experts, I am new to scripting. I have a requirement as below. File1: A|123|NAME1 A|123|NAME2 B|123|NAME3 File2: C|123|NAME4 C|123|NAME5 D|123|NAME6 1) I have 2 merge both the files. 2) need to do a sort ( key fields are first and second field) 3) remove all the instances... (3 Replies)
Discussion started by: vukkusila
3 Replies

10. Shell Programming and Scripting

Count No of Records in File without counting Header and Trailer Records

I have a flat file and need to count no of records in the file less the header and the trailer record. I would appreciate any and all asistance Thanks Hadi Lalani (2 Replies)
Discussion started by: guiguy
2 Replies
Login or Register to Ask a Question