To remove a particular record from File


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting To remove a particular record from File
# 1  
Old 01-06-2010
Bug To remove a particular record from File

Dear Friends,
Need your help to fix the issue we are facing.
We have a requirement - Need to remove specific records from a comma separated file and create new file. It should check "filter.csv" file and then accordingly remove records from the input file and generate the output file.(not case sensitive removal of records)

Please see below example -
Please see below Model/Plant combinations, we want to remove records which contains below combinations and create file.

filter.csv
Code:
TC-P42C1,c231
TC-P50C1.c231
TC-P42X1,C231

Original File -File/Directory: /interfaces/inputfile.csv
Code:
TC-P42C1,C131,0,0,0,0,0,0,0,0,0
TC-P42C1,C201,4261,4261,4261,4261,4261,4261,4261,4261,4261
TC-P42C1,C231,0,0,0,0,0,0,0,0,0
TC-P42C1,C271,0,0,0,0,0,0,0,0,0
TC-P42C1,C311,0,0,0,0,0,0,0,0,0
TC-P42C1,C321,0,0,0,0,0,0,0,0,0
TC-P50C1,C201,1450,4072,4072,4072,4072,4072,4072,4072,4072
TC-P50C1,C231,0,0,0,2,2,2,2,2,2
TC-P50C1,C271,0,0,0,0,0,0,0,0,0
TC-P50C1,C341,0,0,0,0,0,0,0,0,0
TC-P50C1,C501,0,0,0,7,7,7,7,7,7
TC-P50C1,C601,0,0,0,0,0,0,0,0,0
TC-P50C1,C941,0,0,0,0,0,0,0,0,0
TC-P42X1,C141,0,0,0,0,0,0,0,0,0
TC-P42X1,C231,0,0,0,7,7,7,7,7,7
TC-P42X1,C611,0,0,0,0,0,0,0,0,0
TC-P42X1,C621,0,0,0,0,0,0,0,0,0
TC-P42X1,C921,300,300,300,300,300,300,300,300,0
SR-MS102,C141,0,0,0,0,0,0,0,0,0
SR-MS182,C231,0,0,0,0,0,0,0,0,0
SR-TEG18,C322,0,0,0,0,0,0,0,0,0
TC-32LX14,C141,0,0,0,0,0,0,0,0,0
TC-32LX14,C921,0,0,0,0,0,0,0,0,0
TC-32LX14,C231,0,0,0,0,0,0,0,0,0

OutPut File - File/Directory: /interfaces/outputfile.csv
Code:
TC-P42C1,C131,0,0,0,0,0,0,0,0,0
TC-P42C1,C201,4261,4261,4261,4261,4261,4261,4261,4261,4261
TC-P42C1,C271,0,0,0,0,0,0,0,0,0
TC-P42C1,C311,0,0,0,0,0,0,0,0,0
TC-P42C1,C321,0,0,0,0,0,0,0,0,0
TC-P50C1,C201,1450,4072,4072,4072,4072,4072,4072,4072,4072
TC-P50C1,C271,0,0,0,0,0,0,0,0,0
TC-P50C1,C341,0,0,0,0,0,0,0,0,0
TC-P50C1,C501,0,0,0,7,7,7,7,7,7
TC-P50C1,C601,0,0,0,0,0,0,0,0,0
TC-P50C1,C941,0,0,0,0,0,0,0,0,0
TC-P42X1,C141,0,0,0,0,0,0,0,0,0
TC-P42X1,C611,0,0,0,0,0,0,0,0,0
TC-P42X1,C621,0,0,0,0,0,0,0,0,0
TC-P42X1,C921,300,300,300,300,300,300,300,300,0
SR-MS102,C141,0,0,0,0,0,0,0,0,0
SR-MS182,C231,0,0,0,0,0,0,0,0,0
SR-TEG18,C322,0,0,0,0,0,0,0,0,0
TC-32LX14,C141,0,0,0,0,0,0,0,0,0
TC-32LX14,C921,0,0,0,0,0,0,0,0,0
TC-32LX14,C231,0,0,0,0,0,0,0,0,0

We have Unix commands like sed, cut, awk which can help us.
Thanks a lot. Happy New Year to all Unix World Smilie
Satish

Last edited by Scott; 01-06-2010 at 03:06 PM.. Reason: Please use code tags
# 2  
Old 01-06-2010
[edit] I think I misunderstood your problem slightly. Let me rethink.
# 3  
Old 01-06-2010
Thanks Corona for quick reply.

First two columns (model,plant) are maintained in filter.csv file. filter.csv will get modified frequently.
It it not fixed that 2nd will alwyas have value as C231. It can differ.

But whatever is maintained on filter.csv file, we would like Unix to scan the input file(first 2 columns) and remove records which can finds (model,plant) and create output file.

This will help clarify.
# 4  
Old 01-06-2010
Try:

Code:
grep -i -v -f filter.csv inputfile.csv

# 5  
Old 01-06-2010
Franklin got it before I figured it out. Smilie Beware that if there are any blank lines in the filter list, grep will interpret ALL lines as matching and strip out ALL lines, which is what baffled me briefly. Also keep in mind is the -i is there to make it case-insensitive, since the case of two letters in your example input is different from the results you want it to match. If that was accidental you can omit that flag.
# 6  
Old 01-06-2010
Code:
re_string=$( tr '\012' '|' < filter.csv )
re_string=${re_string%\|}
egrep -v "($re_string)" original.csv | tee original.fmt

shoot! nice solution, franklin. <<-- winnar
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Getting error not able remove the record

export PATH=$PATH:/opt/IBM/db2/V9.5/95fp4/bin DB2INSTANCE=bancbcb1 export DB2INSTANCE db2 "connect to dyaxprpt user xyz using byx" echo "Load lastest data start" Row=$(db2 –x “delete from xyz.ROL_yxpwhere axl_ING_ID in(50127,50455,503458,175748)”) If ; then echo “no data deleted” elif ;... (2 Replies)
Discussion started by: jagu
2 Replies

2. Shell Programming and Scripting

Extract timestamp from first record in xml file and it checks if not it will replace first record

I have test.xml <emp><id>101</id><name>AAA</name><date>06/06/14 1811</date></emp> <Join><id>101</id><city>london</city><date>06/06/14 2011</date></join> <Join><id>101</id><city>new york</city><date>06/06/14 1811</date></join> <Join><id>101</id><city>sydney</city><date>06/06/14... (2 Replies)
Discussion started by: vsraju
2 Replies

3. Shell Programming and Scripting

How to remove space from each record in file?

Hi , I want to remove space from each record in one file My file is like BUD, BDL ABC, DDD, ABC ABC, DDD, DDD, KKK The o/p should be BUD,BDL ABC,DDD,ABC ABC,DDD,DDD,KKK Can any one help me regarding this? (9 Replies)
Discussion started by: jagdishrout
9 Replies

4. Shell Programming and Scripting

Remove particular record from a file

Hi:), I am having 110 records in a file and I want to delete particular record (say 63rd) . Which command can be use to achieve this ? (4 Replies)
Discussion started by: devmns
4 Replies

5. Shell Programming and Scripting

remove newline chars in each record of file

Hi, I have a fixed width file with record length 10. I need to remove multiple newline characters present in each record. EX: af\n72/7\n s\n3\nad\n 2\n\n33r\n In the above file I want to remove new lines in red color(\n) but not (\n) Please provide me a solution. Thanks, Sri (1 Reply)
Discussion started by: srilaxmi
1 Replies

6. UNIX for Advanced & Expert Users

To remove the record from a file

Hi , In need to remove the record marked in red My input is as below : 1|ETG|63121387883|Alternate|Y 2||| 3|79.58|||GBP|| 4|001137001 4|0011372 5|1021701 5|1021901 1|ETG|63121387884|Alternate|Y 2||| 3|79.58|||GBP|| 4|001137001 5|1021702 5|1021802 1|ETG|63128363077|Alternate|Y 2|||... (1 Reply)
Discussion started by: laxmi131
1 Replies

7. Shell Programming and Scripting

How to remove a particular record from a file?

Please tell me the command(s) to remove a particular record from the file and placing the rest of the record in a seperate file. (9 Replies)
Discussion started by: kanu_pathak
9 Replies

8. UNIX for Dummies Questions & Answers

Remove First and Last Record from a file

Hi All, Could some one help me how to delete first & last record from a file Thanks in Advance....... Regards Cherrry (1 Reply)
Discussion started by: ravikuc
1 Replies

9. UNIX for Dummies Questions & Answers

command to remove last record on file

Hi, First time on the forum. I have converted some files using the Unix to DOS command but need to strip off the last record that is generated from this conversion that contains just a ^Z. Is there any command that would accomplish this without having to do stream editing? (4 Replies)
Discussion started by: mheinen
4 Replies

10. Shell Programming and Scripting

remove duplicated xml record in a file under unix

Hi, If i have a file with xml format, i would like to remove duplicated records and save to a new file. Is it possible...to write script to do it? (8 Replies)
Discussion started by: happyv
8 Replies
Login or Register to Ask a Question