deleting lines from a delimited files based on a 2nd file


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers deleting lines from a delimited files based on a 2nd file
# 1  
Old 03-30-2009
deleting lines from a delimited files based on a 2nd file

I need a little help...

I have a file with 3 fields,

Time/Date, IP address, UniqueID

I have a 2nd file of UniqueIDs.

I want to either (which ever is easier):
1. delete entries in file 1 that have a UniqueID in file 2
2. create a new file with the fields from File 1, excluding the UniqueIDs in file 2.

Both files are in .CSV format.

Thanks in advance for any assistance.

Peter
# 2  
Old 03-30-2009
Tools without more detail, here is a simple approach

Code:
> cat file4
03/29/2008 1.2.3.4 joeyg
04/01/2009 2.3.3.2 holly
06/01/2007 3.5.6.7 sandy

> cat file4x
holly
steve
ricky

> egrep -vf file4x file4
03/29/2008 1.2.3.4 joeyg
06/01/2007 3.5.6.7 sandy

this could be re-directed to a file by adding
>file5 to the end of the command
# 3  
Old 03-30-2009
Thanks, however that only removes the first entry in File4x (referencing your example). I have a couple hundred different UniqueIDs that i am trying to remove from the File4 file.

Any thoughts on how to get it to loop through and check for all the values included in the File4x file?
# 4  
Old 03-30-2009
Hammer & Screwdriver That will remove all matches; not just the first one

Code:
> cat file4
03/29/2008 1.2.3.4 joeyg
04/01/2009 2.3.3.2 holly
06/01/2007 3.5.6.7 sandy
12/08/1998 2.12.3.45 Matthew
04/02/1994 12.4.2.9 Kerrin
12/25/2008 16.16.2.2 Penelope

> cat file4x
holly
steve
ricky
Penelope
sandy
> egrep -vf file4x file4
03/29/2008 1.2.3.4 joeyg
12/08/1998 2.12.3.45 Matthew
04/02/1994 12.4.2.9 Kerrin

# 5  
Old 03-30-2009
Got it thanks!
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with a deleting lines based on a pattern

I have a header-detail file that goes like this: SHP00288820131021110921 ORDER0156605920131021110921INMMMMFN DETAIL0004 4C2Z 10769 AAFC 0000009600000094 4C2Z 10769 AAFC 0000672107 OIL DETAIL0002 ER3Z 14300 E 0000001300000012 ER3Z 14300 E 0000672107 OIL... (3 Replies)
Discussion started by: rbaggio666
3 Replies

2. Shell Programming and Scripting

Deleting lines based on a condition for a group of files

hi i have a set of similar files. i want to delete lines until certain pattern appears in those files. for a single file the following command can be used but i want to do it for all the files at a time since the number is in thousands. awk '/PATTERN/{i++}i' file (6 Replies)
Discussion started by: anurupa777
6 Replies

3. Shell Programming and Scripting

Generate files from one file based on lines

Hi Friends, I have a file1 file1.txt 1ABC 13478 aqjerh 473 343 2hej 478 5775 24578 23892 3fhd fg 847 brjkb f99345 487 4eh ehjk 84 47589 8947 234 5784 487 738 52895 8975 6 57489 eghe9 4575 859479 7fnbd 4y5 4iuy 458 h irh 8fjdg 74 7845 8475 5789 94yr 48yr 4hr erhj reh... (3 Replies)
Discussion started by: i150371485
3 Replies

4. UNIX for Dummies Questions & Answers

[Solved] deleting pattern based lines in sed

HI, My input file contains below data: DFHDR 12345110 1,200 2,-100 1,100 2,123 12345110 1,300 2,200 DFTLR In the above data, the first line and last lines should be remove as well as the lines in which contains 110 as position(6,7,8 position) should also be removed, How we... (0 Replies)
Discussion started by: pandeesh
0 Replies

5. UNIX for Dummies Questions & Answers

Deleting lines that contain a specific string from a space delimited text file?

Hi, I have a space delimited text file that looks like the following: 250 rs10000056 0.04 0.0888 4 189321617 250 rs10000062 0.05 0.0435 4 5254744 250 rs10000064 0.02 0.2403 4 127809621 250 rs10000068 0.01 NA 250 rs1000007 0.00 0.9531 2 237752054 250 rs10000081 0.03 0.1400 4 17348363... (5 Replies)
Discussion started by: evelibertine
5 Replies

6. UNIX for Dummies Questions & Answers

Deleting columns from a tab delimited text file?

I have a tab limited text file with 10000+ columns. I want to delete columns 6 through 23, how do I go about doing that? Thanks! (1 Reply)
Discussion started by: evelibertine
1 Replies

7. UNIX for Dummies Questions & Answers

Deleting columns from a space delimited text file

I have a space delimited text file with 1,000,000+ columns and 100 rows. I want to delete columns 2 through 5 (2 and 5) included from the text file. How do I do that? Thanks. (3 Replies)
Discussion started by: evelibertine
3 Replies

8. Shell Programming and Scripting

Searching patterns in 1 file and deleting all lines with those patterns in 2nd file

Hi Gurus, I have a file say for ex. file1 which has 3500 lines in it which are different account numbers and another file (file2) which has 230000 lines in it. I want to read all the lines in file1 and delete all those lines from file2 which has that same pattern as in file1. I am not quite... (4 Replies)
Discussion started by: toms
4 Replies

9. Shell Programming and Scripting

Finding duplicate lines and deleting folders based on them

Hi, I have research data, which is organized to 100 folders numbered 00-99. I have many sets of 100 folders, for different values of initial parameters. For some reason, the computer that ran the program to gather the data, didn't always create a unique seed for each folder. I anticipated that... (1 Reply)
Discussion started by: Jopi
1 Replies

10. UNIX for Dummies Questions & Answers

deleting files based on file name and modified time

Hi, I have some log files created in the following fashion Ex: file name modified date 1) s.log1 01-jan-08 2) s.log2 02-jan-08 3) s.log3 03-jan-08 4) s.log4 04-jan-08 Now I want to have the latest 2 logs and delete the others. Can you tell me the one liner /... (1 Reply)
Discussion started by: ammu
1 Replies
Login or Register to Ask a Question