03-24-2009
Remove lines based on contents of another file
So, this issue is driving me nuts! I was hoping to get a lending hand here...
I have 2 files:
file1.txt contains:
this is example1
this is example2
this is example3
this is example4
this is example5
file2.txt contains:
example3
example5
Basically, I need a script or command to generate a new file which contains only the lines that DON'T exist on file2.txt, ideally, the resulting file would look like this:
fileX.txt:
this is example1
this is example2
this is example4
I really appreciate your help, I can't seem to figure this one out..
Thanks a lot!
Dave
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hello,
I have a file with tab delimited columns like:
File1
A 2 C R
F 4 D Q
C 9 A B
......
I want to grep out the lines in a second file, File2, corresponding to each line in File1
Can I do this:
while read a b c d
do
grep '$a\t$b\t$c\t$d' File2 >>... (2 Replies)
Discussion started by: Gussifinknottle
2 Replies
2. Shell Programming and Scripting
Hi All,
I just need a quick fix here. I need to delete all lines containing "." in the 6th column.
Input:
1 1055498 . G T 5.46 .
1 1902377 . C T 7.80 .
1 1031540 . A G 34.01 PASS
1 ... (2 Replies)
Discussion started by: Hkins552
2 Replies
3. Shell Programming and Scripting
Hi, I have multiple large files which consist of the below format:
I am trying to write an awk or sed script to remove all occurrences of the 00 record except the first and remove all of the 80 records except the last one.
Any help would be greatly appreciated. (10 Replies)
Discussion started by: nwalsh88
10 Replies
4. UNIX for Dummies Questions & Answers
Gurus,
I am relatively new to Unix scripting and am struck with a problem in my script. I have positional input file which has a FLAG indicator in at position 11 in every record of the file.
If the Flag has value =Y, then the record from the input needs to be written to a new file.However if... (3 Replies)
Discussion started by: gsam
3 Replies
5. Shell Programming and Scripting
I have two files, a keepout.txt and a database.csv. They're unsorted, but could be sorted.
keepout:
user1
buser3
anuser19
notheruser27
database:
user1,2343,"information about",field,blah,34
user2,4231,"mo info",etc,stuff,43
notheruser27,4344,"hiya",thing,more thing,423... (4 Replies)
Discussion started by: esoffron
4 Replies
6. Shell Programming and Scripting
Good day all.
Using basic UNIX/Linux tools, how would you delete a line based on a character found in column 1?
For example, if the CITY name contains an 'a' or 'A', delete the line:
New York City; New York
Los Angeles; California
Chicago; Illinois
Houston; Texas
Philadelphia;... (3 Replies)
Discussion started by: BRH
3 Replies
7. Shell Programming and Scripting
Dear community,
I have to remove duplicate lines from a file contains a very big ammount of rows (milions?) based on 1st and 3rd columns
The data are like this:
Region 23/11/2014 09:11:36 41752
Medio 23/11/2014 03:11:38 4132
Info 23/11/2014 05:11:09 4323... (2 Replies)
Discussion started by: Lord Spectre
2 Replies
8. Shell Programming and Scripting
I am trying to use the two files shown below to either remove or rename contents in one of those files. If in file1.txt $5 matches $5 of file2.txt and the value in $1 of file1.txt is not "No Match" then that value is substituted for all values in $5 and $1 of file2.txt. If however in $1 ... (5 Replies)
Discussion started by: cmccabe
5 Replies
9. UNIX for Beginners Questions & Answers
Hi,
I'd be grateful for your help with the following. I have a file (file.txt) with 10 columns and about half a million lines, which in simplified form looks like this:
ID Col1 Col2 Col3....
a 4 2 8
b 5 6 1
c 8 4 1
d... (4 Replies)
Discussion started by: aberg
4 Replies
10. UNIX for Beginners Questions & Answers
Hello,
I have two files of the following form. I would like to remove from File.A where the first three colum matches values in File.B to give the output in File.C
File.A
121 54321 PQR CAT
122 765431 ABC DOG
124 98765 ZXY TIGER
125 86432 GEF LION
File.B
122 765431 ABC
125 86432 GEF... (4 Replies)
Discussion started by: Gussifinknottle
4 Replies
DIFF3(1) General Commands Manual DIFF3(1)
NAME
diff3 - 3-way differential file comparison
SYNOPSIS
diff3 [ -exEX3 ] file1 file2 file3
DESCRIPTION
Diff3 compares three versions of a file, and publishes disagreeing ranges of text flagged with these codes:
==== all three files differ
====1 file1 is different
====2 file2 is different
====3 file3 is different
The type of change suffered in converting a given range of a given file to some other is indicated in one of these ways:
f : n1 a Text is to be appended after line number n1 in file f, where f = 1, 2, or 3.
f : n1 , n2 c Text is to be changed in the range line n1 to line n2. If n1 = n2, the range may be abbreviated to n1.
The original contents of the range follows immediately after a c indication. When the contents of two files are identical, the contents of
the lower-numbered file is suppressed.
Under the -e option, diff3 publishes a script for the editor ed that will incorporate into file1 all changes between file2 and file3, i.e.
the changes that normally would be flagged ==== and ====3. Option -x (-3) produces a script to incorporate only changes flagged ====
(====3). The following command will apply the resulting script to `file1'.
(cat script; echo '1,$p') | ed - file1
The -E and -X are similar to -e and -x, respectively, but treat overlapping changes (i.e., changes that would be flagged with ==== in the
normal listing) differently. The overlapping lines from both files will be inserted by the edit script, bracketed by "<<<<<<" and ">>>>>>"
lines.
For example, suppose lines 7-8 are changed in both file1 and file2. Applying the edit script generated by the command
"diff3 -E file1 file2 file3"
to file1 results in the file:
lines 1-6
of file1
<<<<<<< file1
lines 7-8
of file1
=======
lines 7-8
of file3
>>>>>>> file3
rest of file1
The -E option is used by RCS merge(1) to insure that overlapping changes in the merged files are preserved and brought to someone's atten-
tion.
FILES
/tmp/d3?????
/usr/libexec/diff3
SEE ALSO
diff(1)
BUGS
Text lines that consist of a single `.' will defeat -e.
7th Edition October 21, 1996 DIFF3(1)