Sponsored Content
Top Forums Shell Programming and Scripting Remove lines from one file that exist in another file Post 302700835 by alister on Friday 14th of September 2012 09:41:02 AM
Old 09-14-2012
Quote:
Originally Posted by Ygor
Code:
grep -f file1 file2

If the goal is to match identical lines, I would recommend:
Code:
grep -xFf file1 file2

Using fixed string, whole-line matching will prevent false positives (fixed substrings or regexp wildcard matches), false negatives (some regexps don't match themselves, e.g. [a]), and syntax errors from lines that aren't valid regular expressions.

Regards,
Alister

Last edited by alister; 09-14-2012 at 10:47 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

If doc file exist remove

I need help running a script. I have the script looking into a folder and converting .doc files to .odt. The script works fine except that I want it to only run when .doc files are present. If I can do this then I can put .xls files and .ppt files in the folder and convert them when they are... (2 Replies)
Discussion started by: handband2
2 Replies

2. Shell Programming and Scripting

Removing Lines if value exist in first file

I tried a few ways to resolve this using a bash script w/ a loop, no luck. File1: roughly 6,000 account numbers such as: 1111 1512 1113 123 I also have a dozen or so csv files, w/ the account number in the 4th field. What I would like to do is remove all lines if the... (19 Replies)
Discussion started by: svn
19 Replies

3. Shell Programming and Scripting

Deleting lines of a file if they exist in another file

I have a reference file that needs to remain static and another file that may or may not have duplicate rows that match the reference file. I need help with a command that will delete any duplicate rows from the second file while leaving reference file intact For example reference file would... (4 Replies)
Discussion started by: bjdamon
4 Replies

4. Shell Programming and Scripting

Remove duplicate lines from first file comparing second file

Hi, I have two files with below data:: file1:- 123|aaa|ppp 445|fff|yyy 999|ttt|jjj 555|hhh|hhh file2:- 445|fff|yyy 555|hhh|hhh The records present in file1, not present in file 2 should be writtent to the out put file. output:- 123|aaa|ppp 999|ttt|jjj Is there any one line... (3 Replies)
Discussion started by: gani_85
3 Replies

5. Shell Programming and Scripting

How to remove blank lines in a file and save the file with same name?

I have a text file which has blank lines. I want them to be removed before upload it to DB using SQL *Loader. Below is the command line, i use to remove blank lines. sed '/^ *$/d' /loc/test.txt If i use the below command to replace the file after removing the blank lines, it replace the... (6 Replies)
Discussion started by: vel4ever
6 Replies

6. Shell Programming and Scripting

Code to remove files when corresponding file doesnt exist isnt working.

I am trying to add some code to the begging of a script so that it will remove all the .transcript files, when their is no coressponding .wav file. But it doesnt work. This is the code I have added: for transcriptfile in `$voicemaildir/*.transcript`; do wavfile=`echo $transcriptfile | cut -d'.'... (2 Replies)
Discussion started by: ghurty
2 Replies

7. UNIX for Dummies Questions & Answers

Want to remove all lines but not latest 50 lines from a file

Hi, I have a huge file which has Lacs of lines. File system got full. I want your guys help to suggest me a solution so that I can remove all lines from that file but not last 50,000 lines. I want solution which can remove lines from existing file so that I can have some space left with. (28 Replies)
Discussion started by: prashant2507198
28 Replies

8. Shell Programming and Scripting

ksh Loop through file one, remove lines from file two

Good Afternoon, I start with a file named biglist.txt. I have another file smallerlist. txt I want to remove the lines from smallerlist.txt from biglist.txt and leave those lines that do not reside in smallerlist.txt. Thanks !! (2 Replies)
Discussion started by: popeye
2 Replies

9. Shell Programming and Scripting

awk remove/grab lines from file with pattern from other file

Sorry for the weird title but i have the following problem. We have several files which have between 10000 and about 500000 lines in them. From these files we want to remove lines which contain a pattern which is located in another file (around 20000 lines, all EAN codes). We also want to get... (28 Replies)
Discussion started by: SDohmen
28 Replies

10. UNIX for Beginners Questions & Answers

Remove lines from File.A based on criteria in File.B

Hello, I have two files of the following form. I would like to remove from File.A where the first three colum matches values in File.B to give the output in File.C File.A 121 54321 PQR CAT 122 765431 ABC DOG 124 98765 ZXY TIGER 125 86432 GEF LION File.B 122 765431 ABC 125 86432 GEF... (4 Replies)
Discussion started by: Gussifinknottle
4 Replies
GREP(1) 						      General Commands Manual							   GREP(1)

NAME
grep, egrep, fgrep - search a file for a pattern SYNOPSIS
grep [ option ] ... expression [ file ] ... egrep [ option ] ... [ expression ] [ file ] ... fgrep [ option ] ... [ strings ] [ file ] DESCRIPTION
Commands of the grep family search the input files (standard input default) for lines matching a pattern. Normally, each line found is copied to the standard output. Grep patterns are limited regular expressions in the style of ex(1); it uses a compact nondeterministic algorithm. Egrep patterns are full regular expressions; it uses a fast deterministic algorithm that sometimes needs exponential space. Fgrep patterns are fixed strings; it is fast and compact. The following options are recognized. -v All lines but those matching are printed. -x (Exact) only lines matched in their entirety are printed (fgrep only). -c Only a count of matching lines is printed. -l The names of files with matching lines are listed (once) separated by newlines. -n Each line is preceded by its relative line number in the file. -b Each line is preceded by the block number on which it was found. This is sometimes useful in locating disk block numbers by con- text. -i The case of letters is ignored in making comparisons -- that is, upper and lower case are considered identical. This applies to grep and fgrep only. -s Silent mode. Nothing is printed (except error messages). This is useful for checking the error status. -w The expression is searched for as a word (as if surrounded by `<' and `>', see ex(1).) (grep only) -e expression Same as a simple expression argument, but useful when the expression begins with a -. -f file The regular expression (egrep) or string list (fgrep) is taken from the file. In all cases the file name is shown if there is more than one input file. Care should be taken when using the characters $ * [ ^ | ( ) and in the expression as they are also meaningful to the Shell. It is safest to enclose the entire expression argument in single quotes ' '. Fgrep searches for lines that contain one of the (newline-separated) strings. Egrep accepts extended regular expressions. In the following description `character' excludes newline: A followed by a single character other than newline matches that character. The character ^ matches the beginning of a line. The character $ matches the end of a line. A . (period) matches any character. A single character not otherwise endowed with special meaning matches that character. A string enclosed in brackets [] matches any single character from the string. Ranges of ASCII character codes may be abbreviated as in `a-z0-9'. A ] may occur only as the first character of the string. A literal - must be placed where it can't be mistaken as a range indicator. A regular expression followed by an * (asterisk) matches a sequence of 0 or more matches of the regular expression. A regular expression followed by a + (plus) matches a sequence of 1 or more matches of the regular expression. A regular expression followed by a ? (question mark) matches a sequence of 0 or 1 matches of the regular expression. Two regular expressions concatenated match a match of the first followed by a match of the second. Two regular expressions separated by | or newline match either a match for the first or a match for the second. A regular expression enclosed in parentheses matches a match for the regular expression. The order of precedence of operators at the same parenthesis level is [] then *+? then concatenation then | and newline. Ideally there should be only one grep, but we don't know a single algorithm that spans a wide enough range of space-time tradeoffs. SEE ALSO
ex(1), sed(1), sh(1) DIAGNOSTICS
Exit status is 0 if any matches are found, 1 if none, 2 for syntax errors or inaccessible files. BUGS
Lines are limited to 256 characters; longer lines are truncated. 4th Berkeley Distribution April 29, 1985 GREP(1)
All times are GMT -4. The time now is 11:18 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy