Sponsored Content
Top Forums UNIX for Advanced & Expert Users Please Help. Need Help searching for multiple stings in a file and removing them. Post 302085645 by mjs3221 on Friday 18th of August 2006 01:50:45 PM
Old 08-18-2006
Please Help. Need Help searching for multiple stings in a file and removing them.

Please help. Here is my problem. I have 9000 lines in file a and 500,000 lines in file b. For each line in file a I need to search file b and remove that line. I am currently using the grep -v command and loading the output into a new file. However, because of the size of file b this takes an extremely long time to do and I have 50 files similiar to file b. Is there a simpler way to accomplish this. Here is a code snippet of what I have so far.

cat $1 | while read LINE
do
echo $LINE

grep -v $LINE fileName > OutputFile

cp OutputFile fineName

done
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Searching and Removing File Content

Hi, I am trying to search a character in a file and remove it from that file.... My file looks something like this: test1.txt ckj12300_00|123|var1|10.2 ckj00200_12|444|var2|11.2 ckj00200_14|4556|var3|33.5 c00200_00_000|4558|var4|33.5 ckj00200_14|4553|var5|33.5... (7 Replies)
Discussion started by: rkumar28
7 Replies

2. UNIX for Dummies Questions & Answers

removing multiple lines of text in a file

Hi, I'm trying to remove multiple lines of text based off a series of different words and output it to a new file The document contains a ton of data but i want to delete any line that has the following mx1.rr.biz.com or ns2.ri.biz.com i tried using grep -v filename "mx1.rr.biz.com" >... (3 Replies)
Discussion started by: spartan22
3 Replies

3. Shell Programming and Scripting

searching thru or combining multiple lines in a unix file

This is the problem actually: This regex: egrep "low debug.*\".*\"" $dbDir/alarmNotification.log is looking for data between the two quotation marks: ".*\" When I hate data like this: low debug 2009/3/9 8:30:20.47 ICSNotificationAlarm Prodics01ics0003 IC... (0 Replies)
Discussion started by: ndedhia1
0 Replies

4. Shell Programming and Scripting

Perl, searching multiple files and printing returned line to new file

I am trying to find a way to utilise the full potential of my cpu cores and memory on my windows machine. Now, I am quite familiar with grep, however, running a Unix based OS is not an option right now. Unfortunately, the 32 bit grep for windows that I am running, I cannot run multiple... (1 Reply)
Discussion started by: Moloch
1 Replies

5. Shell Programming and Scripting

Searching for multiple patterns in a file

Hi All, I have a file in which i have to search for a pattern from the beginning of the file and if the pattern is found , then i have to perform a reverse search from that line to the beginning of the file to get the first occurrence of another pattern. sample input file hey what are you... (8 Replies)
Discussion started by: Kesavan
8 Replies

6. UNIX for Dummies Questions & Answers

Searching for Multiple texts in a file

Hello guys, I hope anyone can help me with this ... I have a file in which i have around 6000 lines of same format text like 1234567 2345678 3456789 .................... Now what I have to do is that there I have to search these numbers in another file which contains hundreds of... (1 Reply)
Discussion started by: m_usmanayub
1 Replies

7. Shell Programming and Scripting

Help in searching a multiple text in zip file

Hi Gurus, i have 8 zipped files and each file is having more than 100,000 records or more. issue :- i want to search the missing text from each zipped files i have stuck here, the below command works fine if i give the value 10 for the deptno. if i have more than 1 records... (6 Replies)
Discussion started by: SeenuGuddu
6 Replies

8. Shell Programming and Scripting

Removing multiple lines from input file, if multiple lines match a pattern.

GM, I have an issue at work, which requires a simple solution. But, after multiple attempts, I have not been able to hit on the code needed. I am assuming that sed, awk or even perl could do what I need. I have an application that adds extra blank page feeds, for multiple reports, when... (7 Replies)
Discussion started by: jxfish2
7 Replies

9. Shell Programming and Scripting

Script to replace stings in multiple text files

Good Evening Folks - Happy Friday! I have a need to replace a certain string in all .csv files from "0.00" to "#Missing" in my /app/hyp_app/files directory. Does anyone have a script they use regularly that's rather quick in performance? My files are rather large so I'm looking for a... (5 Replies)
Discussion started by: SIMMS7400
5 Replies

10. UNIX for Beginners Questions & Answers

How to generate adler32 stings that convert into hex stings in python 2.7?

I want to generate adler32 stings that converts into hex stings in python 2.7 (1 Reply)
Discussion started by: bigvito19
1 Replies
BM(PUBLIC)																BM(PUBLIC)

NAME
bm - search a file for a string SYNOPSIS
/usr/public/bm [ option ] ... [ strings ] [ file ] DESCRIPTION
Bm searches the input files (standard input default) for lines matching a string. Normally, each line found is copied to the standard out- put. It is blindingly fast. Bm strings are fixed sequences of characters: there are no wildcards, repetitions, or other features of regu- lar expressions. Bm is also case sensitive. The following options are recognized. -x (Exact) only lines matched in their entirety are printed -l The names of files with matching lines are listed (once) separated by newlines. -c Only a count of the number of matches is printed -e string The string is the next argument after the -e flag. This allows strings beginning with '-'. -h No filenames are printed, even if multiple files are searched. -n Each line is preceded by the number of characters from the beginning of the file to the match. -s Silent mode. Nothing is printed (except error messages). This is useful for checking the error status. -f file The string list is taken from the file. Unless the -h option is specified the file name is shown if there is more than one input file. Care should be taken when using the charac- ters $ * [ ^ | ( ) and in the strings (listed on the command line) as they are also meaningful to the Shell. It is safest to enclose the entire expression argument in single quotes ' '. Bm searches for lines that contain one of the (newline-separated) strings, using the Boyer-Moore algorithm. It is far superior in terms of speed to the grep (egrep, fgrep) family of pattern matchers for fixed-pattern searching, and its speed increases with pattern length. SEE ALSO
grep(1) DIAGNOSTICS
Exit status is 0 if any matches are found, 1 if none, 2 for syntax errors or inaccessible files. AUTHOR
Peter Bain (pdbain@wateng), with modifications suggested by John Gilmore BUGS
Only 100 patterns are allowed. Patterns may not contain newlines. If a line (delimited by newlines, and the beginning and end of the file) is longer than 8000 charcters (e.g. in a core dump), it will not be completely printed. If multiple patterns are specified, the order of the ouput lines is not necessarily the same as the order of the input lines. A line will be printed once for each different string on that line. The algorithm cannot count lines. The -n and -c work differently from fgrep. The -v, -i, and -b are not available. 4th Berkeley Distribution 8 July 1985 BM(PUBLIC)
All times are GMT -4. The time now is 12:41 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy