Sponsored Content
Top Forums UNIX for Advanced & Expert Users Please Help. Need Help searching for multiple stings in a file and removing them. Post 302085645 by mjs3221 on Friday 18th of August 2006 01:50:45 PM
Old 08-18-2006
Please Help. Need Help searching for multiple stings in a file and removing them.

Please help. Here is my problem. I have 9000 lines in file a and 500,000 lines in file b. For each line in file a I need to search file b and remove that line. I am currently using the grep -v command and loading the output into a new file. However, because of the size of file b this takes an extremely long time to do and I have 50 files similiar to file b. Is there a simpler way to accomplish this. Here is a code snippet of what I have so far.

cat $1 | while read LINE
do
echo $LINE

grep -v $LINE fileName > OutputFile

cp OutputFile fineName

done
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Searching and Removing File Content

Hi, I am trying to search a character in a file and remove it from that file.... My file looks something like this: test1.txt ckj12300_00|123|var1|10.2 ckj00200_12|444|var2|11.2 ckj00200_14|4556|var3|33.5 c00200_00_000|4558|var4|33.5 ckj00200_14|4553|var5|33.5... (7 Replies)
Discussion started by: rkumar28
7 Replies

2. UNIX for Dummies Questions & Answers

removing multiple lines of text in a file

Hi, I'm trying to remove multiple lines of text based off a series of different words and output it to a new file The document contains a ton of data but i want to delete any line that has the following mx1.rr.biz.com or ns2.ri.biz.com i tried using grep -v filename "mx1.rr.biz.com" >... (3 Replies)
Discussion started by: spartan22
3 Replies

3. Shell Programming and Scripting

searching thru or combining multiple lines in a unix file

This is the problem actually: This regex: egrep "low debug.*\".*\"" $dbDir/alarmNotification.log is looking for data between the two quotation marks: ".*\" When I hate data like this: low debug 2009/3/9 8:30:20.47 ICSNotificationAlarm Prodics01ics0003 IC... (0 Replies)
Discussion started by: ndedhia1
0 Replies

4. Shell Programming and Scripting

Perl, searching multiple files and printing returned line to new file

I am trying to find a way to utilise the full potential of my cpu cores and memory on my windows machine. Now, I am quite familiar with grep, however, running a Unix based OS is not an option right now. Unfortunately, the 32 bit grep for windows that I am running, I cannot run multiple... (1 Reply)
Discussion started by: Moloch
1 Replies

5. Shell Programming and Scripting

Searching for multiple patterns in a file

Hi All, I have a file in which i have to search for a pattern from the beginning of the file and if the pattern is found , then i have to perform a reverse search from that line to the beginning of the file to get the first occurrence of another pattern. sample input file hey what are you... (8 Replies)
Discussion started by: Kesavan
8 Replies

6. UNIX for Dummies Questions & Answers

Searching for Multiple texts in a file

Hello guys, I hope anyone can help me with this ... I have a file in which i have around 6000 lines of same format text like 1234567 2345678 3456789 .................... Now what I have to do is that there I have to search these numbers in another file which contains hundreds of... (1 Reply)
Discussion started by: m_usmanayub
1 Replies

7. Shell Programming and Scripting

Help in searching a multiple text in zip file

Hi Gurus, i have 8 zipped files and each file is having more than 100,000 records or more. issue :- i want to search the missing text from each zipped files i have stuck here, the below command works fine if i give the value 10 for the deptno. if i have more than 1 records... (6 Replies)
Discussion started by: SeenuGuddu
6 Replies

8. Shell Programming and Scripting

Removing multiple lines from input file, if multiple lines match a pattern.

GM, I have an issue at work, which requires a simple solution. But, after multiple attempts, I have not been able to hit on the code needed. I am assuming that sed, awk or even perl could do what I need. I have an application that adds extra blank page feeds, for multiple reports, when... (7 Replies)
Discussion started by: jxfish2
7 Replies

9. Shell Programming and Scripting

Script to replace stings in multiple text files

Good Evening Folks - Happy Friday! I have a need to replace a certain string in all .csv files from "0.00" to "#Missing" in my /app/hyp_app/files directory. Does anyone have a script they use regularly that's rather quick in performance? My files are rather large so I'm looking for a... (5 Replies)
Discussion started by: SIMMS7400
5 Replies

10. UNIX for Beginners Questions & Answers

How to generate adler32 stings that convert into hex stings in python 2.7?

I want to generate adler32 stings that converts into hex stings in python 2.7 (1 Reply)
Discussion started by: bigvito19
1 Replies
GIT-GREP(1)							    Git Manual							       GIT-GREP(1)

NAME
git-grep - Print lines matching a pattern SYNOPSIS
git grep [-a | --text] [-I] [-i | --ignore-case] [-w | --word-regexp] [-v | --invert-match] [-h|-H] [--full-name] [-E | --extended-regexp] [-G | --basic-regexp] [-P | --perl-regexp] [-F | --fixed-strings] [-n | --line-number] [-l | --files-with-matches] [-L | --files-without-match] [(-O | --open-files-in-pager) [<pager>]] [-z | --null] [-c | --count] [--all-match] [-q | --quiet] [--max-depth <depth>] [--color[=<when>] | --no-color] [--break] [--heading] [-p | --show-function] [-A <post-context>] [-B <pre-context>] [-C <context>] [-W | --function-context] [-f <file>] [-e] <pattern> [--and|--or|--not|(|)|-e <pattern>...] [ [--[no-]exclude-standard] [--cached | --no-index | --untracked] | <tree>...] [--] [<pathspec>...] DESCRIPTION
Look for specified patterns in the tracked files in the work tree, blobs registered in the index file, or blobs in given tree objects. Patterns are lists of one or more search expressions separated by newline characters. An empty string as search expression matches all lines. CONFIGURATION
grep.lineNumber If set to true, enable -n option by default. grep.patternType Set the default matching behavior. Using a value of basic, extended, fixed, or perl will enable the --basic-regexp, --extended-regexp, --fixed-strings, or --perl-regexp option accordingly, while the value default will return to the default matching behavior. grep.extendedRegexp If set to true, enable --extended-regexp option by default. This option is ignored when the grep.patternType option is set to a value other than default. OPTIONS
--cached Instead of searching tracked files in the working tree, search blobs registered in the index file. --no-index Search files in the current directory that is not managed by Git. --untracked In addition to searching in the tracked files in the working tree, search also in untracked files. --no-exclude-standard Also search in ignored files by not honoring the .gitignore mechanism. Only useful with --untracked. --exclude-standard Do not pay attention to ignored files specified via the .gitignore mechanism. Only useful when searching files in the current directory with --no-index. -a, --text Process binary files as if they were text. -i, --ignore-case Ignore case differences between the patterns and the files. -I Don't match the pattern in binary files. --max-depth <depth> For each <pathspec> given on command line, descend at most <depth> levels of directories. A negative value means no limit. This option is ignored if <pathspec> contains active wildcards. In other words if "a*" matches a directory named "a*", "*" is matched literally so --max-depth is still effective. -w, --word-regexp Match the pattern only at word boundary (either begin at the beginning of a line, or preceded by a non-word character; end at the end of a line or followed by a non-word character). -v, --invert-match Select non-matching lines. -h, -H By default, the command shows the filename for each match. -h option is used to suppress this output. -H is there for completeness and does not do anything except it overrides -h given earlier on the command line. --full-name When run from a subdirectory, the command usually outputs paths relative to the current directory. This option forces paths to be output relative to the project top directory. -E, --extended-regexp, -G, --basic-regexp Use POSIX extended/basic regexp for patterns. Default is to use basic regexp. -P, --perl-regexp Use Perl-compatible regexp for patterns. Requires libpcre to be compiled in. -F, --fixed-strings Use fixed strings for patterns (don't interpret pattern as a regex). -n, --line-number Prefix the line number to matching lines. -l, --files-with-matches, --name-only, -L, --files-without-match Instead of showing every matched line, show only the names of files that contain (or do not contain) matches. For better compatibility with git diff, --name-only is a synonym for --files-with-matches. -O [<pager>], --open-files-in-pager [<pager>] Open the matching files in the pager (not the output of grep). If the pager happens to be "less" or "vi", and the user specified only one pattern, the first file is positioned at the first match automatically. -z, --null Output instead of the character that normally follows a file name. -c, --count Instead of showing every matched line, show the number of lines that match. --color[=<when>] Show colored matches. The value must be always (the default), never, or auto. --no-color Turn off match highlighting, even when the configuration file gives the default to color output. Same as --color=never. --break Print an empty line between matches from different files. --heading Show the filename above the matches in that file instead of at the start of each shown line. -p, --show-function Show the preceding line that contains the function name of the match, unless the matching line is a function name itself. The name is determined in the same way as git diff works out patch hunk headers (see Defining a custom hunk-header in gitattributes(5)). -<num>, -C <num>, --context <num> Show <num> leading and trailing lines, and place a line containing -- between contiguous groups of matches. -A <num>, --after-context <num> Show <num> trailing lines, and place a line containing -- between contiguous groups of matches. -B <num>, --before-context <num> Show <num> leading lines, and place a line containing -- between contiguous groups of matches. -W, --function-context Show the surrounding text from the previous line containing a function name up to the one before the next function name, effectively showing the whole function in which the match was found. -f <file> Read patterns from <file>, one per line. -e The next parameter is the pattern. This option has to be used for patterns starting with - and should be used in scripts passing user input to grep. Multiple patterns are combined by or. --and, --or, --not, ( ... ) Specify how multiple patterns are combined using Boolean expressions. --or is the default operator. --and has higher precedence than --or. -e has to be used for all patterns. --all-match When giving multiple pattern expressions combined with --or, this flag is specified to limit the match to files that have lines to match all of them. -q, --quiet Do not output matched lines; instead, exit with status 0 when there is a match and with non-zero status when there isn't. <tree>... Instead of searching tracked files in the working tree, search blobs in the given trees. -- Signals the end of options; the rest of the parameters are <pathspec> limiters. <pathspec>... If given, limit the search to paths matching at least one pattern. Both leading paths match and glob(7) patterns are supported. EXAMPLES
git grep 'time_t' -- '*.[ch]' Looks for time_t in all tracked .c and .h files in the working directory and its subdirectories. git grep -e '#define' --and ( -e MAX_PATH -e PATH_MAX ) Looks for a line that has #define and either MAX_PATH or PATH_MAX. git grep --all-match -e NODE -e Unexpected Looks for a line that has NODE or Unexpected in files that have lines that match both. GIT
Part of the git(1) suite Git 1.8.3.1 06/10/2014 GIT-GREP(1)
All times are GMT -4. The time now is 06:13 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy