I don't understand the construction set | grep $value /tmp/date.out -- as far as I can tell, the output from set will not be used for anything.
Also, the conditional is a Useless Use of Test, and it will print to stdout any matches; I imagine that's undesirable. The following avoids those problems.
Still, if you had your list of PDF files in another file, one PDF per line, it could be as simple as
The use of fgrep -x requires an exact match (not a regex match; you know that dot in a regex matches any character, for example) spanning the whole line (that's the -x). The -v causes only lines in /tmp/dave.out which are not anywhere in pdfs.txt to be printed.
I have a file that is 20 - 80+ MB in size that is a certain type of log file.
It logs one of our processes and this process is multi-threaded. Therefore the log file is kind of a mess. Here's an example:
The logfile looks like: "DATE TIME - THREAD ID - Details", and a new file is created... (4 Replies)
I'm trying to make a simple search script but cannot get it right. The script should search for keywords inside files. Then return the file paths in a variable. (Each file path separated with \n).
#!/bin/bash
SEARCHQUERY="searchword1 searchword2 searchword3";
for WORD in $SEARCHQUERY
do
... (6 Replies)
Hi,
I have a text file with data in that I wish to extract, assign to a variable and process through a loop.
Kind of the process that I am after:
1: Grep the text file for the values.
Currently using:
cat /root/test.txt | grep TESTING= | awk -F"=" '{ a = $2 } {print a}' | sort -u
... (0 Replies)
Hi all,
I have problem with searching hundreds of CSV files, the problem is that search is lasting too long (over 5min).
Csv files are "," delimited, and have 30 fields each line, but I always grep same 4 fields - so is there a way to grep just those 4 fields to speed-up search.
Example:... (11 Replies)
Hello,
I am using sed in a for loop to replace text in a 100MB file. I have about 55,000 entries to convert in a csv file with two entries per line. The following script works to search file.txt for the first field from conversion.csv and then replace it with the second field. While it works fine,... (15 Replies)
This is my first experience writing unix script. I've created the following script. It does what I want it to do, but I need it to be a lot faster. Is there any way to speed it up?
cat 'Tax_Provision_Sample.dat' | sort | while read p; do fn=`echo $p|cut -d~ -f2,4,3,8,9`; echo $p >> "$fn.txt";... (20 Replies)
Dear all,
Please help with the following.
I have a file, let's call it data.txt, that has 3 columns and approx 700,000 lines, and looks like this:
rs1234 A C
rs1236 T G
rs2345 G T
Please use code tags as required by forum rules!
I have a second file, called reference.txt,... (1 Reply)
HI Guys hoping some one can help
I have two files on both containing uk phone numbers
master is a file which has been collated over a few years ad currently contains around 4 million numbers
new is a file which also contains 4 million number i need to split new nto two separate files... (4 Replies)
Hi,
I've written a ksh script that read a file and parse/filter/format each line. The script runs as expected but it runs for 24+ hours for a file that has 2million lines. And sometimes, the input file has 10million lines which means it can be running for more than 2 days and still not finish.... (9 Replies)
Hello experts,
we have input files with 700K lines each (one generated for every hour). and we need to convert them as below and move them to another directory once.
Sample INPUT:-
# cat test1
1559205600000,8474,NormalizedPortInfo,PctDiscards,0.0,Interface,BG-CTA-AX1.test.com,Vl111... (7 Replies)
Discussion started by: prvnrk
7 Replies
LEARN ABOUT BSD
fgrep
GREP(1) General Commands Manual GREP(1)NAME
grep, egrep, fgrep - search a file for a pattern
SYNOPSIS
grep [ option ] ... expression [ file ] ...
egrep [ option ] ... [ expression ] [ file ] ...
fgrep [ option ] ... [ strings ] [ file ]
DESCRIPTION
Commands of the grep family search the input files (standard input default) for lines matching a pattern. Normally, each line found is
copied to the standard output. Grep patterns are limited regular expressions in the style of ex(1); it uses a compact nondeterministic
algorithm. Egrep patterns are full regular expressions; it uses a fast deterministic algorithm that sometimes needs exponential space.
Fgrep patterns are fixed strings; it is fast and compact. The following options are recognized.
-v All lines but those matching are printed.
-x (Exact) only lines matched in their entirety are printed (fgrep only).
-c Only a count of matching lines is printed.
-l The names of files with matching lines are listed (once) separated by newlines.
-n Each line is preceded by its relative line number in the file.
-b Each line is preceded by the block number on which it was found. This is sometimes useful in locating disk block numbers by con-
text.
-i The case of letters is ignored in making comparisons -- that is, upper and lower case are considered identical. This applies to
grep and fgrep only.
-s Silent mode. Nothing is printed (except error messages). This is useful for checking the error status.
-w The expression is searched for as a word (as if surrounded by `<' and `>', see ex(1).) (grep only)
-e expression
Same as a simple expression argument, but useful when the expression begins with a -.
-f file
The regular expression (egrep) or string list (fgrep) is taken from the file.
In all cases the file name is shown if there is more than one input file. Care should be taken when using the characters $ * [ ^ | ( ) and
in the expression as they are also meaningful to the Shell. It is safest to enclose the entire expression argument in single quotes ' '.
Fgrep searches for lines that contain one of the (newline-separated) strings.
Egrep accepts extended regular expressions. In the following description `character' excludes newline:
A followed by a single character other than newline matches that character.
The character ^ matches the beginning of a line.
The character $ matches the end of a line.
A . (period) matches any character.
A single character not otherwise endowed with special meaning matches that character.
A string enclosed in brackets [] matches any single character from the string. Ranges of ASCII character codes may be abbreviated
as in `a-z0-9'. A ] may occur only as the first character of the string. A literal - must be placed where it can't be mistaken as
a range indicator.
A regular expression followed by an * (asterisk) matches a sequence of 0 or more matches of the regular expression. A regular
expression followed by a + (plus) matches a sequence of 1 or more matches of the regular expression. A regular expression followed
by a ? (question mark) matches a sequence of 0 or 1 matches of the regular expression.
Two regular expressions concatenated match a match of the first followed by a match of the second.
Two regular expressions separated by | or newline match either a match for the first or a match for the second.
A regular expression enclosed in parentheses matches a match for the regular expression.
The order of precedence of operators at the same parenthesis level is [] then *+? then concatenation then | and newline.
Ideally there should be only one grep, but we don't know a single algorithm that spans a wide enough range of space-time tradeoffs.
SEE ALSO ex(1), sed(1), sh(1)DIAGNOSTICS
Exit status is 0 if any matches are found, 1 if none, 2 for syntax errors or inaccessible files.
BUGS
Lines are limited to 256 characters; longer lines are truncated.
4th Berkeley Distribution April 29, 1985 GREP(1)