If all of the files are in a single directory (not spread out among subdirectories), you could try the following first:
or, if you just want the names of files containing that string (with a case insensitive match):
These might file with "argument list too long" errors depending on how many thousands of files you have, but it is easy to try it. And, if it works, it will hundreds or thousands of times faster that performing a grep on each file individually. If they won't work, you'll know quickly.
Note that I added a -F option to grep. Searching for fixed strings (even if you're doing a case-insensitive match), is faster than going through the extra complexity of matching regular expressions.
If all of the log entries use the same case, dropping the -i option will also make it run faster.
This User Gave Thanks to Don Cragun For This Post:
Dear all,
I am reading a file that has 1 column. While reading I must find the line references from the another file. The following shell doesn't works.
Please help
#!/bin/bash
while read filename; do
grep ${filename} fs_full.dat >> unprocfull.dat;
done < unproc.dat
But when... (2 Replies)
Hello! I think this should be an easy solution. I have a large file with many fields of data. The first field has a unique identifier (a subject number) for every record for a chunk of data. Something like this:
There were ten experimental conditions (ec), but the ec is identified by only... (11 Replies)
I have inherited the UNIX scripting duties for our team and have only been at this for a few months so I apologize for what may be a simple question.
We have a script we are reworking. Part of this script takes a parameter and reads a csv file. When it finds a matching line, the cut command is... (4 Replies)
Hi all,
I would like to ask whether in Unix shell/perl have any functions or command to allow grep/cat/read a file inside compressed .tgz without extract it?
I know we can tar tvf a compressed tgz but this only allow we read the path/filename contained inside the tarball. If we want to read... (3 Replies)
I have two files
file1.txt
angie
mary
susan
file2.txt
angie blond
mary brunnet
susan red
christine blackI want to get this output
angie blond
mary brunnet
susan redI write
grep --file=file1.txt file2.txtand i get no results
i also wrote
cat file1.txt|while read line... (19 Replies)
Hi Guys,
I have a small script which greps for the username reading from stdinput.
./file.sh pattern
pattern=$1
grep "blah blah.*$pattern" /home/user/log.txt
Instead of typing the pattern everytime i want to read the pattern from a file inside the shell script and execute the... (5 Replies)
Hi,
I'm trying to read a fille into a loop and grep for a string and print the last field of the string in the second file. Then redirect the output to another file but keeping the output in the same order as the original file.
I've tried using the following but the ouput from this does not... (3 Replies)
Hi,
I have a number of files containing the information below.
"""""
Fundallinfo
6.3950 14.9715 14.0482
"""""
I would like to grep for Fundallinfo and use it to read the next line? I ideally would like to read the three numbers that follow in the next line and... (2 Replies)
Hi All,
I am having a job and I need to send email when the job is running. On any other case (success,fail) I don't needed to send email. I check with BMC they told they dont have that in the version I am using.
So I created a dependent job and grepped for the status and sent email. My... (1 Reply)
I have a very big input file <inputFile1.txt> which has list of mobile no
inputFile1.txt
3434343
3434323
0970978
85233
... around 1 million records
i have another file as inputFile2.txt which has some log detail big file
inputFile2.txt
afjhjdhfkjdhfkd df h8983 3434343 | 3483 | myout1 |... (3 Replies)
Discussion started by: reldb
3 Replies
LEARN ABOUT SUSE
unbuffer
UNBUFFER(1) General Commands Manual UNBUFFER(1)NAME
unbuffer - unbuffer output
SYNOPSIS
unbuffer program [ args ]
INTRODUCTION
unbuffer disables the output buffering that occurs when program output is redirected from non-interactive programs. For example, suppose
you are watching the output from a fifo by running it through od and then more.
od -c /tmp/fifo | more
You will not see anything until a full page of output has been produced.
You can disable this automatic buffering as follows:
unbuffer od -c /tmp/fifo | more
Normally, unbuffer does not read from stdin. This simplifies use of unbuffer in some situations. To use unbuffer in a pipeline, use the
-p flag. Example:
process1 | unbuffer -p process2 | process3
CAVEATS
unbuffer -p may appear to work incorrectly if a process feeding input to unbuffer exits. Consider:
process1 | unbuffer -p process2 | process3
If process1 exits, process2 may not yet have finished. It is impossible for unbuffer to know long to wait for process2 and process2 may
not ever finish, for example, if it is a filter. For expediency, unbuffer simply exits when it encounters an EOF from either its input or
process2.
In order to have a version of unbuffer that worked in all situations, an oracle would be necessary. If you want an application-specific
solution, workarounds or hand-coded Expect may be more suitable. For example, the following example shows how to allow grep to finish pro-
cessing when the cat before it finishes first. Using cat to feed grep would never require unbuffer in real life. It is merely a place-
holder for some imaginary process that may or may not finish. Similarly, the final cat at the end of the pipeline is also a placeholder
for another process.
$ cat /tmp/abcdef.log | grep abc | cat
abcdef
xxxabc defxxx
$ cat /tmp/abcdef.log | unbuffer grep abc | cat
$ (cat /tmp/abcdef.log ; sleep 1) | unbuffer grep abc | cat
abcdef
xxxabc defxxx
$
BUGS
The man page is longer than the program.
SEE ALSO
"Exploring Expect: A Tcl-Based Toolkit for Automating Interactive Programs" by Don Libes, O'Reilly and Associates, January 1995.
AUTHOR
Don Libes, National Institute of Standards and Technology
1 June 1994 UNBUFFER(1)