Sponsored Content
Top Forums Shell Programming and Scripting How to read each file grep a value from that? Post 303005811 by Don Cragun on Monday 23rd of October 2017 02:14:16 PM
Old 10-23-2017
If all of the files are in a single directory (not spread out among subdirectories), you could try the following first:
Code:
cd /tmp
grep -FHi 'Calling appalert' *.log

or, if you just want the names of files containing that string (with a case insensitive match):
Code:
cd /tmp
grep -Fli 'Calling appalert' *.log

These might file with "argument list too long" errors depending on how many thousands of files you have, but it is easy to try it. And, if it works, it will hundreds or thousands of times faster that performing a grep on each file individually. If they won't work, you'll know quickly.

Note that I added a -F option to grep. Searching for fixed strings (even if you're doing a case-insensitive match), is faster than going through the extra complexity of matching regular expressions.

If all of the log entries use the same case, dropping the -i option will also make it run faster.
This User Gave Thanks to Don Cragun For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Read file then grep the line

Dear all, I am reading a file that has 1 column. While reading I must find the line references from the another file. The following shell doesn't works. Please help #!/bin/bash while read filename; do grep ${filename} fs_full.dat >> unprocfull.dat; done < unproc.dat But when... (2 Replies)
Discussion started by: mr_bold
2 Replies

2. Shell Programming and Scripting

read in variable data from another file - grep

Hello! I think this should be an easy solution. I have a large file with many fields of data. The first field has a unique identifier (a subject number) for every record for a chunk of data. Something like this: There were ten experimental conditions (ec), but the ec is identified by only... (11 Replies)
Discussion started by: ccox85
11 Replies

3. Shell Programming and Scripting

grep vs do while read

I have inherited the UNIX scripting duties for our team and have only been at this for a few months so I apologize for what may be a simple question. We have a script we are reworking. Part of this script takes a parameter and reads a csv file. When it finds a matching line, the cut command is... (4 Replies)
Discussion started by: 15a0
4 Replies

4. UNIX for Advanced & Expert Users

how to grep/read a file inside compressed tgz without extract?

Hi all, I would like to ask whether in Unix shell/perl have any functions or command to allow grep/cat/read a file inside compressed .tgz without extract it? I know we can tar tvf a compressed tgz but this only allow we read the path/filename contained inside the tarball. If we want to read... (3 Replies)
Discussion started by: mayshy
3 Replies

5. UNIX for Dummies Questions & Answers

Strange error: grep does not read from file

I have two files file1.txt angie mary susan file2.txt angie blond mary brunnet susan red christine blackI want to get this output angie blond mary brunnet susan redI write grep --file=file1.txt file2.txtand i get no results i also wrote cat file1.txt|while read line... (19 Replies)
Discussion started by: FelipeAd
19 Replies

6. Shell Programming and Scripting

read from file and grep using shell

Hi Guys, I have a small script which greps for the username reading from stdinput. ./file.sh pattern pattern=$1 grep "blah blah.*$pattern" /home/user/log.txt Instead of typing the pattern everytime i want to read the pattern from a file inside the shell script and execute the... (5 Replies)
Discussion started by: Irishboy24
5 Replies

7. Shell Programming and Scripting

read a file for input and grep in another file

Hi, I'm trying to read a fille into a loop and grep for a string and print the last field of the string in the second file. Then redirect the output to another file but keeping the output in the same order as the original file. I've tried using the following but the ouput from this does not... (3 Replies)
Discussion started by: elmesy
3 Replies

8. UNIX for Dummies Questions & Answers

Piping grep into awk, read the next line using grep

Hi, I have a number of files containing the information below. """"" Fundallinfo 6.3950 14.9715 14.0482 """"" I would like to grep for Fundallinfo and use it to read the next line? I ideally would like to read the three numbers that follow in the next line and... (2 Replies)
Discussion started by: Paul Moghadam
2 Replies

9. UNIX for Beginners Questions & Answers

Read a file and send mail based on grep

Hi All, I am having a job and I need to send email when the job is running. On any other case (success,fail) I don't needed to send email. I check with BMC they told they dont have that in the version I am using. So I created a dependent job and grepped for the status and sent email. My... (1 Reply)
Discussion started by: arunkumar_mca
1 Replies

10. UNIX for Advanced & Expert Users

Need help for faster file read and grep in big files

I have a very big input file <inputFile1.txt> which has list of mobile no inputFile1.txt 3434343 3434323 0970978 85233 ... around 1 million records i have another file as inputFile2.txt which has some log detail big file inputFile2.txt afjhjdhfkjdhfkd df h8983 3434343 | 3483 | myout1 |... (3 Replies)
Discussion started by: reldb
3 Replies
CG(1)																	     CG(1)

NAME
cg - Recursively grep for a pattern and store it. SYNOPSIS
cg [ -l ] | [ [ -i ] pattern [ files ] ] DESCRIPTION
cg does a search though text files (usually source code) recursively for a pattern, storing matches and displaying the output in a human- readable fashion. It is intended to give some of the functionaly of AT&T's cscope(1) tool, with the advantages of simplicity and not being language-specific. The script will colorize output if configured as such. It is typically run with a Perl regular expression to search for. The search can be made case insensitive by using the -i option. A list of files may also be specified with an additional argument after the pattern. Put the files pattern in quotes to make it be matched by Perl rather than by the shell. Running the script with no arguments will recall the results of the previous search. After the search, entries found can be edited using the vg(1) script. The -l option shows the last log made. SOME EXAMPLES
cg - alone recalls the previous search results. cg -i pattern - search the default list of files for all files matching the pattern (and case-insensitively). cg pattern '*.c' - search recursively for pattern in all *.c files. This automatically converts '*' to '.*' and '.' to '.' for you and does a Perl pattern match on all files in the tree. cg pattern *.c - search through the shell-expanded list of *.c files, so not done recursively (in other words, only the files your shell pass to the script as arguments). cg -l - show the last log made. COMMAND-LINE OPTIONS -i Do a case-insensitive search. -l Show the last log made. -p Toggle the default pager option. cg has a bulit-in pager function, which can be enabled or disabled by default (in .cgvgrc). If the default is enabled, this option disables the pager; if the default is disabled, this option enables it. -P Force the built-in pager to be disabled. FILES
${HOME}/.cglast Log file of the last search. ${HOME}/.cgvgrc Per-user configuration file (if the defaults are not desireable). ${HOME}/.cgvg/* Log files in $HOSTNAME.shell_pid form with the log of the last search. SEE ALSO
vg(1), perl(1), find(1), grep(1), cscope(1) AUTHOR
cg was written by Joshua Uziel <uzi@uzix.org>. 13 Mar 2002 CG(1)
All times are GMT -4. The time now is 09:55 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy