Searching set of string from Live Running Logs


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Searching set of string from Live Running Logs
# 8  
Old 09-17-2010
you can change the above script in this way..
Code:
searchStrFile="SearchStrings.txt"
for file in `ls queryinfoservice_*`
do
echo "searching for the search pattern in the file $file"
while read mLine
do
echo "Searching for the line :-- $mLine"
egrep ''"${mLine}"'' "${file}"
done < "${searchStrFile}"
done

# 9  
Old 09-17-2010
Quote:
Originally Posted by dazdseg
you can change the above script in this way..
Code:
searchStrFile="SearchStrings.txt"
for file in `ls queryinfoservice_*`
do
echo "searching for the search pattern in the file $file"
while read mLine
do
echo "Searching for the line :-- $mLine"
egrep ''"${mLine}"'' "${file}"
done < "${searchStrFile}"
done

You don't really need the "ls queryinfoservice_*" inside the for loop, just:
Code:
searchStrFile="SearchStrings.txt"
searchFPattern="queryinfoservice_*"
for file in ${searchFPattern}
do
echo "searching for the search pattern in the file $file"
while read mLine
do
echo "Searching for the line :-- $mLine"
egrep ''"${mLine}"'' "${file}"
done < "${searchStrFile}"
done

# 10  
Old 09-17-2010
Above syntax wont get me the result for multiple logs. Here is the pattern of logs wat i have.

queryFileService.log1 - queryFileService.log5
getURdetails.log1 - getURdetails.log5
...........
...........
............
nth.log1 - nth.log5

the search should not be an limited to only one log file. It should be user defined, but will exists in Log dir.
# 11  
Old 09-17-2010
tell me one thing are there any specific logs that you dont want to process. if the user mention one particular file. you need all the files with that name and any extension .. then you can modify the above script like this... :-

Code:
echo "enter the name of the log u want to search"
read search_file
searchStrFile="SearchStrings.txt"
for file in `ls search_file*`
do
echo "searching for the search pattern in the file $file"
while read mLine
do
echo "Searching for the line :-- $mLine"
egrep ''"${mLine}"'' "${file}"
done < "${searchStrFile}"
done

# 12  
Old 09-17-2010
This one will ask for the directory and file pattern to validate:
Code:
echo "Please, enter directory and log file name pattern: "
read searchDirFPattern

if [ ! -f "${searchDirFPattern}" ]
then
	echo "Invalid log file. It does not exist."
	exit 1
fi

searchStrFile="SearchStrings.txt"
for file in ${searchDirFPattern}
do
	echo "searching for the search pattern in the file $file"
	while read mLine
	do
		echo "Searching for the line :-- $mLine"
		egrep ''"${mLine}"'' "${file}"
	done < "${searchStrFile}"
done

# 13  
Old 09-18-2010
Please ignore the previous instances of the log file format. Everything is fine with the script before and now also, but while entering the Path & File as an input to search for the string, I would rather provide in this way.

Code:
./search.sh
Please, enter directory and log file name pattern:
/logs/production/mlog/getinfo_LoggerGroup

From the above Input File am not providing any extension parameter like 1,2,3,4 or 5. Actually there are 5 instances of each log file.

Quote:
/getinfo_LoggerGroup1
/getinfo_LoggerGroup2
/getinfo_LoggerGroup3
/getinfo_LoggerGroup4
/getinfo_LoggerGroup5
The script should only search from above 5 files excluding zipped files which are also placed in the same dir, when I provide the input file name " getinfo_LoggerGroup " from the above syntax.

---------- Post updated 09-18-10 at 06:40 AM ---------- Previous update was 09-17-10 at 07:36 PM ----------

Can anybody help me to reorganize this script according to usage shown above.
# 14  
Old 09-20-2010
All above method failed .. to get the string from multiple files. I just wanted to the script should read the File name String and search within all those files whose Pattern matches.

I have enclosed the attachment for File names, from which script should match the pattern and find in the LOG_DIR.

And here is the piece of Code,

Code:
#!/bin/bash

# Script name without path

BASENAME=${0##*/}

#Default Config & Log Path

myLocation=/home/rags234
myInst=${HOME}/config/application.lst
myLog="/logs/production/Application/$myInst"
myString="${HOME}/config/string

#Main Script

echo "Please, enter log file name pattern for SEARCH: ${myInst} "

read -r ${myInst}

if [ ! -f "${myLog}" ]
then
        echo "Invalid log file. It does not exist."
        exit 1
fi

#Searching the Pattern

for ${myInst} in ${myLog}
do
        echo "searching for the search pattern in the file $myInst"
        while read mLine
        do
                echo
                echo "============================================"
                echo "Searching for the line :-- $mLine"
                echo "============================================"
                echo
                egrep ''"${mLine}"'' "${myInst}"
        done <<EOF > "${myString}"
done;

Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Searching exception keyword in all logs in last 5 minutes

Hello Folks , I am a new bie to the world of unix , what i am planning to do is the I have the location in server to which i am access through the putty and the location is /mt/ttlog/avccomn/logs/201901/19 and at this location the files are listed as show startjmsnode1.sh_03.out... (7 Replies)
Discussion started by: punpun26262626
7 Replies

2. Red Hat

Collecting logs by running command

Hi, i am a general user of linux but we work mostly on windows next i am moving full time on linux. here is my question: We have product which consist or several subsystem each subsystem has one module to create logs file dump. and i am going to write that support dump tool. we need to... (3 Replies)
Discussion started by: ajayyadavmca
3 Replies

3. Shell Programming and Scripting

To find latest set of logs among new and old

Hi All I am writing a script which will select the latest logs (which are generated every night via a script) among old one and new. Script generates set of 3 logs each time it runs. Example : log-WedJun082011_bkt1.log log-WedJun082011_bkt2.log log-WedJun082011_bkt3.log I have... (1 Reply)
Discussion started by: ratneshnagori
1 Replies

4. Shell Programming and Scripting

searching the required string and appending string to it.

Hi all, I have some data in the form of adc|nvhs|nahssn|njadnk|nkfds in the above data i need to write a script so thet it will append "|||" to the third occurnace in the string ..... the outout should look like adc|nvhs|nahssn||||njadnk|nkfds Thanks, Firestar. (6 Replies)
Discussion started by: firestar
6 Replies

5. Shell Programming and Scripting

Searching String from set of similar File pattern from the Dir

Guys, Here is the script that searches string from the set of similar files from the log directory, All the file patterns are defined as input file, from where the script should map to those files in the LOG_DIR and should start searching the strings from all those similar files. ... (1 Reply)
Discussion started by: raghunsi
1 Replies

6. UNIX for Advanced & Expert Users

How to find one partucular user logs when there are lots of users running on it

On my application there are lots of users are doing there work or tasks? ...In my SSH or in 'Putty' i am observing logs? Hot to observe one particular 'user' logs.. even through there are lots of users working on it? For EX: i am log in with use rid:nikhil@in.com. another one log in with... (4 Replies)
Discussion started by: ksr.test
4 Replies

7. Shell Programming and Scripting

Searching *.gz logs

I have been trying to search for a string from close to 200 *.gz file, But i get a error. Can someone suggest a bulletproof solution Please. zgrep 20/Aug/2008:13:50:23 request.log.*.gz -bash: /usr/bin/zgrep: /bin/sh: bad interpreter: Argument list too long also zgrep 20/Aug/2008:13:50:23... (9 Replies)
Discussion started by: openspark
9 Replies

8. Shell Programming and Scripting

Extracting a string from one file and searching the same string in other files

Hi, Need to extract a string from one file and search the same in other files. Ex: I have file1 of hundred lines with no delimiters not even space. I have 3 more files. I should get 1 to 10 characters say substring from each line of file1 and search that string in rest of the files and get... (1 Reply)
Discussion started by: mohancrr
1 Replies

9. UNIX for Dummies Questions & Answers

Process launched by user who logs out, continue running ?

Lets say a user starts a process (either a shell script or a Perl script) and before that process finishes, he logs out (either intentionaly or network problems or ...), does the process continu running ? Default shell is Korn. This is because at my job (being trained), there are tasks to run... (2 Replies)
Discussion started by: Browser_ice
2 Replies
Login or Register to Ask a Question