Sponsored Content
Top Forums Shell Programming and Scripting pattern search for multiple log files and counting Post 302352170 by gubbu on Thursday 10th of September 2009 03:05:46 PM
Old 09-10-2009
pattern search for multiple log files and counting

I have 10 appservers and each appserver has 4 jvms . Each of these logs is archived and stored on a nfs directory . For example the files are

/logs/200907/ap1-jvm1.server.log.20090715.gz
/logs/200907/ap2-jvm2.server.log.20090714.gz
/logs/200908/ap1-jvm1.server.log.20090812.gz

I want to search for say some pattern from these files from say august 15th to sept 15th and give me a count of that pattern .For example I want a daily count from aug15-sept 15 for app server and jvm for pattern "Exception" from the log . It should look like

-----
Aug 15
3434
Aug 16
2323
....
....
Sept10
5656

thanks
Gubbs
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Counting files in a directory that match a pattern

I have 20 files in a direcotry like BARE01_DLY_MKT_YYYYMMDD. The MKT differes for all these files but the remaining syntax remains the same for a particular day. If I am checking for today I need to make sure that there are 20 files that start with BARE01_DLY_MKT_20060720. How can I write a... (31 Replies)
Discussion started by: dsravan
31 Replies

2. UNIX for Dummies Questions & Answers

count the number of files which have a search string, but counting the file only once

I need to count the number of files which have a search string, but counting the file only once if search string is found. eg: File1: Please note that there are 2 occurances of "aaa" aaa bbb ccc aaa File2: Please note that there are 3 occurances of "aaa" aaa bbb ccc... (1 Reply)
Discussion started by: sudheshnaiyer
1 Replies

3. Shell Programming and Scripting

Counting lines in multiple files

Hi, I have couple of .txt files (say 50 files) in a folder. For each file: I need to get the number of lines in each file and then that count -1 (I wanted to exclude the header. Then sum the counts of all files and output the total sum. Is there an efficient way to do this using shell... (7 Replies)
Discussion started by: Lucky Ali
7 Replies

4. Shell Programming and Scripting

multiple files: counting

In a directory, I have 5000 multiple files that contains around 4000 rows with 10 columns in each file containing a unique string 'AT' located at 4th column. OM 3328 O BT 268 5.800 7.500 4.700 0.000 1.400 OM 3329 O BT 723 8.500 8.900... (7 Replies)
Discussion started by: asanjuan
7 Replies

5. Shell Programming and Scripting

Counting occurrences of all words in multiple files

Hey Unix gurus, I would like to count the number occurrences of all the words (regardless of case) across multiple files, preferably outputting them in descending order of occurrence. This is well beyond my paltry shell scripting ability. Researching, I can find many scripts/commands that... (4 Replies)
Discussion started by: twjolson
4 Replies

6. Shell Programming and Scripting

Multiple search pattern

Hello :) I have this file cat employee_list Name : jack Gender: m ID : 4512 DOB : 03/27/1980 hire date : 04/23/2012 Nationality: US marital status : single ===================== Name : mick Gender: m ID : 1256 DOB : 03/27/1970 Hire date : 012/10/2011 Nationality: US Marital... (4 Replies)
Discussion started by: Sara_84
4 Replies

7. Shell Programming and Scripting

awk with multiple pattern search

Want to fetch a column with multiple pattern using awk. How to achieve the same. Tried cat test address : 10.63.20.92/24 address : 10.64.22.93/24 address : 10.53.40.91/24 cat test | awk '{print $3}' |awk -F "/" '{print $1}' 10.63.20.92 10.64.22.93 10.53.40.91 Is there any... (2 Replies)
Discussion started by: Manasa Pradeep
2 Replies

8. Linux

Search multiple pattern from list

I am working on AIX operating system. I want to search list of Article Id for given Set Date (which are present in a seperate file input.txt) art_list.csv ------------ "Article ID" |"Ad Description" |"Pyramid"|"Pyramid Desc "|"ProductTypeId"|"Set Date "|... (3 Replies)
Discussion started by: rajivrsk
3 Replies

9. Shell Programming and Scripting

Search multiple pattern in a file

I have a sample file with following output: HTTP/1.1 200 OK User: admin Set-Cookie: AMBARISESSIONID=y3v3648yqcno32nq478kw7ar;Path=/;HttpOnly Expires: Thu, 01 Jan 1970 00:00:00 GMT Content-Type: text/plain Vary: Accept-Encoding, User-Agent Content-Length: 6057 Server:... (4 Replies)
Discussion started by: saurau
4 Replies

10. Shell Programming and Scripting

Pattern search multiple files

#!/usr/bin/ksh a="Run successfully" cd $APPS ls -l *.txt | while read $txt do if then cp $APPS/$txt cp $hist/$txt else rm $APPS/$txt echo "Files has been removed" fi done New in shell script please help me out Around 100 txt files in $APPS dir i want to search pattern from... (8 Replies)
Discussion started by: Kalia
8 Replies
nfslog.conf(4)							   File Formats 						    nfslog.conf(4)

NAME
nfslog.conf - NFS server logging configuration file SYNOPSIS
/etc/nfs/nfslog.conf DESCRIPTION
The nfslog.conf file specifies the location of the NFS server logs, as well as the location of the private work files used by the NFS server and nfslogd(1M) daemon during logging. Each entry in the file consists of a mandatory tag identifier and one or more parameter iden- tifiers. The parameter identifier specifies the value or location of the specific parameter. For instance, the parameter identifier "log=/var/nfs/logs/serverLog" specifies the location of the NFS server activity log. The mandatory tag identifier serves as an index into the /etc/nfs/nfslog.conf file to identify the various parameters to be used. At export time, the share_nfs(1M) command specifies the NFS server logging parameters to use by associating a tag from the /etc/nfs/nfslog.conf file to the exported file system. It is legal for more than one file system to be exported using the same logging tag identifier. NFS server logging is not supported on Solaris machines that are using NFS Version 4. A "global" tag identifier is included in /etc/nfs/nfslog.conf. It specifies the default set of values to be used during logging. If no tag identifier is specified at export time, then the values in the "global" entry are used. The "global" values can be modified by updating this entry in /etc/nfs/nfslog.conf. Each entry in the file must contain a mandatory tag identifier and at least one parameter/value pair. If a parameter is not specified in a given entry, the global value of the parameter will be used. The exact entry syntax follows: <tag> [defaultdir=<path>] [log=<path><file>] [fhtable=<path><file>] [buffer=<path><file>] [logformat=basic|extended] defaultdir=<path> Specifies the directory where the logging files and working files will be placed. This path is prepended to all relative paths specified in other parameters. log=<path><file> Specifies the location of the user-readable log file. The log will be located in the defaultdir, unless <path> is an absolute path. fhtable=<path><file> Specifies the location of the private file handle to path mapping database files. These database files are for the private use of the NFS server kernel module and the nfslogd daemon. These files will be located in the defaultdir, unless <path> is an absolute path. These database files are permanently stored in the file system. Consult nfslogd(1M) for information on pruning the database files. buffer=<path><file> Specifies the location of the private work buffer file used by the NFS server kernel module to record raw RPC information. This file is later processed by the nfslog daemon, which in turn generates the user-readable log file. This work buffer file will be located in the defaultdir, unless <path> is an absolute path. logformat=basic|extended Sets the format of the user-readable log file. If not specified, the basic format is used. The basic format is compatible with log files generated by the Washington University FTPd. The extended format provides a more detailed log, which includes directory modification operations not included in the basic format, such as mkdir, rmdir and remove. Note that the extended format is not compatible with Washington University's FTPd log format. EXAMPLES
Example 1 Using the global Tag The "global" tag may be modified so that all exported file systems that enabled logging use a common set of parameters that conform to the specific needs of the user. These values are used until a specific tag identifier overrides them. global defaultdir=/var/nfs log=logs/nfslog fhtable=tables/fhtable buffer=buffers/nfslog_workbuffer logformat=basic Example 2 Overriding the Global defaultdir and logformat Because log files can become very large, it may be desirable to store the logs and working files in separate file systems. This can be eas- ily accomplished by simply specifying a different defaultdir for every file system exported by means of a unique tag: engineering defaultdir=/engineering/logging logformat=extended accounting defaultdir=/accounting/logging marketing defaultdir=/marketing/logging File systems shared with the engineering identifier will have their logs and workfiles located in /engineering/logging. For instance, the log file will be located at /engineering/logging/logs/nfslog. Note that the engineering log file will be stored in the extended format, while the rest of the log files will remain in the basic format. Any of the parameters can be updated in a tag identifier, which overrides the global settings. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWnfssr | +-----------------------------+-----------------------------+ SEE ALSO
nfslogd(1M), share_nfs(1M), attributes(5) NOTES
Logs, work files, and file handle to path mapping database can become very large. Be aware of appropriate placement within the file system name space. See nfslogd(1M)) for information on pruning the database files and cycling logs. SunOS 5.11 2 Dec 2004 nfslog.conf(4)
All times are GMT -4. The time now is 03:25 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy