Sponsored Content
Top Forums Shell Programming and Scripting pattern search for multiple log files and counting Post 302352182 by gubbu on Thursday 10th of September 2009 03:33:35 PM
Old 09-10-2009
Thanks for the response . Here is what is working if I need it for one day

Code:
for m in 25;do for q in 1 2 3  ; do for r in 1 2  ; do ssh -q hostname "/bin/zcat /logs/ap${q}-jvm${r}.server.log.200908${k}.gz|grep 'Exception'|wc -l";done;done;done|awk '{SUM+=$1} END {print SUM}'

This gives me for that particular day ...I am having trouble when I want to give it a range .Essentially "wc -l" does the same as "zgrep -c "

For example I tried

Code:
for m in 20 21 22 23 24 25;do for q in 1 2 3 4 5 6 7 8 9 10 ; do for r in 1 2 3 4 ; do ssh -q hostname "/bin/zcat /logs/ap${q}-jvm${r}.server.log.200908${k}.gz|grep 'Exception'|wc -l|awk '{SUM+=$1} END {print SUM}'";done;done;done

I get the following error
Code:
awk: cmd. line:1: {SUM+=} END {print SUM}
awk: cmd. line:1:       ^ syntax error


so basically I want a count of "Exception" per day from multiple files and want to report it for each day

Code:
day1 
23232
day 2
23232
day3 
42323


Last edited by vgersh99; 09-10-2009 at 04:57 PM.. Reason: code tags, PLEASE!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Counting files in a directory that match a pattern

I have 20 files in a direcotry like BARE01_DLY_MKT_YYYYMMDD. The MKT differes for all these files but the remaining syntax remains the same for a particular day. If I am checking for today I need to make sure that there are 20 files that start with BARE01_DLY_MKT_20060720. How can I write a... (31 Replies)
Discussion started by: dsravan
31 Replies

2. UNIX for Dummies Questions & Answers

count the number of files which have a search string, but counting the file only once

I need to count the number of files which have a search string, but counting the file only once if search string is found. eg: File1: Please note that there are 2 occurances of "aaa" aaa bbb ccc aaa File2: Please note that there are 3 occurances of "aaa" aaa bbb ccc... (1 Reply)
Discussion started by: sudheshnaiyer
1 Replies

3. Shell Programming and Scripting

Counting lines in multiple files

Hi, I have couple of .txt files (say 50 files) in a folder. For each file: I need to get the number of lines in each file and then that count -1 (I wanted to exclude the header. Then sum the counts of all files and output the total sum. Is there an efficient way to do this using shell... (7 Replies)
Discussion started by: Lucky Ali
7 Replies

4. Shell Programming and Scripting

multiple files: counting

In a directory, I have 5000 multiple files that contains around 4000 rows with 10 columns in each file containing a unique string 'AT' located at 4th column. OM 3328 O BT 268 5.800 7.500 4.700 0.000 1.400 OM 3329 O BT 723 8.500 8.900... (7 Replies)
Discussion started by: asanjuan
7 Replies

5. Shell Programming and Scripting

Counting occurrences of all words in multiple files

Hey Unix gurus, I would like to count the number occurrences of all the words (regardless of case) across multiple files, preferably outputting them in descending order of occurrence. This is well beyond my paltry shell scripting ability. Researching, I can find many scripts/commands that... (4 Replies)
Discussion started by: twjolson
4 Replies

6. Shell Programming and Scripting

Multiple search pattern

Hello :) I have this file cat employee_list Name : jack Gender: m ID : 4512 DOB : 03/27/1980 hire date : 04/23/2012 Nationality: US marital status : single ===================== Name : mick Gender: m ID : 1256 DOB : 03/27/1970 Hire date : 012/10/2011 Nationality: US Marital... (4 Replies)
Discussion started by: Sara_84
4 Replies

7. Shell Programming and Scripting

awk with multiple pattern search

Want to fetch a column with multiple pattern using awk. How to achieve the same. Tried cat test address : 10.63.20.92/24 address : 10.64.22.93/24 address : 10.53.40.91/24 cat test | awk '{print $3}' |awk -F "/" '{print $1}' 10.63.20.92 10.64.22.93 10.53.40.91 Is there any... (2 Replies)
Discussion started by: Manasa Pradeep
2 Replies

8. Linux

Search multiple pattern from list

I am working on AIX operating system. I want to search list of Article Id for given Set Date (which are present in a seperate file input.txt) art_list.csv ------------ "Article ID" |"Ad Description" |"Pyramid"|"Pyramid Desc "|"ProductTypeId"|"Set Date "|... (3 Replies)
Discussion started by: rajivrsk
3 Replies

9. Shell Programming and Scripting

Search multiple pattern in a file

I have a sample file with following output: HTTP/1.1 200 OK User: admin Set-Cookie: AMBARISESSIONID=y3v3648yqcno32nq478kw7ar;Path=/;HttpOnly Expires: Thu, 01 Jan 1970 00:00:00 GMT Content-Type: text/plain Vary: Accept-Encoding, User-Agent Content-Length: 6057 Server:... (4 Replies)
Discussion started by: saurau
4 Replies

10. Shell Programming and Scripting

Pattern search multiple files

#!/usr/bin/ksh a="Run successfully" cd $APPS ls -l *.txt | while read $txt do if then cp $APPS/$txt cp $hist/$txt else rm $APPS/$txt echo "Files has been removed" fi done New in shell script please help me out Around 100 txt files in $APPS dir i want to search pattern from... (8 Replies)
Discussion started by: Kalia
8 Replies
NEWSYSLOG(8)						    BSD System Manager's Manual 					      NEWSYSLOG(8)

NAME
newsyslog -- maintain system log files to manageable sizes SYNOPSIS
newsyslog [-CFNnrsv] [-R tagname] [-a directory] [-d directory] [-f config_file] [file ...] DESCRIPTION
The newsyslog utility should be scheduled to run periodically by cron(8). When it is executed it archives log files if necessary. If a log file is determined to require archiving, newsyslog rearranges the files so that ``logfile'' is empty, ``logfile.0'' has the last period's logs in it, ``logfile.1'' has the next to last period's logs in it, and so on, up to a user-specified number of archived logs. Optionally the archived logs can be compressed to save space. A log can be archived for three reasons: 1. It is larger than the configured size (in kilobytes). 2. A configured number of hours have elapsed since the log was last archived. 3. This is the specific configured hour for rotation of the log. The granularity of newsyslog is dependent on how often it is scheduled to run by cron(8). Since the program is quite fast, it may be sched- uled to run every hour without any ill effects, and mode three (above) assumes that this is so. OPTIONS
The following options can be used with newsyslog: -f config_file Instruct newsyslog to use config_file instead of /etc/newsyslog.conf and /etc/newsyslog.d/*.conf for its configuration file. -a directory Specify a directory into which archived log files will be written. If a relative path is given, it is appended to the path of each log file and the resulting path is used as the directory into which the archived log for that log file will be written. If an abso- lute path is given, all archived logs are written into the given directory. If any component of the path directory does not exist, it will be created when newsyslog is run. -d directory Specify a directory which all log files will be relative to. To allow archiving of logs outside the root, the directory passed to the -a option is unaffected. -v Place newsyslog in verbose mode. In this mode it will print out each log and its reasons for either trimming that log or skipping it. -n Cause newsyslog not to trim the logs, but to print out what it would do if this option were not specified. -r Remove the restriction that newsyslog must be running as root. Of course, newsyslog will not be able to send a HUP signal to syslogd(8) so this option should only be used in debugging. -s Specify that newsyslog should not send any signals to any daemon processes that it would normally signal when rotating a log file. For any log file which is rotated, this option will usually also mean the rotated log file will not be compressed if there is a dae- mon which would have been signalled without this option. However, this option is most likely to be useful when specified with the -R option, and in that case the compression will be done. -C If specified once, then newsyslog will create any log files which do not exist, and which have the C flag specified in their config file entry. If specified multiple times, then newsyslog will create all log files which do not already exist. If log files are given on the command-line, then the -C or -CC will only apply to those specific log files. -F Force newsyslog to trim the logs, even if the trim conditions have not been met. This option is useful for diagnosing system prob- lems by providing you with fresh logs that contain only the problems. -N Do not perform any rotations. This option is intended to be used with the -C or -CC options when creating log files is the only objective. -R tagname Specify that newsyslog should rotate a given list of files, even if trim conditions are not met for those files. The tagname is only used in the messages written to the log files which are rotated. This differs from the -F option in that one or more log files must also be specified, so that newsyslog will only operate on those specific files. This option is mainly intended for the daemons or programs which write some log files, and want to trigger a rotate based on their own criteria. With this option they can execute newsyslog to trigger the rotate when they want it to happen, and still give the system administrator a way to specify the rules of rotation (such as how many backup copies are kept, and what kind of compression is done). When a daemon does execute newsyslog with the -R option, it should make sure all of the log files are closed before calling newsyslog, and then it should re-open the files after newsyslog returns. Usually the calling process will also want to specify the -s option, so newsyslog will not send a signal to the very process which called it to force the rotate. Skipping the signal step will also mean that newsyslog will return faster, since newsyslog normally waits a few seconds after any signal that is sent. If additional command line arguments are given, newsyslog will only examine log files that match those arguments; otherwise, it will examine all files listed in the configuration file(s). FILES
/etc/newsyslog.conf newsyslog configuration file /etc/newsyslog.d/ newsyslog configuration directory COMPATIBILITY
Previous versions of the newsyslog utility used the dot (``.'') character to distinguish the group name. Beginning with FreeBSD 3.3, this has been changed to a colon (``:'') character so that user and group names may contain the dot character. The dot (``.'') character is still accepted for backwards compatibility. HISTORY
The newsyslog utility originated from NetBSD and first appeared in FreeBSD 2.2. AUTHORS
Theodore Ts'o, MIT Project Athena Copyright 1987, Massachusetts Institute of Technology SEE ALSO
bzip2(1), gzip(1), syslog(3), newsyslog.conf(5), chown(8), syslogd(8) BUGS
Does not yet automatically read the logs to find security breaches. BSD
February 24, 2005 BSD
All times are GMT -4. The time now is 07:49 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy