Sponsored Content
Top Forums Shell Programming and Scripting awk script to detect specific string in a log file and count it Post 303006188 by ktisbest on Saturday 28th of October 2017 10:50:54 PM
Old 10-28-2017
Question awk script to detect specific string in a log file and count it

Hello, can someone guide me on this?

I don't know what is the best approach, (awk script, shell script)

I am using RedHat Linux version 6.5. There is a third party application deployed on that server. This app by default generates 5 log files and each file is 20MB. These log rollover automatically so at any given time there always be 5 log file like below

appname.log
appname.log.1
appname.log.2
appname.log.3
appname.log.4

When the app runs, it creates the following entries in the log file. for simplicity, i have removed the other information.

Requirement is whenever this string RESTful/BR4/Dispatchers/Drop Down Dispatch Process.process/Log appears in the log file, count it. So at the end of the day, we wanted to know on Oct 20, 2018, how many times that entry appeared in the log file. Basically, everyday, we need to collect statics about this app. The script should run automatically and spit out the stats in a file like this

appname date count
Code:
BW.AMI_Services-Dropdown-AMI_Services-Dropdown-ProcessArchive Oct 20, 2018 3

-----log files entries------
Code:
2017 Oct 20 13:44:03:359 GMT -0600 BW.AMI_Services-Dropdown-AMI_Services-Dropdown-ProcessArchive User [BW-User] - Job-20540 [Services - RESTful/BR4/Dispatchers/Drop Down Dispatch Process.process/Log]:
 
2017 Oct 20 13:45:02:170 GMT -0600 BW.AMI_Services-Dropdown-AMI_Services-Dropdown-ProcessArchive User [BW-User] - Job-20546 [Services - RESTful/BR4/Dispatchers/Drop Down Dispatch Process.process/Log]:
 
2017 Oct 20 13:45:10:605 GMT -0600 BW.AMI_Services-Dropdown-AMI_Services-Dropdown-ProcessArchive User [BW-User] - Job-20547 [Services - RESTful/BR4/Dispatchers/Drop Down Dispatch Process.process/Log]:

Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!

Last edited by RudiC; 10-29-2017 at 06:24 AM.. Reason: Added CODE tags.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with scritp to count an specific word into a log

Hello my friends, I need to count how many words are into a log file, I'm using: cat logfile | grep 'word' | wc -l Cuz the 'word' appears once per line. But my logfile grow faster and at the end ofthe day is really big, so how i can count the 'word' only from (by example) line 4000 of... (5 Replies)
Discussion started by: lestat_ecuador
5 Replies

2. Shell Programming and Scripting

awk script to count percentage from log file

Hi, I have a log like this : actually i want to get the log like this : where % can get from : 100 * pmTotNoRrcConnectReqSucc / pmTotNoRrcConnectReq Thanks in advance.. :) (8 Replies)
Discussion started by: justbow
8 Replies

3. Shell Programming and Scripting

AWK script to detect webpages from file

Hi guys I'm very new to unix and I have to create an awk script that detects webpage addresses from a file/webpage and outputs how many times each webpage was detected.e.g. if my file was: (Note: The symbol " was added to stop them being created into links) "www.google.com"... (1 Reply)
Discussion started by: ROFL
1 Replies

4. Shell Programming and Scripting

AWK script to detect webpages from file

Hi guys I'm very new to unix and I have to create an awk script that detects webpage addresses from a file/webpage and outputs how many times each webpage was detected.e.g. if my file was: www.google.com www.facebook.com www.google.com the output should be: www.google.com x2... (2 Replies)
Discussion started by: ROFL
2 Replies

5. UNIX for Dummies Questions & Answers

How to Detect Specific Pattern and Print the Specific String after It?

I'm still beginner and maybe someone can help me. I have this input: the great warrior a, b, c and what i want to know is, with awk, how can i detect the string with 'warrior' string on it and print the a, b, and c seperately, become like this : Warrior Type a b c Im still very... (3 Replies)
Discussion started by: radynaraya
3 Replies

6. Shell Programming and Scripting

awk script to count characters in file 1 in file 2

I need a scripting AWK to compare 2 files. file 1 and 2 are list of keywords 1 is a b c d 2 is aa aaa b bb ccc d I want the AWK script to give us the number of times every keyword in file 1 occurs in file 2. output should be a 2 (7 Replies)
Discussion started by: anhtt
7 Replies

7. UNIX for Dummies Questions & Answers

Search for a specific String in a log file for a specific date range

Hi, I have log file which rolls out every second which is as this. HttpGenRequest - -<!--OXi dbPublish--> <created="2014-03-24 23:45:37" lastMsgId="" requestTime="0.0333"> <response request="getOutcomeDetails" code="114" message="Request found no matching data" debug="" provider="undefined"/>... (3 Replies)
Discussion started by: karthikprakash
3 Replies

8. Shell Programming and Scripting

awk to output match and mismatch with count using specific fields

In the below awk I am trying output to one file those lines that match between $2,$3,$4 of file1 and file2 with the count in (). I am also trying to output those lines that are missing between $2,$3,$4 of file1 and file2 with the count of in () each. Both input files are tab-delimited, but the... (7 Replies)
Discussion started by: cmccabe
7 Replies

9. Shell Programming and Scripting

awk to change specific string to new value if found in text file

I am trying to use awk to change a specific string in a field, if it is found, to another value. In the tab-delimited file the text in bold in $3 contains the string 23, which is always right before a ., if it is present. I am trying to change that string to X, keeping the formatting and the... (3 Replies)
Discussion started by: cmccabe
3 Replies

10. Shell Programming and Scripting

Count specific character of a file in each line and delete this character in a specific position

I will appreciate if you help me here in this script in Solaris Enviroment. Scenario: i have 2 files : 1) /tmp/TRANSACTIONS_DAILY_20180730.txt: 201807300000000004 201807300000000005 201807300000000006 201807300000000007 201807300000000008 2)... (10 Replies)
Discussion started by: teokon90
10 Replies
Log::Dispatch::Output(3)				User Contributed Perl Documentation				  Log::Dispatch::Output(3)

NAME
Log::Dispatch::Output - Base class for all Log::Dispatch::* objects SYNOPSIS
package Log::Dispatch::MySubclass; use Log::Dispatch::Output; use base qw( Log::Dispatch::Output ); sub new { my $proto = shift; my $class = ref $proto || $proto; my %p = @_; my $self = bless {}, $class; $self->_basic_init(%p); # Do more if you like return $self; } sub log_message { my $self = shift; my %p = @_; # Do something with message in $p{message} } 1; DESCRIPTION
This module is the base class from which all Log::Dispatch::* objects should be derived. CONSTRUCTOR
The constructor, "new", must be overridden in a subclass. See Output Classes for a description of the common parameters accepted by this constructor. METHODS
o _basic_init(%p) This should be called from a subclass's constructor. Make sure to pass the arguments in @_ to it. It sets the object's name and minimum level. It also sets up two other attributes which are used by other Log::Dispatch::Output methods, level_names and level_numbers. o name Returns the object's name. o min_level Returns the object's minimum log level. o max_level Returns the object's maximum log level. o accepted_levels Returns a list of the object's accepted levels (by name) from minimum to maximum. o log( level => $, message => $ ) Sends a message if the level is greater than or equal to the object's minimum level. This method applies any message formatting callbacks that the object may have. o _should_log ($) This method is called from the "log()" method with the log level of the message to be logged as an argument. It returns a boolean value indicating whether or not the message should be logged by this particular object. The "log()" method will not process the message if the return value is false. o _level_as_number ($) This method will take a log level as a string (or a number) and return the number of that log level. If not given an argument, it returns the calling object's log level instead. If it cannot determine the level then it will croak. o add_callback( $code ) Adds a callback (like those given during construction). It is added to the end of the list of callbacks. Subclassing This class should be used as the base class for all logging objects you create that you would like to work under the Log::Dispatch architecture. Subclassing is fairly trivial. For most subclasses, if you simply copy the code in the SYNOPSIS and then put some functionality into the "log_message" method then you should be all set. Please make sure to use the "_basic_init" method as directed. The actual logging implementation should be done in a "log_message" method that you write. Do not override "log"!. AUTHOR
Dave Rolsky, <autarch@urth.org> perl v5.12.1 2009-09-22 Log::Dispatch::Output(3)
All times are GMT -4. The time now is 08:28 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy