Sponsored Content
Top Forums Shell Programming and Scripting Grep patterns and group counts Post 302953960 by finn on Thursday 3rd of September 2015 05:50:25 AM
Old 09-03-2015
Grep patterns and group counts

Hi,

I have a continuous log file which has the following format:-

Code:
02/Sep/2015: IP 11.151.108.166 error occurred etc
03/Sep/2015: IP 11.151.108.188 error occurred etc
03/Sep/2015: IP 11.152.178.250 error occurred etc
03/Sep/2015: IP 11.188.108.176 error occurred etc
03/Sep/2015: IP 11.14.108.146 error occurred etc
03/Sep/2015: IP 11.188.178.1 error occurred etc
03/Sep/2015: IP 11.151.188.142 error occurred etc
03/Sep/2015: IP 11.151.21.188 error occurred etc

I want to create a daily report of total counts of IPs grouped by the first two sections of the IP eg 11.151 etc. So, for the above it would show:-

Code:
IP total counts for 03/Sep/2015:-

11.151 = 3 (entries)
11.152 = 1
11.188 = 2
11.14 = 1
etc

I think i'm okay doing the count by the date but i'm not sure how to grep the pattern of the IP.
I do know which IP ranges are in the log but would rather not hard code each individual pattern (e.g. grep 11.151*, grep 11.152* etc) in case new ones appear.

Thanks

Last edited by Scrutinizer; 09-03-2015 at 07:51 AM.. Reason: CODE tags
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Group by columns and get the counts

Hi Gurus, I have a file 1|usa|hh 2|usa|ll 3|usa|vg 4|usa|vg 5|usa|vg 6|usa|vg 7|usa|ll 8|uk|nn 9|uk|bb 10|uk|bb 11|kuwait|mm 12|kuwait|jkj 13|kuwait|mm 14|dubai|hh I want to group by last two columns and get the last two recs and count. (3 Replies)
Discussion started by: sumeet
3 Replies

2. Shell Programming and Scripting

to grep and print their counts

suppose u have a file ACFCFACCACARCSHFARCVJVASTVAJFTVAJVGHBAJ another file A C F R then output shud be A= 9 C=7 F=3 R=2 Thanks (12 Replies)
Discussion started by: cdfd123
12 Replies

3. Shell Programming and Scripting

grep or sed patterns

i want to match a pattern * ^Subject:.*<\hello\> for my emails to me that has 'hello' anywhere in the subject. (3 Replies)
Discussion started by: oxoxo
3 Replies

4. Shell Programming and Scripting

Grep between 2 patterns in a line

Hi I have a file containing 1000+ lines of netlist data. I need to search for text between two key words on each line for e.g my input file has "ABC.ABC__312.deftr_0.X143.Xntys_0.\Xetabc__ABC_test_tz .X1023" "ABC.ABC__312.asjartwtsj"... (16 Replies)
Discussion started by: naveen@
16 Replies

5. Shell Programming and Scripting

grep value between two patterns

Hi All, I've been trying solve this with a simple command but not having much luck. I have a file like this: Line 1: random_description 123/alert/high random_description2 356/alert/slow Line 2: random_description3 654/alert/medium Line 3: random_description4 234/alert/critical I'm... (7 Replies)
Discussion started by: joe19
7 Replies

6. Shell Programming and Scripting

How to group matched patterns in different files

Hi, I have a master file that i need to split into multiple files based on matched patterns. sample of my data as follows:- scaff_1 a e 123 130 c_scaff_100 scaff_1 a e 132 138 c_scaff_101 scaff_1 a e 140 150 ... (2 Replies)
Discussion started by: redse171
2 Replies

7. UNIX for Dummies Questions & Answers

Grep - various patterns

I have a file with the following text: grep \$ grep \\$ grep \\\$ grep '\$' grep '\'$' grep \\ grep \\\\ grep "\$" grep '"$' grep "$" When I perform these same commands on this file, the result are never what I would expect them to be. Could someone please comment on the results and... (3 Replies)
Discussion started by: uran101
3 Replies

8. Shell Programming and Scripting

need a one liner to grep a group info from /etc/group and use that result to search passwd file

/etc/group tiadm::345:mk789,po312,jo343,ju454,ko453,yx879,iy345,hn453 bin::2:root,daemon sys::3:root,bin,adm adm::4:root,daemon uucp::5:root /etc/passwd mk789:x:234:1::/export/home/dummy:/bin/sh po312:x:234:1::/export/home/dummy:/bin/sh ju454:x:234:1::/export/home/dummy:/bin/sh... (6 Replies)
Discussion started by: chidori
6 Replies

9. Shell Programming and Scripting

Grep patterns

Hi Experts, I have a log file like this.I need to filter the Index name and elapsed time(only created ). 06:36:39 SQL> create index XYZ_F75 on XYZ 06:36:39 2 ("GRP_ID", "_ID") parallel 64 nologging 06:36:39 3 tablespace XARGS_IDX 06:36:39 4 ; Index created. Elapsed:... (2 Replies)
Discussion started by: navsan420
2 Replies

10. Shell Programming and Scripting

How to use grep with multiple patterns?

I am trying to grep a variable with multiple lines with multiple patterns below is the pattern list in a variable called "grouplst", each pattern is speerated by "|" grouplst="example1|example2|example3|example4|example5|example6|example7" I need to use the patterns above to grep a... (2 Replies)
Discussion started by: ajetangay
2 Replies
Mail::Mbox::MessageParser::Grep(3pm)			User Contributed Perl Documentation		      Mail::Mbox::MessageParser::Grep(3pm)

NAME
Mail::Mbox::MessageParser::Grep - A GNU grep-based mbox folder reader SYNOPSIS
#!/usr/bin/perl use Mail::Mbox::MessageParser; my $filename = 'mail/saved-mail'; my $filehandle = new FileHandle($filename); my $folder_reader = new Mail::Mbox::MessageParser( { 'file_name' => $filename, 'file_handle' => $filehandle, 'enable_grep' => 1, } ); die $folder_reader unless ref $folder_reader; # Any newlines or such before the start of the first email my $prologue = $folder_reader->prologue; print $prologue; # This is the main loop. It's executed once for each email while(!$folder_reader->end_of_file()); { my $email = $folder_reader->read_next_email(); print $email; } DESCRIPTION
This module implements a GNU grep-based mbox folder reader. It can only be used when GNU grep is installed on the system. Users must not instantiate this class directly--use Mail::Mbox::MessageParser instead. The base MessageParser module will automatically manage the use of grep and non-grep implementations. METHODS AND FUNCTIONS The following methods and functions are specific to the Mail::Mbox::MessageParser::Grep package. For additional inherited ones, see the Mail::Mbox::MessageParser documentation. $ref = new( { 'file_name' => <mailbox file name>, 'file_handle' => <mailbox file handle> }); <file_name> - The full filename of the mailbox <file_handle> - An opened file handle for the mailbox The constructor for the class takes two parameters. The file_name parameter is the filename of the mailbox. The file_handle argument is the opened file handle to the mailbox. Returns a reference to a Mail::Mbox::MessageParser object, or a string describing the error. BUGS
No known bugs. Contact david@coppit.org for bug reports and suggestions. AUTHOR
David Coppit <david@coppit.org>. LICENSE
This software is distributed under the terms of the GPL. See the file "LICENSE" for more information. HISTORY
This code was originally part of the grepmail distribution. See http://grepmail.sf.net/ for previous versions of grepmail which included early versions of this code. SEE ALSO
Mail::Mbox::MessageParser perl v5.10.1 2009-08-09 Mail::Mbox::MessageParser::Grep(3pm)
All times are GMT -4. The time now is 01:18 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy