Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Extract IP from logs and

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extract IP from logs and
# 22  
Old 04-02-2013
thanks man that's work perfect!

Code:
more code1.out
rm -f b.out c.out
human=0
machine=0

while read line
do
   TM=${line%%AM*}AM
   [ ${#TM} -ge ${#line} ] && TM=${line%%PM*}PM
   IP=${line##*login:}
   IP=${IP%%:*}
   printf "%s %d\n" "$IP" $(./strepoch.pl "$TM")
done < a.out | sort -t' ' -k1,1 -k2,2n | while read IP time
do
   if [ "$IP" = "$prev_ip" ]
   then
      if [ $human -eq 1 -o $(( time - prev_time )) -gt 4 ]
      then
         human=1
      else
         machine=1
      fi
   else
      [ $human -eq 1 ] && echo $prev_ip >> b.out
      [ $machine -eq 1 ] && echo $prev_ip >> c.out
      prev_ip=$IP
      prev_time=$time
      human=0
      machine=0
   fi
done
[ $human -eq 1 ] && echo $prev_ip >> b.out
[ $machine -eq 1 ] && echo $prev_ip >> c.out

with
Code:
 more strepoch.pl
#!/usr/bin/perl
my @months = ("JAN", "FEB", "MAR", "APR", "MAY",
"JUN","JUL", "AUG", "SEP", "OCT", "NOV", "DEC");
use Time::Local 'timelocal';

my $mname, $month, $year, $month, $day, $hour, $min, $sec, $pm;

my $ln = $ARGV[0];
if($ln =~ /([A-z]{3}) (\d+), (\d{4}) (\d+):(\d\d):(\d\d) ([PA]M)/) {
  $ln =~ s/([A-z]{3}) (\d+), (\d{4}) (\d+):(\d\d):(\d\d) ([PA]M)/\1 \2 \3 \4 \5 \6 \7/;
  ($mname, $day, $year, $hour, $min, $sec, $pm) = split(/\s/, $ln);
  $hour += 12 if ($pm eq "PM" && $hour lt 12);
  ($month)=grep { $months[$_] eq uc($mname) } 0..$#months;
  printf "%d\n", timelocal($sec,$min,$hour,$day,$month,$year);
  exit(0);
} else {
  print { *STDERR } "Invalid date must be \"mmm dd, YYYY HH:MM:SS PM\"\n";
  exit(1);
}

with
Code:
 more a.out
Jan 9, 2013 23:59:59 PM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:220.255.2.108:null:login successful
Jan 9, 2013 0:00:01 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:220.255.2.108:null:login successful
Jan 9, 2013 2:32:29 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:220.255.2.108:null:login successful
Jan 9, 2013 2:34:31 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:116.197.5.39:null:login successful
Jan 9, 2013 2:36:51 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:210.186.188.172:null:login successful
Jan 9, 2013 2:37:00 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:202.162.27.126:null:login successful
Jan 9, 2013 2:37:12 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:220.255.2.128:null:login successful
Jan 9, 2013 2:38:11 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:210.186.241.248:null:login successful
Jan 9, 2013 2:41:06 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:220.255.2.159:null:login successful
Jan 9, 2013 2:42:34 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:124.13.171.171:null:login successful
Jan 9, 2013 2:45:28 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:202.188.41.226:null:login successful
Jan 9, 2013 2:48:08 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:210.195.222.207:null:login successful
Jan 9, 2013 2:49:22 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:60.52.28.59:null:login successful
Jan 9, 2013 2:52:39 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:41.138.175.227:null:login successful
Jan 9, 2013 2:55:36 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:116.197.5.39:null:login successful
Jan 9, 2013 2:56:04 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:175.144.156.85:null:login successful
Jan 9, 2013 2:58:17 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:58.71.157.170:null:login successful
Jan 9, 2013 2:58:18 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:58.71.157.170:null:login successful
Jan 9, 2013 2:58:19 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:58.71.157.170:null:login successful
Jan 9, 2013 3:01:36 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:41.203.67.54:null:login successful
Jan 9, 2013 3:08:25 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:175.136.58.228:null:login successful
Jan 9, 2013 3:17:03 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:41.138.180.156:null:login successful
Jan 9, 2013 3:19:10 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:41.138.180.156:null:login successful
Jan 9, 2013 3:26:57 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:41.138.180.156:null:login successful
Jan 9, 2013 3:28:53 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:49.249.114.228:null:login successful
Jan 9, 2013 3:30:30 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:41.138.180.156:null:login successful
Jan 9, 2013 3:43:08 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:188.52.46.85:null:login successful
Jan 9, 2013 3:47:46 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:58.27.85.20:null:login successful
Jan 9, 2013 3:47:47 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:202.162.27.126:null:login successful
Jan 9, 2013 3:49:22 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:175.140.126.74:null:login successful
Jan 9, 2013 3:49:35 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:58.27.85.20:null:login successful
Jan 9, 2013 3:49:54 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:58.27.85.20:null:login successful
Jan 9, 2013 3:51:45 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:60.50.3.143:null:login successful
Jan 9, 2013 3:52:32 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:175.141.95.105:null:login successful
Jan 9, 2013 3:52:34 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:175.141.95.105:null:login successful
Jan 9, 2013 3:52:36 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:175.141.95.105:null:login successful
Jan 9, 2013 3:54:25 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:175.143.242.152:null:login successful
Jan 9, 2013 3:56:09 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:175.143.242.152:null:login successful
Jan 9, 2013 3:59:38 AM com.sun.uwc.common.auth.IdentitySSOAuthFilter doFilter INFO login:124.13.188.59:null:login successful

result
Code:
bash-3.00# more b.out
116.197.5.39
175.143.242.152
202.162.27.126
220.255.2.108
41.138.180.156
58.27.85.20
bash-3.00# more c.out
175.141.95.105
58.71.157.170

thanks for all the hard work guys... really2 appreciate it
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

If I ran perl script again,old logs should move with today date and new logs should generate.

Appreciate help for the below issue. Im using below code.....I dont want to attach the logs when I ran the perl twice...I just want to take backup with today date and generate new logs...What I need to do for the below scirpt.............. 1)if logs exist it should move the logs with extention... (1 Reply)
Discussion started by: Sanjeev G
1 Replies

2. Shell Programming and Scripting

How to extract logs between the current time and the last 15 minutes ?

I want to extract the logs between the current time stamp and 15 minutes before and sent an email to the people configured. I developed the below script but it's not working properly; can someone help me?? I have a log file containing this pattern: Constructor QuartzJob ... (3 Replies)
Discussion started by: puneetkhullar
3 Replies

3. Shell Programming and Scripting

Perl script to extract last 24 hrs logs from cronlog

Hi Friends, Can anybody help me to create a perl script to generate log file for last 24 hrs from cron log file ?? Thank You (2 Replies)
Discussion started by: syamji.vm
2 Replies

4. Shell Programming and Scripting

Help with extract application logs through shell script in performance testing

Hi Experts, I am new to shell.How to extract logs (Web,APP,Database) using shell in performance testing? Need for webserver logs,app server logs and d/b logs code. Thanks in advance Sree (3 Replies)
Discussion started by: sree vasu
3 Replies

5. Shell Programming and Scripting

Search for a pattern,extract value(s) from next line, extract lines having those extracted value(s)

I have hundreds of files to process. In each file I need to look for a pattern then extract value(s) from next line and then search for value(s) selected from point (2) in the same file at a specific position. HEADER ELECTRON TRANSPORT 18-MAR-98 1A7V TITLE CYTOCHROME... (7 Replies)
Discussion started by: AshwaniSharma09
7 Replies

6. Shell Programming and Scripting

Help needed to extract distinct logs from a Log File

Hi, I urgently need some help how to extract distinct entries from a Log file. The Log File may have same error occuring many times so how do i count the occurance of an error in file and also extract out distinct errors in a file. Eg:- I have a file name A.log it contains entries as below:-... (5 Replies)
Discussion started by: roro
5 Replies

7. Shell Programming and Scripting

Grep yesterday logs from weblogic logs

Hi, I am trying to write a script which would go search and get the info from the logs based on yesterday timestamp and write yesterday logs in new file. The log file format is as follows: """"""""""""""""""""""""""... (3 Replies)
Discussion started by: harish.parker
3 Replies

8. UNIX for Advanced & Expert Users

logs

Hy, I have a question I have a directory in a unix server, Some of my files have a diffrent access time, from the time i accessed them last, I think some one has copied it,it's not an important file,but none the less,it is my file,It mistakenly had a 777 permission( yes ,I know it is a noob's... (1 Reply)
Discussion started by: lordmod
1 Replies

9. Shell Programming and Scripting

Logs

Hey Guys, i am new into shell programming and i have to do one script which have to record all the commands entered by a specific user. Example of that, i have a system running on unix, several users are using this system, i have to create like a databse which will record every user entered that... (5 Replies)
Discussion started by: charbel
5 Replies

10. UNIX for Dummies Questions & Answers

logs

can i include this command into my crontab file > /var/adm/wtmp to clear the contents on a regular basis ? what about file permissions ? (6 Replies)
Discussion started by: cubicle^dweller
6 Replies
Login or Register to Ask a Question