Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Pulling out entries from file based on IP - awk
Top Forums Shell Programming and Scripting Pulling out entries from file based on IP - awk Post 302785315 by SkySmart on Monday 25th of March 2013 10:19:50 AM
Old 03-25-2013
Quote:
Originally Posted by Yoda
I misread your requirement! To find an IP that has 10 or more occurrence:
Code:
awk ' /FAILURE/ {
        match( $0, /[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+\-[0-9]+/ )
        IP = sprintf ("%s", substr($0, RSTART, RLENGTH))
        A[IP]++
} END {
        for ( ip in A )
        {
                if ( A[ip] >= 10 )
                        print "IP: " ip " found " A[ip] " times in the log"
        }
} ' file

BTW that blank line might be due to a record with pattern: FAILURE but no IP address in it.
perfect. this works as expected. i have one more need. the above tells you how many times an ip is found in the file if the port number of the IP is the same.

but i'd also like to alert if an IP is found in the log with different port numbers.

for instance, your code currently alerts on this type of scenario:

Code:
2013-03-23 13:05:55,987  [-46] OracleSensor-10.19.20.111-1207 ERROR util.URIUtils - [PLATFORM.UTIL.E.15] URI syntax
2013-03-23 13:05:55,987  [-46] OracleSensor-10.19.20.111-1207 ERROR util.URIUtils - [PLATFORM.UTIL.E.15] URI syntax
2013-03-23 13:05:55,987  [-46] OracleSensor-10.19.20.111-1207 ERROR util.URIUtils - [PLATFORM.UTIL.E.15] URI syntax
2013-03-23 13:05:55,987  [-46] OracleSensor-10.19.20.111-1207 ERROR util.URIUtils - [PLATFORM.UTIL.E.15] URI syntax
2013-03-23 13:05:55,987  [-46] OracleSensor-10.19.20.111-1207 ERROR util.URIUtils - [PLATFORM.UTIL.E.15] URI syntax
2013-03-23 13:05:55,987  [-46] OracleSensor-10.19.20.111-1207 ERROR util.URIUtils - [PLATFORM.UTIL.E.15] URI syntax
2013-03-23 13:05:55,987  [-46] OracleSensor-10.19.20.111-1207 ERROR util.URIUtils - [PLATFORM.UTIL.E.15] URI syntax
2013-03-23 13:05:55,987  [-46] OracleSensor-10.19.20.111-1207 ERROR util.URIUtils - [PLATFORM.UTIL.E.15] URI syntax
2013-03-23 13:05:55,987  [-46] OracleSensor-10.19.20.111-1207 ERROR util.URIUtils - [PLATFORM.UTIL.E.15] URI syntax
2013-03-23 13:05:55,987  [-46] OracleSensor-10.19.20.111-1207 ERROR util.URIUtils - [PLATFORM.UTIL.E.15] URI syntax

notice the port numbers are all the same.

can it be tweaked to also alert when the ports are different for any of the IPs and the number all these occurrences is 15 or more?:

Code:
2013-03-23 13:05:55,987  [-46] OracleSensor-10.19.24.164-1920 ERROR util.URIUtils - [PLATFORM.UTIL.E.15] URI syntax
2013-03-23 13:05:55,987  [-46] OracleSensor-10.19.24.164-1321 ERROR util.URIUtils - [PLATFORM.UTIL.E.15] URI syntax
2013-03-23 13:05:55,987  [-46] OracleSensor-10.19.24.164-1822 ERROR util.URIUtils - [PLATFORM.UTIL.E.15] URI syntax
2013-03-23 13:05:55,987  [-46] OracleSensor-10.19.24.164-1960 ERROR util.URIUtils - [PLATFORM.UTIL.E.15] URI syntax
2013-03-23 13:05:55,987  [-46] OracleSensor-10.19.24.164-1023 ERROR util.URIUtils - [PLATFORM.UTIL.E.15] URI syntax
2013-03-23 13:05:55,987  [-46] OracleSensor-10.19.24.164-1420 ERROR util.URIUtils - [PLATFORM.UTIL.E.15] URI syntax
2013-03-23 13:05:55,987  [-46] OracleSensor-10.19.24.164-1930 ERROR util.URIUtils - [PLATFORM.UTIL.E.15] URI syntax
2013-03-23 13:05:55,987  [-46] OracleSensor-10.19.24.164-1920 ERROR util.URIUtils - [PLATFORM.UTIL.E.15] URI syntax
2013-03-23 13:05:55,987  [-46] OracleSensor-10.19.24.164-1290 ERROR util.URIUtils - [PLATFORM.UTIL.E.15] URI syntax
2013-03-23 13:05:55,987  [-46] OracleSensor-10.19.24.164-1207 ERROR util.URIUtils - [PLATFORM.UTIL.E.15] URI syntax

 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Pulling out fields from a file

Hi, I have a file that contains 1400 lines similar to the one shown below: NAME=sara, TOWN=southampton, POSTCODE=SO18777, EMAIL=sara@hotmail.com, PASSWORD=asjdflkjds etc etc (note: this is one line). Each line has the same fields, but on each line they are in a different order. Eg. the line... (2 Replies)
Discussion started by: Saz
2 Replies

2. UNIX for Dummies Questions & Answers

pulling the following line from a file

I have return files from a process that has then original input record followed on the next line by a response record..either AA,........... for accepted or EE,.......... for errored. i.e 11,new,123 AA,accepted 12,exist,443 EE,rejected 13,old,223 AA,accepted I want to write a small... (4 Replies)
Discussion started by: peter.herlihy
4 Replies

3. Shell Programming and Scripting

pulling a column from a file in ksh

I would like to pull a column from a file and place it in a variable: The file would look like this: N.Korea gibberish garbage S.Korea gibberish garbage USA gibberish garbage Iraq gibberish garbage Canada gibberish garbage and items in the first... (8 Replies)
Discussion started by: dangral
8 Replies

4. Shell Programming and Scripting

Pulling data and following lines from file

I saw a few posts close to what i want to do, but they didn't look like they would work exactly.. or I need to think out of the box on this. I have a file that I keep server stats in for my own performance analysis. this file has the output from many commands in it (uptime, vmstats, ps, swap... (2 Replies)
Discussion started by: MizzGail
2 Replies

5. UNIX for Dummies Questions & Answers

Pulling a file off a backup tape

I have AIX 5.1 This may sound like a really dumb question but I have never done this before. I would like to pull a file off a backup tape and put back on the AIX is this as simple as as doing a mount /dev/rmt1 then the file name that is on the tape /dump/rpt/xxxxxx Do I just copy it... (14 Replies)
Discussion started by: rocker40
14 Replies

6. Shell Programming and Scripting

Pulling Ip's from log and redirecting to a file

Hi all, I am fairly new to scripting, but I do try and script as much as possible but the more advanced stuff does tend to boggle my mind a bit. I am at a bit of a loss with this one. I get entries in my DNS logs, like the below: I want to extract only the IP address, without the hashes... (5 Replies)
Discussion started by: codenjanod
5 Replies

7. Shell Programming and Scripting

pulling different fields from a csv file

Hi, I have a requirment where I need to pull different columns from a .csv file. Here is the sample of the csv file. account,item,flag1,flag2,flag3,flag4,flag5,......feed,tran I will be have a config.txt file which will have the following information. item,flag5,flag10,feed,tran... (2 Replies)
Discussion started by: akdevula
2 Replies

8. UNIX for Dummies Questions & Answers

Pulling Parms from Config File

Hello all, I'm working on a general script for something at work. I'm an up-and-comer backup for a Shell Scripter this company has had for 35 years lol. Anyway, I have a config file I'm trying to pull Variables from as the Config File is used for multiple scripts. Does the below make sense and... (7 Replies)
Discussion started by: phunk
7 Replies

9. Shell Programming and Scripting

Pulling Data, Then Moving to the Next File

I'm scanning a list of emails- I need to pull 2 pieces of data, then move to the next file: Sender's Email Address Email Date I need these to be outputted into a single column- separated by a ",". Like this: Email1's Address, Email1's Date Stamp Email2's Address, Email2's Date Stamp... (4 Replies)
Discussion started by: sudo
4 Replies

10. Shell Programming and Scripting

Partial file pulling

I am connecting to another server through sftp. I am running one batch script to pull file from another server. sometimes i am receiving partial files. I am using below commands in batch script. ls -ltr new.txt mget new.txt bye The file is of 1 MB only.In most of the cases , i received... (6 Replies)
Discussion started by: srinath01
6 Replies
All times are GMT -4. The time now is 03:13 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy