UNIX for Dummies Questions & Answers

Hi forum. I am fairly new to scripting and use a simple script to process e-mails for my work. These e-mails contain a list of IPv4 IPs that I process and seperate into text files, which are then attached to a larger, 'digest' e-mail. I also put some of the output from the text files into the body of the e-mail. The script currently outputs supported and unsupported IPs into the body of the e-mail, with these two lines:

Code:

egrep "^Subject: \[xxx\] xxx$" ~/.maildir/new/* -50 -h \
        | egrep "^`echo $ipregex`$" | egrep -v "`echo $iplist`" | sort -t . -k 1,1n -k 2,2n -k 3,3n -k 4,4n -u \
        | while read LINE ; do nslookup $LINE ; done | grep -i ".in-addr.arpa" \
        | grep -i "name = " | egrep -v "`echo $rdnslist`" | grep -v '^$' | grep -v "canonical" > /tmp/xxx_filtered_rdns-${date}.log

And unsupported:

Code:

egrep "^Subject: \[xxx\] xxx$" ~/.maildir/new/* -50 -h \
        | egrep "^`echo $ipregex`$" | egrep -i "`echo $iplist`" | sort -t . -k 1,1n -k 2,2n -k 3,3n -k 4,4n -u \
        | while read LINE ; do nslookup $LINE ; done | grep -i ".in-addr.arpa" \
        | grep -i "name = " | grep -v '^$' | grep -v "canonical" > /tmp/xxx_unsupported_filtered_rdns-${date}.log

A mess of greps, I'm sure. The script pulls all IP addresses from the e-mail (the IPs reside in the body of the e-mail) via a regexp match (1), either compares or takes a difference to a list of IPs, then goes further and compares the IPs to a list of unsupported reverse DNSes.

The end result is a raw output of a reverse DNS lookup, such as :

Code:

99.207.14.72.in-addr.arpa       name = eh-in-f99.google.com

What I would like to do, while still retaining the functionality I have outlined, is display the IP, count how many occurrences, and put the number in a formatted manner adjacent to the IP.

Ideally, it would also show which ISP the IP belongs to, either by attaching part of a DHCP-style reverse lookup to it, such as which would require a whois lookup.

This is similar to the reverse DNS lookup output I would like to process:

91.120.97.76.in-addr.arpa name = c-76-97-120-91.hsd1.ga.comcast.net

To display only the "ga.comcast.net" next to it, but just showing the "comcast.net" would be sufficient and probably easier, since every IP I process isn't always in a similar format. My scripting abilities are somewhat limited, and while I am pretty handy with grep and it's variants, I know very little of the magic of awk and more specifically, sed. Can this be done?


1:

Code:

(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)

Hi,

Can you post sample input and output files to help understand what you try to achieve?

Output of egrep "^Subject: \[xxx\] xxx$" ~/.maildir/new/* -50 -h would be enough to start with

Here is the output of one match to that grep (it has been obfuscated):

The IP is always at the end of the match/email (the e-mails are text files generated by fetchmail); normally there are multiple IPs, but it's always one IP per line.

- How do you genarate $iplist and $rdnslist
- what occurrences do you want to count?

Can you also give exact output you want.

Both aforementioned variables are partial IP or RDNS matches listed in the script and separated by "|". iplist="94.225.|44.169.|30.45." and rdnslist="joeville.ca.comcast.net|johnville.wa.comcast.net"


The Output I want is:

IP IP_occurance_count AND partial_rdns_name OR whois_ip_owner

So like this:

12.24.255.25 x16 johnville.wa.comcast.net OR just, "comcast"