Unix/Linux Go Back    


Shell Programming and Scripting BSD, Linux, and UNIX shell scripting — Post awk, bash, csh, ksh, perl, php, python, sed, sh, shell scripts, and other shell scripting languages questions here.

Convert ip ranges to CIDR netblocks

Shell Programming and Scripting


Reply    
 
Thread Tools Search this Thread Display Modes
    #8  
Old Unix and Linux 04-27-2017   -   Original Discussion by ripat
rgopichand's Unix or Linux Image
rgopichand rgopichand is offline
Registered User
 
Join Date: Apr 2011
Last Activity: 27 March 2018, 3:49 PM EDT
Posts: 4
Thanks: 0
Thanked 0 Times in 0 Posts
Instead of the first suggestion, I would rather go with second option you mentioned.
Sponsored Links
    #9  
Old Unix and Linux 03-26-2018   -   Original Discussion by ripat
Chubler_XL's Unix or Linux Image
Chubler_XL Chubler_XL is offline Forum Staff  
Moderator
 
Join Date: Oct 2010
Last Activity: 21 June 2018, 11:38 PM EDT
Posts: 3,535
Thanks: 156
Thanked 1,261 Times in 1,155 Posts
Quote:
Originally Posted by rgopichand View Post
Hi, I have list of IP's ~3k, which are from very small to large subnets. So, I want the IPs to be grouped into subnets that makes sense. The scenario is several groups get IP's based on availability and none of the group should not touch or scan the other IP's. We get the list of IPs based on manual inventory from each group and the key to this part is the provider doesn't manage which set of IPs belong to which group.
So the task is I collected manually all the IPs (which are around 3K) and want to make them into subnets to the nearest class. For example if I have a single IP address it should round off to /32 or if it has 4 ip's it should round off to /29 or /30. I have CIDR tools to do this task, but it needs manual input each time.

I'm looking for a way if I put the 3K ip's into excel or any format the script should round off to nearest subnet class.

You could try sorting the IPs in ascending order and continue to add IPs to the subnet until the standard deviation exceeds a limit and rule the subnet of at that point.

Below I've chosen limit at 200, you can play with different values and see how the grouping comes out (somewhere between 100 and 3000 seems fairly good).

Note: just the starting and ending IPs are output for each subnet found, it wouldn't be too complex to determine the closest mask that covers both of these, if needed.



Code:
@include "lib_netaddr.awk"

function sanitize(ip) {
    split(ip, slice, ".")
    return slice[1]/1 "." slice[2]/1 "." slice[3]/1 "." slice[4]/1
}

function grpstd(val, tot, cnt, mean, sqtot) {
    for(val in grp) {
       tot=tot + grp[val]
       cnt++
    }
    mean = tot / cnt
    for(val in grp) {
       sqtot = sqtot + (grp[val] - mean) * (grp[val] - mean)
    }
    return sqrt(sqtot / cnt)
}

BEGIN { limit=200 }

{ k[NR]=ip2dec(sanitize($1)) }

END {
    n=asort(k)

    for(idx=1; idx <= n ; idx++) {
       grp[++have]=k[idx]
       if(grpstd() > limit) {
          print "Subnet from " dec2ip(grp[1]) " to " dec2ip(grp[have-1])
          have=split(grp[have], grp)
       }
    }
    if (have)
          print "Subnet from " dec2ip(grp[1]) " to " dec2ip(grp[have])
}

Also note I'm not a statistician and there are probably much more efficient ways this sort of thing could be achieved.

Last edited by Chubler_XL; 03-26-2018 at 11:16 PM.. Reason: Calculation error in standard deviation function - updated default limit value
Sponsored Links
    #10  
Old Unix and Linux 03-27-2018   -   Original Discussion by ripat
rgopichand's Unix or Linux Image
rgopichand rgopichand is offline
Registered User
 
Join Date: Apr 2011
Last Activity: 27 March 2018, 3:49 PM EDT
Posts: 4
Thanks: 0
Thanked 0 Times in 0 Posts
Thanks, I will try testing and update back.
    #11  
Old Unix and Linux 03-28-2018   -   Original Discussion by ripat
Chubler_XL's Unix or Linux Image
Chubler_XL Chubler_XL is offline Forum Staff  
Moderator
 
Join Date: Oct 2010
Last Activity: 21 June 2018, 11:38 PM EDT
Posts: 3,535
Thanks: 156
Thanked 1,261 Times in 1,155 Posts
Here is an update that takes into account the subnet outer bounds. This reduces the occurrence of IPs belonging to adjacent subnets being swept up.

The output now includes the subnet mask and a count of IP(s) bounded.



Code:
@include "lib_netaddr.awk"

function sanitize(ip) {
    split(ip, slice, ".")
    return slice[1]/1 "." slice[2]/1 "." slice[3]/1 "." slice[4]/1
}

function snbounds(to,i) {
    sn_min=grp[1]
    sn_max=grp[to]

    for(sn_mask=32; sn_mask && sn_min != sn_max; sn_mask--) {
        sn_min = rshift(sn_min,1)
        sn_max = rshift(sn_max,1)
    }

    for(i=32; i>sn_mask; i--) {
        sn_min = lshift(sn_min,1) 
        sn_max = lshift(sn_max,1) + 1
    }
}

function grpstd(val, tot, cnt, mean, sqtot) {
    cnt = length(grp)
    snbounds(cnt)
    tot = sn_min + sn_max
    cnt += 2
    for(val in grp) tot=tot + grp[val]
    mean = tot / cnt
    sqtot = (sn_min - mean) * (sn_min - mean) + \
            (sn_max - mean) * (sn_max - mean)
    for(val in grp) {
       sqtot = sqtot + (grp[val] - mean) * (grp[val] - mean)
    }
    return sqrt(sqtot / cnt)
}

BEGIN { limit=1000 }

{ k[NR]=ip2dec(sanitize($1)) }

END {
    n=asort(k)

    for(idx=1; idx <= n ; idx++) {
       grp[++have]=k[idx]
       # print dec2ip(grp[have]) " std: " grpstd()
       if(grpstd() > limit) {
          snbounds(length(grp)-1)
          print "\nSubnet from " dec2ip(grp[1]) " to " dec2ip(grp[have-1]) " " have - 1 " IP(s)"
          print "Mask " dec2ip(sn_min) "/" sn_mask
          have=split(grp[have], grp)
       }
    }
    if (have) {
          snbounds(length(grp))
          print "\nSubnet from " dec2ip(grp[1]) " to " dec2ip(grp[have]) " " have " IP(s)"
          print "Mask " dec2ip(sn_min) "/" sn_mask
    }
}

Test file example:



Code:
$ cat infile
255.20.19.0
10.10.1.25
10.10.2.16
10.10.1.45
192.168.1.129
192.168.1.166
192.168.1.133
10.10.3.30
192.168.1.188
10.10.3.29
10.10.2.20
220.16.53.1
10.10.3.31
10.10.3.16
$ awk -f rgopichand.awk infile

Subnet from 10.10.1.25 to 10.10.3.31 8 IP(s)
Mask 10.10.0.0/22

Subnet from 192.168.1.129 to 192.168.1.188 4 IP(s)
Mask 192.168.1.128/26

Subnet from 220.16.53.1 to 220.16.53.1 1 IP(s)
Mask 220.16.53.1/32

Subnet from 255.20.19.0 to 255.20.19.0 1 IP(s)
Mask 255.20.19.0/32

Sponsored Links
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
How to convert multiple number ranges into sequence? jcue25 Shell Programming and Scripting 3 11-11-2012 10:32 AM
How to parse IP range in CIDR format in C semash! Programming 3 09-28-2009 11:09 AM
date ranges akil Shell Programming and Scripting 2 07-15-2009 05:50 AM
Get IP list from CIDR Santi Shell Programming and Scripting 10 12-10-2007 12:09 PM



All times are GMT -4. The time now is 05:30 AM.