Visit Our UNIX and Linux User Community


Convert ip ranges to CIDR netblocks


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Convert ip ranges to CIDR netblocks
# 8  
Old 04-27-2017
Instead of the first suggestion, I would rather go with second option you mentioned.
# 9  
Old 03-26-2018
Quote:
Originally Posted by rgopichand
Hi, I have list of IP's ~3k, which are from very small to large subnets. So, I want the IPs to be grouped into subnets that makes sense. The scenario is several groups get IP's based on availability and none of the group should not touch or scan the other IP's. We get the list of IPs based on manual inventory from each group and the key to this part is the provider doesn't manage which set of IPs belong to which group.
So the task is I collected manually all the IPs (which are around 3K) and want to make them into subnets to the nearest class. For example if I have a single IP address it should round off to /32 or if it has 4 ip's it should round off to /29 or /30. I have CIDR tools to do this task, but it needs manual input each time.

I'm looking for a way if I put the 3K ip's into excel or any format the script should round off to nearest subnet class.

You could try sorting the IPs in ascending order and continue to add IPs to the subnet until the standard deviation exceeds a limit and rule the subnet of at that point.

Below I've chosen limit at 200, you can play with different values and see how the grouping comes out (somewhere between 100 and 3000 seems fairly good).

Note: just the starting and ending IPs are output for each subnet found, it wouldn't be too complex to determine the closest mask that covers both of these, if needed.

Code:
@include "lib_netaddr.awk"

function sanitize(ip) {
    split(ip, slice, ".")
    return slice[1]/1 "." slice[2]/1 "." slice[3]/1 "." slice[4]/1
}

function grpstd(val, tot, cnt, mean, sqtot) {
    for(val in grp) {
       tot=tot + grp[val]
       cnt++
    }
    mean = tot / cnt
    for(val in grp) {
       sqtot = sqtot + (grp[val] - mean) * (grp[val] - mean)
    }
    return sqrt(sqtot / cnt)
}

BEGIN { limit=200 }

{ k[NR]=ip2dec(sanitize($1)) }

END {
    n=asort(k)

    for(idx=1; idx <= n ; idx++) {
       grp[++have]=k[idx]
       if(grpstd() > limit) {
          print "Subnet from " dec2ip(grp[1]) " to " dec2ip(grp[have-1])
          have=split(grp[have], grp)
       }
    }
    if (have)
          print "Subnet from " dec2ip(grp[1]) " to " dec2ip(grp[have])
}

Also note I'm not a statistician and there are probably much more efficient ways this sort of thing could be achieved.

Last edited by Chubler_XL; 03-27-2018 at 12:16 AM.. Reason: Calculation error in standard deviation function - updated default limit value
# 10  
Old 03-27-2018
Thanks, I will try testing and update back.
# 11  
Old 03-28-2018
Here is an update that takes into account the subnet outer bounds. This reduces the occurrence of IPs belonging to adjacent subnets being swept up.

The output now includes the subnet mask and a count of IP(s) bounded.

Code:
@include "lib_netaddr.awk"

function sanitize(ip) {
    split(ip, slice, ".")
    return slice[1]/1 "." slice[2]/1 "." slice[3]/1 "." slice[4]/1
}

function snbounds(to,i) {
    sn_min=grp[1]
    sn_max=grp[to]

    for(sn_mask=32; sn_mask && sn_min != sn_max; sn_mask--) {
        sn_min = rshift(sn_min,1)
        sn_max = rshift(sn_max,1)
    }

    for(i=32; i>sn_mask; i--) {
        sn_min = lshift(sn_min,1) 
        sn_max = lshift(sn_max,1) + 1
    }
}

function grpstd(val, tot, cnt, mean, sqtot) {
    cnt = length(grp)
    snbounds(cnt)
    tot = sn_min + sn_max
    cnt += 2
    for(val in grp) tot=tot + grp[val]
    mean = tot / cnt
    sqtot = (sn_min - mean) * (sn_min - mean) + \
            (sn_max - mean) * (sn_max - mean)
    for(val in grp) {
       sqtot = sqtot + (grp[val] - mean) * (grp[val] - mean)
    }
    return sqrt(sqtot / cnt)
}

BEGIN { limit=1000 }

{ k[NR]=ip2dec(sanitize($1)) }

END {
    n=asort(k)

    for(idx=1; idx <= n ; idx++) {
       grp[++have]=k[idx]
       # print dec2ip(grp[have]) " std: " grpstd()
       if(grpstd() > limit) {
          snbounds(length(grp)-1)
          print "\nSubnet from " dec2ip(grp[1]) " to " dec2ip(grp[have-1]) " " have - 1 " IP(s)"
          print "Mask " dec2ip(sn_min) "/" sn_mask
          have=split(grp[have], grp)
       }
    }
    if (have) {
          snbounds(length(grp))
          print "\nSubnet from " dec2ip(grp[1]) " to " dec2ip(grp[have]) " " have " IP(s)"
          print "Mask " dec2ip(sn_min) "/" sn_mask
    }
}

Test file example:

Code:
$ cat infile
255.20.19.0
10.10.1.25
10.10.2.16
10.10.1.45
192.168.1.129
192.168.1.166
192.168.1.133
10.10.3.30
192.168.1.188
10.10.3.29
10.10.2.20
220.16.53.1
10.10.3.31
10.10.3.16
$ awk -f rgopichand.awk infile

Subnet from 10.10.1.25 to 10.10.3.31 8 IP(s)
Mask 10.10.0.0/22

Subnet from 192.168.1.129 to 192.168.1.188 4 IP(s)
Mask 192.168.1.128/26

Subnet from 220.16.53.1 to 220.16.53.1 1 IP(s)
Mask 220.16.53.1/32

Subnet from 255.20.19.0 to 255.20.19.0 1 IP(s)
Mask 255.20.19.0/32


Previous Thread | Next Thread
Test Your Knowledge in Computers #972
Difficulty: Easy
The Linux kernel is written in assembly language.
True or False?

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Convert ip ranges to CIDR netblock

2 scripts to convert IP ranges to CIDR notation using awk, gawk or mawk. The scripts are much faster than using ipcalc and will return the same results. The first script is reliably compatible with awk, gawk and mawk but is over 3 times as slow as the second script which is reliably compatible with... (38 Replies)
Discussion started by: azdps
38 Replies

2. Shell Programming and Scripting

How to change ip addressing format from CIDR notation to netmask and vice versa?

Hi all, I would appreciate if someone could share how to convert CIDR notation to netmask and vice versa. The value below is just an example. it could be different numbers/ip addresses. Initial Output, let say file1.txt Final Output, let say file2.txt (3 Replies)
Discussion started by: type8code0
3 Replies

3. Shell Programming and Scripting

How to convert multiple number ranges into sequence?

Looking for a simple way to convert ranges to a numerical sequence that would assign the original value of the range to the individual numbers that are on the range. Thank you given data 13196-13199 0 13200 4 13201 10 13202-13207 3 13208-13210 7 desired... (3 Replies)
Discussion started by: jcue25
3 Replies

4. Shell Programming and Scripting

Values between ranges

Hi, I have two files file1 chr1_22450_22500 chr2_12300_12350 chr1_34500_34550 file2 11000_13000 15000_19000 33000_44000 If the file 1 ranges fall between file2 ranges then assign the value of file2 in column 2 to file1 output: chr2_12300_12350 11000_13000 chr1_34500_34550 ... (7 Replies)
Discussion started by: Diya123
7 Replies

5. UNIX for Dummies Questions & Answers

Need help filling in ranges

I have a list of about 200,000 lines in a text file that look like this: 1 1 120 1 80 200 1 150 270 5 50 170 5 100 220 5 300 420 The first column is an identifier, the next 2 columns are a range (always 120 value range) I'm trying fill in the values of those ranges, and remove... (4 Replies)
Discussion started by: knott76
4 Replies

6. Programming

How to parse IP range in CIDR format in C

Hello everybody, I'm coding a network program and i need it to "understand" ip ranges, but i don't know how to make to parse an IP CIDR range, let's say "172.16.10.0/24" to work with the specified IP range. I've found a program which does it, but i don't understand the code. Here is the... (3 Replies)
Discussion started by: semash!
3 Replies

7. Shell Programming and Scripting

date ranges

Hi, Please anyone help to achive this using perl or unix scripting . This is date in my table 20090224,based on the date need to check the files,If file exist for that date then increment by 1 for that date and check till max date 'i.e.20090301 and push those files . files1_20090224... (2 Replies)
Discussion started by: akil
2 Replies

8. Shell Programming and Scripting

Get IP list from CIDR

Dear Srs :-) I'm looking for a shell script, that given a network in CIDR format it lists all IPs, for example: Preferredly a shell script, but a Perl, Python, C, etc.. is also welcome :-) I have been looking in sipcalc, ipcalc, etc.. options but this feature is not implemented :-( ... (10 Replies)
Discussion started by: Santi
10 Replies

9. HP-UX

Valid ranges for uids for HP-UX

Hi , I am using adduser in hp-ux to create users in Hp-ux. i would like to know what are the valid values for uids and gids in hp-ux what are the rannges for the valid uids . How to check what are the used uids in Hp-ux . Thanks Narendra babu C (7 Replies)
Discussion started by: naren_chella
7 Replies

Featured Tech Videos