Page 1 of 1

How to opitimize list of IP4 addresses

Posted: Thu May 02, 2019 11:43 pm
by BartoszP
I was thinking how to optimize big IP lists before importing them to Mikrotik. It ended as this program.
Feel free to use it. Comments welcome.

Written with GNU Linux and gcc. Standard usage ... takes data from stdin and outputs to stdout

Program tries to merge consecutive IP addresses or IP ranges.
This list
10.10.10.4
10.10.10.5
10.10.10.6
10.10.10.7
192.168.1.128/32
192.168.1.129/32
192.168.1.130
192.168.1.131
192.168.1.132
192.168.1.133
192.168.1.134/32
192.168.1.135/32
192.168.1.136/29
192.168.2.128/31
192.168.2.130
192.168.2.131
192.168.2.132
192.168.2.133
192.168.2.134/32
192.168.2.135/32
192.168.2.136/29
192.168.2.144/28
172.16.1.1
1.1.1.1
8.8.8.8
9.9.9.9
172.16.1.2
172.16.2.0
172.16.2.1
172.16.1.3
is optimized to
192.168.2.128/27
192.168.1.128/28
10.10.10.4/30
172.16.1.2/31
172.16.2.0/31
1.1.1.1
8.8.8.8
9.9.9.9
172.16.1.1
Source:
optimizeip.c
Exapmple ip list:
testip.txt
Optimized list:
optimized.txt

Re: How to opitimize list of IP4 addresses

Posted: Sat May 04, 2019 5:53 pm
by nostromog
I think it is not working 100% right.

Example. Llet's get all facebook IPv4 address ranges and process them with your program:
$ (for orig in AS32934 AS63293 AS54115; do whois -h whois.radb.net -- "-i origin $orig"; done) | grep route: | awk '{print $2}' >facebook4.txt
$ gcc -o optimizeip optimizeip.c
$ ./optimizeip <facebook4.txt | grep 129
129.134.0.0/16
129.134.0.0/17
129.134.128.0/23
I still have had no time to study the code to look for the bug, but I wanted to warn you of this behaviour. The way I understand it 129.134.0.0/16 should be the only one left in the output

Re: How to opitimize list of IP4 addresses

Posted: Sat May 04, 2019 8:09 pm
by BartoszP
Thank you for the report.
It is example of situation when one subnet is fully included in another. I do not look for such optimization ... yet :)
IMHO it is not "a bug" .. output is fully valid however not optimized to "deep roots".

Re: How to opitimize list of IP4 addresses

Posted: Fri May 17, 2019 10:42 pm
by ludvik

Re: How to opitimize list of IP4 addresses

Posted: Tue May 28, 2019 10:59 am
by taichinhhanaHN
How many possible host addresses are available for an IPv4 address?

Re: How to opitimize list of IP4 addresses

Posted: Tue May 28, 2019 11:16 am
by mkx
How many possible host addresses are available for an IPv4 address?
Depends on how IPv4 is divided to subnets. Theoretical that's 2^32 addresses (slightly less than 4.2*10^9). Some are used for network addresses (and same number for broadcast addresses) - exact number depends on sizes of subnets and with CIDR addressing you can't know how some particular address range might be divided into subnets just by look of it. Some addresses are private (10/8, 172.16/12, 192.168/16) and don't really count. Some addresses are used for multicasts (224/4) and don't count either. My guess is that realistic number is around 3.9 billion (3.9*10^9) of IPv4 host addresses. Give or take a few millions.

Re: How to opitimize list of IP4 addresses

Posted: Tue May 28, 2019 12:33 pm
by maznu
Also you can use https://github.com/snar/bgpq3, for example:
bgpq3 -A -4 -j AS-FACEBOOK
Here I'm using JSON output format so that I can have access to a prefix-length range, but you could equally use
bgpq3 -A -4 AS-FACEBOOK
and get a Cisco-style output:
ip prefix-list NN permit 31.13.24.0/21
ip prefix-list NN permit 31.13.64.0/18 le 19
ip prefix-list NN permit 31.13.64.0/19 ge 24 le 24
ip prefix-list NN permit 45.64.40.0/22
ip prefix-list NN permit 66.111.48.0/22
ip prefix-list NN permit 66.111.48.0/23 ge 24 le 24
…snip…
ip prefix-list NN permit 199.201.64.0/22
ip prefix-list NN permit 199.201.64.0/22 ge 24 le 24
ip prefix-list NN permit 204.15.20.0/22

Re: How to opitimize list of IP4 addresses

Posted: Fri Jun 25, 2021 6:37 pm
by rextended
search tag # rextended short address list ip address aggregator

Thanks for this share!!!