How to opitimize list of IP4 addresses

RouterOS Useful user articles
1

I was thinking how to optimize big IP lists before importing them to Mikrotik. It ended as this program.
Feel free to use it. Comments welcome.

Written with GNU Linux and gcc. Standard usage … takes data from stdin and outputs to stdout

Program tries to merge consecutive IP addresses or IP ranges.
This list

10.10.10.4
10.10.10.5
10.10.10.6
10.10.10.7
192.168.1.128/32
192.168.1.129/32
192.168.1.130
192.168.1.131
192.168.1.132
192.168.1.133
192.168.1.134/32
192.168.1.135/32
192.168.1.136/29
192.168.2.128/31
192.168.2.130
192.168.2.131
192.168.2.132
192.168.2.133
192.168.2.134/32
192.168.2.135/32
192.168.2.136/29
192.168.2.144/28
172.16.1.1
1.1.1.1
8.8.8.8
9.9.9.9
172.16.1.2
172.16.2.0
172.16.2.1
172.16.1.3

is optimized to

192.168.2.128/27
192.168.1.128/28
10.10.10.4/30
172.16.1.2/31
172.16.2.0/31
1.1.1.1
8.8.8.8
9.9.9.9
172.16.1.1

Source:
optimizeip.c (7.9 KB)
Exapmple ip list:
testip.txt (464 Bytes)
Optimized list:
optimized.txt (178 Bytes)

Importing IP List from file
Under flood attack, how resolve this ?
2

I think it is not working 100% right.

Example. Llet’s get all facebook IPv4 address ranges and process them with your program:

$ (for orig in AS32934 AS63293 AS54115; do whois -h whois.radb.net -- "-i origin $orig"; done) | grep route: | awk '{print $2}' >facebook4.txt
$ gcc -o optimizeip optimizeip.c
$ ./optimizeip <facebook4.txt | grep 129
129.134.0.0/16
129.134.0.0/17
129.134.128.0/23

I still have had no time to study the code to look for the bug, but I wanted to warn you of this behaviour. The way I understand it 129.134.0.0/16 should be the only one left in the output

3

Thank you for the report.
It is example of situation when one subnet is fully included in another. I do not look for such optimization … yet :slight_smile:
IMHO it is not “a bug” .. output is fully valid however not optimized to “deep roots”.

4

This tool exists http://manpages.ubuntu.com/manpages/disco/en/man1/aggregate.1.html

5

How many possible host addresses are available for an IPv4 address?

6

Depends on how IPv4 is divided to subnets. Theoretical that’s 2^32 addresses (slightly less than 4.210^9). Some are used for network addresses (and same number for broadcast addresses) - exact number depends on sizes of subnets and with CIDR addressing you can’t know how some particular address range might be divided into subnets just by look of it. Some addresses are private (10/8, 172.16/12, 192.168/16) and don’t really count. Some addresses are used for multicasts (224/4) and don’t count either. My guess is that realistic number is around 3.9 billion (3.910^9) of IPv4 host addresses. Give or take a few millions.

7

Also you can use https://github.com/snar/bgpq3, for example:

bgpq3 -A -4 -j AS-FACEBOOK

Here I’m using JSON output format so that I can have access to a prefix-length range, but you could equally use

bgpq3 -A -4 AS-FACEBOOK

and get a Cisco-style output:

ip prefix-list NN permit 31.13.24.0/21
ip prefix-list NN permit 31.13.64.0/18 le 19
ip prefix-list NN permit 31.13.64.0/19 ge 24 le 24
ip prefix-list NN permit 45.64.40.0/22
ip prefix-list NN permit 66.111.48.0/22
ip prefix-list NN permit 66.111.48.0/23 ge 24 le 24
…snip…
ip prefix-list NN permit 199.201.64.0/22
ip prefix-list NN permit 199.201.64.0/22 ge 24 le 24
ip prefix-list NN permit 204.15.20.0/22
8

search tag # rextended short address list ip address aggregator

Thanks for this share!!!

✂ Rextended Fragments of Snippets