I seem to have an issue any time I try to use any IP address within 172.16.0.0/12 RFC1918 range in ROS.
I can use anything within 10.0.0.0/8 or 192.168.0.0/16 without any issue.
Any ideas?
(I wrote a longer post detailing the exact scenario in my configuration, but have an issue with this Win10 machine locking up at random today so I lost it all and can’t be bothered to re-type just yet. I’ll edit to add detail again later if required.)
Probably need more details, I’ve used 172.16.0.0/12 without issue for years in RouterOS…what’s the specific configuration giving you issues?
I’m looking to use this addressing with a bridge with no member ports (replicating a loopback interface) and also a GRE tunnel to run GRE over IPsec for OSPF.
I have the setup working fine using different addressing, but it seems to fail if I try to use 172.16.x.x addressing. I’d like to be able to use 172.16.x.x addressing for “loopback” and tunnel addresses as that keeps it clear of 10.x.x.x and 192.168.x.x addressing used in other areas.
Working setup:
R1:
Bridge: 192.168.255.1/30 - ideally, I’d have this as a /32
GRE tunnel interface: 192.168.10.1/30
IPsec policy src 192.168.255.0/30 dst: 192.168.255.4/30
R2:
Bridge: 192.168.255.5/30 - ideally, I’d have this as a /32
GRE: 192.168.10.2/30
IPsec policy src: 192.168.255.4/30 dst: 192.168.255.0/30
(Firewall filter accept rules are all present, plus srcnat action accept & bridge subnets are discarded on route filtering at both ends)
If I replace the bridge IPs with 172.16.0.1/32 and 172.16.0.2/32 respectively (I’ve also tried various combinations within 172.16.x.x as /30 networks) - amending firewall & IPsec as appropriate, if I ping R2 from R1, the firewall output and srcnat rules show hits on R1, but the input counter is 0 on R2. The IPsec phase 2 state is established and I know my is config OK, as it is identical to the working setup besides the addresses used.
I added the srcnat action: accept rules to be safe, but I don’t believe they should be needed as my other srcnat rules for internet access all have source addresses/subnets specified - there are no other rules that would match a 172.16.x.x address
GRE has become a headache generator since the security patch in 6.45.something, as received GRE packets are labeled with connection-state=invalid even if the device sends its own GRE packets in the opposite direction - unless the PPTP helper (/ip firewall service-port) is enabled. So if you can replace the GRE tunnel by IPIP one, it may start working even with the addresses from 172.16.0.0/12. I’m not saying it’s definitely the reason, but I’ve spent hours on these GRE issues, and I’m far from sure I have discovered all the caveats.
It’s not a GRE issue in this case. I haven’t got as far as setting up the GRE tunnel on 172 addresses as I can’t ping the bridge addresses over the IPsec tunnel.
GRE tunnel is working fine with 192.168 addressing.
It is hard to debug things remotely without at least seeing the configuration. When you ping while the 172.x.x.x addresses are used, do you also specify the src-address or you let the machine choose one autonomously?
I agree with that - been there, many times! Yes I do specify the source address… It would be overkill to export the entire configuration I think but anything that I can add to give some context, I’m happy to do so.
The problem is that the issue is always where you don’t suspect it could be - if it was where you expect it, you would find it, right?
So a missed NAT rule, a missed item of some apparently unrelated address-list, an IPsec policy shadowing the one you need… a lot of things may go wrong especially if the configuration is complex. So nothing less than complete configurations from both ends exported while it doesn’t work is useful (of course anonymised, hide-sensitive only suppresses export of passwords/secrets but not of usernames, see my signature).
I’ve just added a 172.16.99.1 <=> 172.16.99.2 policy to one of my running IPsec IKEv2 links, added the respective addresses to interfaces on the routers, and the ping goes through allright. Both are ARM ones and with quite outdated versions (6.45.9 and 6.46.8), but I don’t expect such kind of a bug to be introduced into recent RouterOS releases.
Here’s some relevant R1 config. The R2 config related to the 172.16.x.x addressing is identical but where 172.16.0.1 replace with 172.16.0.2 and where 172.16.0.2 replace with 172.16.0.1
Firewall and IPsec sections are truncated, the others represent complete export.
The GREoIPsec 192.168.5.0/30,192.168.5.4/30,192.168.10.0/30 OSPF area 0 config is working just fine to redistribute connected & other (OSPF area 1) routes. This is what I’m seeking to use 172.16.x.x addressing on.
/ip addresses (includes one further public IP dynamically assigned to PPPoE client WAN interface)
add address=192.168.88.1/24 interface=bridge network=192.168.88.0
add address=10.5.0.254/16 interface=VLAN10 network=10.5.0.0
add address=10.6.0.254/16 interface=VLAN20 network=10.6.0.0
add address=10.8.0.254/16 interface=VLAN60 network=10.8.0.0
add address=10.9.0.254/16 interface=VLAN80 network=10.9.0.0
add address=xx.xx.xx.xx/29 interface=ether3 network=xx.xx.xx.xx
add address=xx.xx.xx.xx/29 interface=ether3 network=xx.xx.xx.xx
add address=xx.xx.xx.xx/29 interface=ether3 network=xx.xx.xx.xx
add address=xx.xx.xx.xx/29 interface=ether3 network=xx.xx.xx.xx
add address=xx.xx.xx.xx/29 interface=ether3 network=xx.xx.xx.xx
add address=xx.xx.xx.xx/29 interface=ether3 network=xx.xx.xx.xx
add address=192.168.2.100/24 interface=ether1 network=192.168.2.0
add address=192.168.5.1/30 interface=VLAN5 network=192.168.5.0
add address=10.8.6.1/16 interface=VLAN60 network=10.8.0.0
add address=192.168.6.1/29 interface=VLAN6 network=192.168.6.0
add address=192.168.32.1/30 interface=ether5 network=192.168.32.0
add address=192.168.10.1/30 interface=ospf_gre network=192.168.10.0
add address=192.168.25.254/24 interface=192_168_25_0 network=192.168.25.0
add address=192.168.255.1/30 interface=GRE_for_OSPF network=192.168.255.0
add address=192.168.15.254/24 interface=VLAN15 network=192.168.15.0
add address=172.16.0.1 interface=Lo0 network=172.16.0.1
/ip firewall filter
add action=accept chain=input dst-address=172.16.0.1 protocol=gre \
src-address=172.16.0.2
add action=accept chain=output dst-address=172.16.0.2 protocol=gre \
src-address=172.16.0.1
add action=accept chain=input dst-address=172.16.0.1 protocol=icmp \
src-address=172.16.0.2
add action=accept chain=output dst-address=172.16.0.2 protocol=icmp \
src-address=172.16.0.1
/ip firewall nat
add action=accept chain=srcnat dst-address=172.16.0.2 src-address=172.16.0.1
/ip ipsec policy
add dst-address=172.16.0.2/32 peer=xx proposal=xx sa-dst-address=\
xx.xx.xx.xx sa-src-address=0.0.0.0 src-address=172.16.0.1/32 tunnel=yes
/routing filters
add action=discard chain=xx-In prefix=0.0.0.0/0
add action=discard chain=xx-In prefix=xx.xx.xx.xx (R2 PPPoE client WAN remote address)
add action=discard chain=xx-In disabled=yes prefix=10.0.0.0/16
add action=discard chain=xx-In prefix=192.168.2.0/24
add action=discard chain=xx-In prefix=192.168.88.0/24
add action=discard chain=xx-In prefix=192.168.255.4/30
add action=discard chain=xx-In prefix=xx.xx.xx.xx/29 (R2 public addresses)
add action=discard chain=xx-Out prefix=0.0.0.0/0
add action=discard chain=xx-Out prefix=xx.xx.xx.xx (R1 PPPoE client WAN remote address)
add action=discard chain=xx-Out disabled=yes prefix=10.5.0.0/16
add action=discard chain=xx-Out prefix=192.168.2.0/24
add action=discard chain=xx-Out prefix=192.168.88.0/24
add action=discard chain=xx-Out prefix=192.168.255.0/30
add action=discard chain=xx-Out prefix=xx.xx.xx.xx/29 (R1 public addresses)
add action=discard chain=yy_In prefix=192.168.88.0/24 (yy_In/Out is OSPF area 1 physically connected to eth5)
add action=discard chain=yy_Out prefix=192.168.88.0/24
add action=discard chain=yy_In prefix=192.168.32.0/30
add action=discard chain=yy_Out prefix=192.168.32.0/30
add action=discard chain=xx-In prefix=172.16.0.2
add action=discard chain=xx-Out prefix=172.16.0.1
/routing ospf
/routing ospf instance
set [ find default=yes ] distribute-default=if-installed-as-type-1 in-filter=\
xx-In name=xx out-filter=xx-Out redistribute-connected=as-type-1 \
redistribute-other-ospf=as-type-1 router-id=192.168.10.1
add distribute-default=always-as-type-1 in-filter=yy_In name=yy out-filter=\
yy_Out redistribute-connected=as-type-1 redistribute-other-ospf=as-type-1 \
router-id=192.168.32.1
/routing ospf area
add area-id=0.0.0.1 instance=yy name=yy
/routing ospf network
add area=yy network=192.168.32.0/30
add area=backbone network=192.168.10.0/30
What does /ip route check 172.16.0.2 show? I.e. is there any route at all (even if the default one) for that destination?
> ip route check 172.16.0.2
status: ok
interface: Uno FTTC
nexthop: 172.16.0.2
& "R2"
> ip route check 172.16.0.1
status: ok
interface: Uno ADSL
nexthop: 172.16.0.1
Sorry, I forgot to state you have to add also src-ip=172.16.0.x (1 or 2) to the /ip route check. What surprises me is the nexthop value, but that should not change the outcome. Hence chances are high that the packet makes it to the stage IPsec policy matching.
So run the ping at R1, and tell me the output of
/ip firewall connection print detail where protocol=icmp dst-address~“172.16.0”
and also the output of
/ip ipsec installed-sa print detail interval=1s where dst-address=the.ip.of.the.remote.peer
If the setup at this end is OK, the number of packets sent should grow by one every second.
So I have a ping running via winbox to destination 172.16.0.2 from source 172.16.0.1
Wasn’t seeing anything in connections via SSH and it wouldn’t accept the syntax on the IPsec command - expected : (line 1 column 79)
However checking firewall connections via Winbox, I do see the ICMP source 172.16.0.1 and destination 172.16.0.2
IPSec installed SAs in Winbox shows 4 (I have 2 x policies with this peer, one is 192.168.255.0/30<->192.168.255.4/30, the other is the 172.16.0.1<->172.16.0.2)
My mistake with the installed-sa, it must be dst-address~“the.ip.of.the.remote.peer” so that the port need not be specified.
What I was interested in was whether the installed-sa counts - assuming the traffic via the other SA (for 192.168…) is intentionally stopped to avoid confusion.
As for the connection, I was interested in the reply-dst-address value, that’s why I’ve asked for print details. In Winbox, you have to double-click the connection to see that.
I’m with you now. 
The reply-dst-address is 172.16.0.1 as expected. Originating connection counters are increasing with ping running, reply counters are 0.
With the installed SAs, I assume it’s one pair per policy hence 4 for 2 policies? That being the case, it looks like the counters are increasing on the pair for the 192.168.255.x subnets, but those associated to the 172.16.x.x counters are not changing. I presume the current byte count (not increasing in either direction) relates to when the tunnel was first established.
/ip firewall connection print detail where protocol=icmp dst-address="172.16.0.2"
Flags: E - expected, S - seen-reply, A - assured, C - confirmed, D - dying,
F - fasttrack, s - srcnat, d - dstnat
0 C protocol=icmp src-address=172.16.0.1 dst-address=172.16.0.2
reply-src-address=172.16.0.2 reply-dst-address=172.16.0.1
icmp-type=8 icmp-code=0 icmp-id=23334 timeout=9s orig-packets=243
orig-bytes=12 150 orig-fasttrack-packets=0 orig-fasttrack-bytes=0
repl-packets=0 repl-bytes=0 repl-fasttrack-packets=0
repl-fasttrack-bytes=0 orig-rate=400bps repl-rate=0bps
> ip ipsec installed-sa print detail interval=1s where dst-address="xx.xx.xx.xx"
Flags: H - hw-aead, A - AH, E - ESP
0 E spi=0x11E09A5 src-address=xx.xx.xx.xx dst-address=xx.xx.xx.xx
state=mature enc-algorithm=aes-gcm enc-key-size=288
enc-key="xx"
addtime=feb/21/2021 11:26:33 expires-in=17m20s
add-lifetime=24m14s/30m18s current-bytes=209839 current-packets=2692
replay=128
These counters are NOT increasing
1 E spi=0x5C968BB src-address=xx.xx.xx.xx dst-address=xx.xx.xx.xx
state=mature enc-algorithm=aes-gcm enc-key-size=288
enc-key="xx"
addtime=feb/21/2021 11:34:09 expires-in=24m48s
add-lifetime=24m8s/30m10s current-bytes=163365 current-packets=2190
replay=128
These counters are increasing
So the connection print shows no hidden src-nat/masquerade rule is the culprit as the reply-dst-address is the same like the src-address.
The thing is that you cannot know which pair of SAs is linked to which policy by any indicator. You would have to disable the policy for 192.168 to be sure. But a question - what level is set on the individual policies, require or unique? And is the level the same at both ends? The thing is that unique causes strict use and checking of the particular SA per policy (so packets which come through a wrong SA are dropped), whereas require uses and allows any of the SAs between the peers to be used. So if this setting doesn’t match between the peers, it could be the explanation.
The pace of packet count growth is another way to see whether it’s the pings that is being forwarded through the SA - if the ping is stopped and the packet count keeps growing when you run the /ip ipsec installed-sa print with interval=1s, it means some other traffic is sent via that SA. If that other traffic is sufficiently low and stable, pinging with interval=30ms might help as it would add 33 more packets per second instead of just one.
The SAs normally get replaced by ones every about 30 minutes by default, so the fact that the silent one did count some packets in the past is suspicious. Some traffic must have been sent down that SA during past 30 minutes at maximum.
OK, so I disabled both policies and killed connections from active peers on both sides.
I then re-enabled only the 172.16.x.x policy on both sides.
The SA from R1 to R2 shows increasing counters on both R1 and R2, however the R2 to R1 counters remain at 0 on both sides.
The Firewall connections do not see the ICMP on R2.
Having sent a couple of 50 byte pings from R1 to R2, the SA counters for 172.16.x.x are (matching both on R1 & R2):
R1 to R2: 3500 bytes
R2 to R1: 48 bytes
With both policies enabled, the above counters do not increase when a second ping from R1 to R2 is initiated.
Both policies on R1 and R2 have level set to require.
So are you saying that the sending SA on R1 and the corresponding receiving SA on R2 count simultaneously as you ping, but nevertheless the firewall rule for icmp at R2 doesn’t?
Yes that’s correct. Also don’t see an entry in the firewall connection tracking on R2.