172.16.0.0/12 RFC1918 in ROS

sjoram · Sat Feb 20, 2021 7:10 pm

I seem to have an issue any time I try to use any IP address within 172.16.0.0/12 RFC1918 range in ROS.
I can use anything within 10.0.0.0/8 or 192.168.0.0/16 without any issue.

Any ideas?

(I wrote a longer post detailing the exact scenario in my configuration, but have an issue with this Win10 machine locking up at random today so I lost it all and can't be bothered to re-type just yet. I'll edit to add detail again later if required.)

StubArea51 · Sat Feb 20, 2021 8:07 pm

Probably need more details, I've used 172.16.0.0/12 without issue for years in RouterOS...what's the specific configuration giving you issues?

sjoram · Sat Feb 20, 2021 11:05 pm

I'm looking to use this addressing with a bridge with no member ports (replicating a loopback interface) and also a GRE tunnel to run GRE over IPsec for OSPF.
I have the setup working fine using different addressing, but it seems to fail if I try to use 172.16.x.x addressing. I'd like to be able to use 172.16.x.x addressing for "loopback" and tunnel addresses as that keeps it clear of 10.x.x.x and 192.168.x.x addressing used in other areas.

Working setup:
R1:
Bridge: 192.168.255.1/30 - ideally, I'd have this as a /32
GRE tunnel interface: 192.168.10.1/30
IPsec policy src 192.168.255.0/30 dst: 192.168.255.4/30
R2:
Bridge: 192.168.255.5/30 - ideally, I'd have this as a /32
GRE: 192.168.10.2/30
IPsec policy src: 192.168.255.4/30 dst: 192.168.255.0/30

(Firewall filter accept rules are all present, plus srcnat action accept & bridge subnets are discarded on route filtering at both ends)

If I replace the bridge IPs with 172.16.0.1/32 and 172.16.0.2/32 respectively (I've also tried various combinations within 172.16.x.x as /30 networks) - amending firewall & IPsec as appropriate, if I ping R2 from R1, the firewall output and srcnat rules show hits on R1, but the input counter is 0 on R2. The IPsec phase 2 state is established and I know my is config OK, as it is identical to the working setup besides the addresses used.
I added the srcnat action: accept rules to be safe, but I don't believe they should be needed as my other srcnat rules for internet access all have source addresses/subnets specified - there are no other rules that would match a 172.16.x.x address

sindy · Sat Feb 20, 2021 11:31 pm

GRE has become a headache generator since the security patch in 6.45.something, as received GRE packets are labeled with connection-state=invalid even if the device sends its own GRE packets in the opposite direction - unless the PPTP helper (/ip firewall service-port) is enabled. So if you can replace the GRE tunnel by IPIP one, it may start working even with the addresses from 172.16.0.0/12. I'm not saying it's definitely the reason, but I've spent hours on these GRE issues, and I'm far from sure I have discovered all the caveats.

sjoram · Sun Feb 21, 2021 9:43 am

It's not a GRE issue in this case. I haven't got as far as setting up the GRE tunnel on 172 addresses as I can't ping the bridge addresses over the IPsec tunnel.

GRE tunnel is working fine with 192.168 addressing.

sindy · Sun Feb 21, 2021 10:46 am

It is hard to debug things remotely without at least seeing the configuration. When you ping while the 172.x.x.x addresses are used, do you also specify the src-address or you let the machine choose one autonomously?

sjoram · Sun Feb 21, 2021 11:20 am

It is hard to debug things remotely without at least seeing the configuration. When you ping while the 172.x.x.x addresses are used, do you also specify the src-address or you let the machine choose one autonomously?

I agree with that - been there, many times! Yes I do specify the source address.... It would be overkill to export the entire configuration I think but anything that I can add to give some context, I'm happy to do so.

sindy · Sun Feb 21, 2021 11:27 am

It would be overkill to export the entire configuration I think

The problem is that the issue is always where you don't suspect it could be - if it was where you expect it, you would find it, right?

So a missed NAT rule, a missed item of some apparently unrelated address-list, an IPsec policy shadowing the one you need... a lot of things may go wrong especially if the configuration is complex. So nothing less than complete configurations from both ends exported while it doesn't work is useful (of course anonymised, hide-sensitive only suppresses export of passwords/secrets but not of usernames, see my signature).

sindy · Sun Feb 21, 2021 11:41 am

I've just added a 172.16.99.1 <=> 172.16.99.2 policy to one of my running IPsec IKEv2 links, added the respective addresses to interfaces on the routers, and the ping goes through allright. Both are ARM ones and with quite outdated versions (6.45.9 and 6.46.8), but I don't expect such kind of a bug to be introduced into recent RouterOS releases.

sjoram · Sun Feb 21, 2021 11:51 am

Here's some relevant R1 config. The R2 config related to the 172.16.x.x addressing is identical but where 172.16.0.1 replace with 172.16.0.2 and where 172.16.0.2 replace with 172.16.0.1
Firewall and IPsec sections are truncated, the others represent complete export.
The GREoIPsec 192.168.5.0/30,192.168.5.4/30,192.168.10.0/30 OSPF area 0 config is working just fine to redistribute connected & other (OSPF area 1) routes. This is what I'm seeking to use 172.16.x.x addressing on.

/ip addresses (includes one further public IP dynamically assigned to PPPoE client WAN interface)
add address=192.168.88.1/24 interface=bridge network=192.168.88.0
add address=10.5.0.254/16 interface=VLAN10 network=10.5.0.0
add address=10.6.0.254/16 interface=VLAN20 network=10.6.0.0
add address=10.8.0.254/16 interface=VLAN60 network=10.8.0.0
add address=10.9.0.254/16 interface=VLAN80 network=10.9.0.0
add address=xx.xx.xx.xx/29 interface=ether3 network=xx.xx.xx.xx
add address=xx.xx.xx.xx/29 interface=ether3 network=xx.xx.xx.xx
add address=xx.xx.xx.xx/29 interface=ether3 network=xx.xx.xx.xx
add address=xx.xx.xx.xx/29 interface=ether3 network=xx.xx.xx.xx
add address=xx.xx.xx.xx/29 interface=ether3 network=xx.xx.xx.xx
add address=xx.xx.xx.xx/29 interface=ether3 network=xx.xx.xx.xx
add address=192.168.2.100/24 interface=ether1 network=192.168.2.0
add address=192.168.5.1/30 interface=VLAN5 network=192.168.5.0
add address=10.8.6.1/16 interface=VLAN60 network=10.8.0.0
add address=192.168.6.1/29 interface=VLAN6 network=192.168.6.0
add address=192.168.32.1/30 interface=ether5 network=192.168.32.0
add address=192.168.10.1/30 interface=ospf_gre network=192.168.10.0
add address=192.168.25.254/24 interface=192_168_25_0 network=192.168.25.0
add address=192.168.255.1/30 interface=GRE_for_OSPF network=192.168.255.0
add address=192.168.15.254/24 interface=VLAN15 network=192.168.15.0
add address=172.16.0.1 interface=Lo0 network=172.16.0.1
/ip firewall filter
add action=accept chain=input dst-address=172.16.0.1 protocol=gre \
    src-address=172.16.0.2
add action=accept chain=output dst-address=172.16.0.2 protocol=gre \
    src-address=172.16.0.1
add action=accept chain=input dst-address=172.16.0.1 protocol=icmp \
    src-address=172.16.0.2
add action=accept chain=output dst-address=172.16.0.2 protocol=icmp \
    src-address=172.16.0.1
/ip firewall nat
add action=accept chain=srcnat dst-address=172.16.0.2 src-address=172.16.0.1
/ip ipsec policy
add dst-address=172.16.0.2/32 peer=xx proposal=xx sa-dst-address=\
    xx.xx.xx.xx sa-src-address=0.0.0.0 src-address=172.16.0.1/32 tunnel=yes
/routing filters
add action=discard chain=xx-In prefix=0.0.0.0/0
add action=discard chain=xx-In prefix=xx.xx.xx.xx (R2 PPPoE client WAN remote address)
add action=discard chain=xx-In disabled=yes prefix=10.0.0.0/16
add action=discard chain=xx-In prefix=192.168.2.0/24
add action=discard chain=xx-In prefix=192.168.88.0/24
add action=discard chain=xx-In prefix=192.168.255.4/30
add action=discard chain=xx-In prefix=xx.xx.xx.xx/29 (R2 public addresses)
add action=discard chain=xx-Out prefix=0.0.0.0/0
add action=discard chain=xx-Out prefix=xx.xx.xx.xx (R1 PPPoE client WAN remote address)
add action=discard chain=xx-Out disabled=yes prefix=10.5.0.0/16
add action=discard chain=xx-Out prefix=192.168.2.0/24
add action=discard chain=xx-Out prefix=192.168.88.0/24
add action=discard chain=xx-Out prefix=192.168.255.0/30
add action=discard chain=xx-Out prefix=xx.xx.xx.xx/29 (R1 public addresses)
add action=discard chain=yy_In prefix=192.168.88.0/24 (yy_In/Out is OSPF area 1 physically connected to eth5)
add action=discard chain=yy_Out prefix=192.168.88.0/24
add action=discard chain=yy_In prefix=192.168.32.0/30
add action=discard chain=yy_Out prefix=192.168.32.0/30
add action=discard chain=xx-In prefix=172.16.0.2
add action=discard chain=xx-Out prefix=172.16.0.1
/routing ospf
/routing ospf instance
set [ find default=yes ] distribute-default=if-installed-as-type-1 in-filter=\
    xx-In name=xx out-filter=xx-Out redistribute-connected=as-type-1 \
    redistribute-other-ospf=as-type-1 router-id=192.168.10.1
add distribute-default=always-as-type-1 in-filter=yy_In name=yy out-filter=\
    yy_Out redistribute-connected=as-type-1 redistribute-other-ospf=as-type-1 \
    router-id=192.168.32.1
/routing ospf area
add area-id=0.0.0.1 instance=yy name=yy
/routing ospf network
add area=yy network=192.168.32.0/30
add area=backbone network=192.168.10.0/30

sindy · Sun Feb 21, 2021 12:01 pm

What does /ip route check 172.16.0.2 show? I.e. is there any route at all (even if the default one) for that destination?

sjoram · Sun Feb 21, 2021 12:21 pm

What does /ip route check 172.16.0.2 show? I.e. is there any route at all (even if the default one) for that destination?

> ip route check 172.16.0.2
     status: ok
  interface: Uno FTTC
    nexthop: 172.16.0.2
& "R2"
> ip route check 172.16.0.1
     status: ok
  interface: Uno ADSL
    nexthop: 172.16.0.1

sindy · Sun Feb 21, 2021 12:32 pm

Sorry, I forgot to state you have to add also src-ip=172.16.0.x (1 or 2) to the /ip route check. What surprises me is the nexthop value, but that should not change the outcome. Hence chances are high that the packet makes it to the stage IPsec policy matching.

So run the ping at R1, and tell me the output of
/ip firewall connection print detail where protocol=icmp dst-address~"172.16.0"

and also the output of
/ip ipsec installed-sa print detail interval=1s where dst-address=the.ip.of.the.remote.peer

If the setup at this end is OK, the number of packets sent should grow by one every second.

sjoram · Sun Feb 21, 2021 1:02 pm

So I have a ping running via winbox to destination 172.16.0.2 from source 172.16.0.1

Wasn't seeing anything in connections via SSH and it wouldn't accept the syntax on the IPsec command - expected : (line 1 column 79)

However checking firewall connections via Winbox, I do see the ICMP source 172.16.0.1 and destination 172.16.0.2
IPSec installed SAs in Winbox shows 4 (I have 2 x policies with this peer, one is 192.168.255.0/30<->192.168.255.4/30, the other is the 172.16.0.1<->172.16.0.2)

sindy · Sun Feb 21, 2021 1:12 pm

My mistake with the installed-sa, it must be dst-address~"the.ip.of.the.remote.peer" so that the port need not be specified.
What I was interested in was whether the installed-sa counts - assuming the traffic via the other SA (for 192.168...) is intentionally stopped to avoid confusion.

As for the connection, I was interested in the reply-dst-address value, that's why I've asked for print details. In Winbox, you have to double-click the connection to see that.

sjoram · Sun Feb 21, 2021 1:43 pm

I'm with you now. :-)

The reply-dst-address is 172.16.0.1 as expected. Originating connection counters are increasing with ping running, reply counters are 0.

With the installed SAs, I assume it's one pair per policy hence 4 for 2 policies? That being the case, it looks like the counters are increasing on the pair for the 192.168.255.x subnets, but those associated to the 172.16.x.x counters are not changing. I presume the current byte count (not increasing in either direction) relates to when the tunnel was first established.

/ip firewall connection print detail where protocol=icmp dst-address="172.16.0.2"
Flags: E - expected, S - seen-reply, A - assured, C - confirmed, D - dying,
F - fasttrack, s - srcnat, d - dstnat
 0    C     protocol=icmp src-address=172.16.0.1 dst-address=172.16.0.2
            reply-src-address=172.16.0.2 reply-dst-address=172.16.0.1
            icmp-type=8 icmp-code=0 icmp-id=23334 timeout=9s orig-packets=243
            orig-bytes=12 150 orig-fasttrack-packets=0 orig-fasttrack-bytes=0
            repl-packets=0 repl-bytes=0 repl-fasttrack-packets=0
            repl-fasttrack-bytes=0 orig-rate=400bps repl-rate=0bps

> ip ipsec installed-sa print detail interval=1s where dst-address="xx.xx.xx.xx"
Flags: H - hw-aead, A - AH, E - ESP
 0  E spi=0x11E09A5 src-address=xx.xx.xx.xx dst-address=xx.xx.xx.xx
      state=mature enc-algorithm=aes-gcm enc-key-size=288
      enc-key="xx"
      addtime=feb/21/2021 11:26:33 expires-in=17m20s
      add-lifetime=24m14s/30m18s current-bytes=209839 current-packets=2692
      replay=128

These counters are NOT increasing

 1  E spi=0x5C968BB src-address=xx.xx.xx.xx dst-address=xx.xx.xx.xx
      state=mature enc-algorithm=aes-gcm enc-key-size=288
      enc-key="xx"
      addtime=feb/21/2021 11:34:09 expires-in=24m48s
      add-lifetime=24m8s/30m10s current-bytes=163365 current-packets=2190
      replay=128

These counters are increasing

sindy · Sun Feb 21, 2021 2:51 pm

So the connection print shows no hidden src-nat/masquerade rule is the culprit as the reply-dst-address is the same like the src-address.

With the installed SAs, I assume it's one pair per policy hence 4 for 2 policies? That being the case, it looks like the counters are increasing on the pair for the 192.168.255.x subnets, but those associated to the 172.16.x.x counters are not changing.

The thing is that you cannot know which pair of SAs is linked to which policy by any indicator. You would have to disable the policy for 192.168 to be sure. But a question - what level is set on the individual policies, require or unique? And is the level the same at both ends? The thing is that unique causes strict use and checking of the particular SA per policy (so packets which come through a wrong SA are dropped), whereas require uses and allows any of the SAs between the peers to be used. So if this setting doesn't match between the peers, it could be the explanation.

The pace of packet count growth is another way to see whether it's the pings that is being forwarded through the SA - if the ping is stopped and the packet count keeps growing when you run the /ip ipsec installed-sa print with interval=1s, it means some other traffic is sent via that SA. If that other traffic is sufficiently low and stable, pinging with interval=30ms might help as it would add 33 more packets per second instead of just one.

I presume the current byte count (not increasing in either direction) relates to when the tunnel was first established.

The SAs normally get replaced by ones every about 30 minutes by default, so the fact that the silent one did count some packets in the past is suspicious. Some traffic must have been sent down that SA during past 30 minutes at maximum.

sjoram · Sun Feb 21, 2021 3:30 pm

OK, so I disabled both policies and killed connections from active peers on both sides.
I then re-enabled only the 172.16.x.x policy on both sides.
The SA from R1 to R2 shows increasing counters on both R1 and R2, however the R2 to R1 counters remain at 0 on both sides.
The Firewall connections do not see the ICMP on R2.
Having sent a couple of 50 byte pings from R1 to R2, the SA counters for 172.16.x.x are (matching both on R1 & R2):
R1 to R2: 3500 bytes
R2 to R1: 48 bytes
With both policies enabled, the above counters do not increase when a second ping from R1 to R2 is initiated.
Both policies on R1 and R2 have level set to require.

sindy · Sun Feb 21, 2021 3:44 pm

So are you saying that the sending SA on R1 and the corresponding receiving SA on R2 count simultaneously as you ping, but nevertheless the firewall rule for icmp at R2 doesn't?

sjoram · Sun Feb 21, 2021 4:00 pm

Yes that's correct. Also don't see an entry in the firewall connection tracking on R2.

sindy · Sun Feb 21, 2021 9:00 pm

Ok.

If you run /tool sniffer quick ip-address=172.16.0.1 on R2 while pinging from R1, does it show anything?
Are the /ip firewall filter rules you've posted above the only ones in that table?
Is the IPsec policy for 172.16.0.1<->172.16.0.2 the only one at R2 or are there any other policies which are not disabled?

sjoram · Sun Feb 21, 2021 9:58 pm

> /tool sniffer quick ip-address=172.16.0.1
INTERFACE TIME NUM DI SRC-MAC DST-MAC VLAN
Uno ADSL 19.287 1 <-
Uno ADSL 20.291 2 <-
...you get the idea

No, R1 has a fairly large firewall table, R2 is a lot simpler. However I've gone over the filter and NAT lists on both R1 & R2 several times looking for anything that could be tripping it up to no avail (and I think we've already ruled out a NAT issue). On R1, the relevant filter icmp in/out rules are position 143 & 144, R2 113 & 114. I don't have logging enabled on the rules as I haven't yet found a clean way to do that which doesn't clog up the logs or have them grow to a ridiculous size.

There are only two IPsec policies on both R1 and R2. The 172.16.0.x and 192.168.255.x - the latter of which is currently working.

If needs be, I can post the filter rulesets, but it'll take me a while to redact public IPs, so if it's needed, I won't do that tonight...

sindy · Sun Feb 21, 2021 10:06 pm

> /tool sniffer quick ip-address=172.16.0.1
INTERFACE TIME NUM DI SRC-MAC DST-MAC VLAN
Uno ADSL 19.287 1 <-
Uno ADSL 20.291 2 <-
...you get the idea

Yes, this is enough, so the transport packets get received and the payload ones are extracted from them allright.

If needs be, I can post the filter rulesets, but it'll take me a while to redact public IPs, so if it's needed, I won't do that tonight...

It need not. Just put a rule chain=input dst-address=172.16.0.2 action=passthrough as the very first static one in chain input of filter,chain=filter dst-address=172.16.0.2 action=passthrough as the very first static one in chain forward of filter, and chain=dstnat dst-address=172.16.0.2 action=passthrough as the very first static one in chain dstnat of nat, all three on R2, do some pinging from R1 and see whether any of the three rules has counted.

sjoram · Sun Feb 21, 2021 11:28 pm

With those rules in place, input and dstnat counters are both increasing.
I really can't see anything in the existing rule base that would be tripping it up, but I guess I should just move those down through the rule base and re-try and see where they place in the list when the counters stop increasing...

sindy · Sun Feb 21, 2021 11:37 pm

Exactly. This result illustrates that the packet reaches the input chain of filter as expected, and some rule in there drops it.

sjoram · Sun Feb 21, 2021 11:55 pm

Thanks, really appreciate your time & input on this thread.
I've found the problem & it's one of those "how did I miss that, I'm sure I checked for that" moments....
I have drop rules at the top of the filter list for input and forward chains for anything with a source address contained within several dynamic blacklists imported from external sources every 3 hours, one of which lists 'bogons'. My import script excludes 10.0.0.0/8 and 192.168.0.0/16 from those lists, but not 172.16.0.0/12.......

Quite literally the first enabled rule in my filter list, never mind that chain!

add action=drop chain=input comment="Firehol L1 Blacklist" src-address-list=firehol_L1

I've now edited this section in the Powershell script that creates the ROS script - I think you can see what was missing!:

# Create Firehol L1 import script
$script= "C:\inetpub\wwwroot\blacklists\firehol1.rsc"
"/ip firewall address-list remove [/ip firewall address-list find list=firehol_L1]" | Add-Content $script
$blacklist= Get-Content "C:\inetpub\wwwroot\blacklists\firehol_L1.txt" | Where-Object {$_ -notmatch "^#|0.0.0.0|10.0.0.0|172.16.0.0/12|192.168.0.0|224.0.0.0"}
ForEach ($address in $blacklist){
"/ip firewall address-list add addres=$address list=firehol_L1 timeout=2d" | Add-Content $script
}

172.16.0.0/12 RFC1918 in ROS [SOLVED]

172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS

Re: 172.16.0.0/12 RFC1918 in ROS [SOLVED]

Who is online