I’m in the process of replacing as much of my home network and lab gear with greener gear (quieter, low powered, smaller form factor, etc). A colleague and I had a GRE tunnel between 2 Cisco routers, connecting our home labs. I’ve been struggling with migrating my end of that tunnel over to an RB4011 running routeros v6.46.1.
Here's a somewhat sanitized copy of the configs:
! cisco router gre tunnel side
interface Tunnel1
ip address 192.168.13.1 255.255.255.252
keepalive 10 3
tunnel source cisco_router_public_ip
tunnel destination mikrotik_router_public_ip
! mikrotik router gre tunnel side
/interface gre
add allow-fast-path=no clamp-tcp-mss=no keepalive=10s,3 local-address=mikrotik_router_public_ipname=Tunnel1 remote-address=cisco_router_public_ip
/ip address
add address=192.168.13.2/30 interface=Tunnel1 network=192.168.13.0
/ip firewall filter
add action=accept chain=input comment="" connection-state=invalid,established,related,new,untracked protocol=gre src-address=cisco_router_public_ip
add action=accept chain=output connection-state=invalid,established,related,new,untracked dst-address=cisco_router_public_ip protocol=gre
I’m fairly sure that I don’t have to explicitly allow the output chain rule above, but I wanted to see the counters increment as a quick sanity check. On the Cisco router side, the tunnel is up/down. On the Mikrotik side, I see the tunnel interface running (which I guess is just the response to the keepalives). In another thread, someone pointed to a change in v6.45.1 (onntrack - fixed GRE protocol packet connection-state matching (CVE-2014-8160)).
I’m guessing that I’m missing something in the firewall rules, but hoping to get some guidance.
That someone may have been me. There is an issue that incoming GRE packets match on connection-state=invalid even if GRE packets in the opposite direction are being sent (so seen by chain output) and hence get dropped if the factory default firewall rule set is used, which contains a chain=input connection-state=invalid action=drop rule. I usually add protocol=!gre to that rule’s list of match conditions, but that’s not enough and you have to disable the GRE tunnel at both ends for more than 10 minutes after adding that rule; the tunnel then comes up once you re-enable it after that time. It seems to me that only some CPU architectures are affected by this.
Mikrotik support suggests that enabling pptp under /ip firewall service-port resolves this issue too.
It is not clear to me whether you’ve only posted the firewall filter rules you deem relevant or whether there is really nothing else in your firewall (which would mean that your router can be easily attacked from the internet). Better post a complete /ip firewall filter export and /interface list export (or even better, the export of the complete configuration, following the hint in my automatic signature here below).
However, if the Mikrotik end says the tunnel is up, it means that you’re not affected by the connection tracking issue, otherwise the keepalive responses coming from the Cisco end would not reach the GRE stack and the interface would be deemed down.
The fact that it goes up and down at Cisco is surprising, but I don’t have much experience with Cisco. In any case, due to the way how GRE keepalive is designed, the keepalive response is not generated by the recipient of the keepalive request - it is decapsulated from the request, into which the sender has pre-generated it. This means that from the point of view of the firewall, the keepalive response packet is forwarded from the GRE tunnel interface to the WAN interface. So again depending on your complete firewall rules, it may be dropped; but in my understanding, if that was the case, the interface should be constantly seen as down at the Cisco end, not keep flashing between down and up.
You can open a CLI window, make it as wide as your screen allows, and run /tool sniffer quick ip-protocol=gre ip-address=cisco_router_public_ip in it. This should show you whether the keepalive response packets emerge from the GRE interface with source address mikrotik_router_public_ip and leave through the WAN one.
The cisco line/protocol status of up/down doesn’t mean that it’s flapping. It just means that the line (L3 reachability in this case) status is up, but the protocol status (gre) is down. Apologies for that not being clear.
I added protocol=!gre to the connection-state=invalid action=drop rule, and enabled pptp. You’re correct that I didn’t post the full /ip firewall config, so there are additional rules. I shut the tunnel interface on each end, and I’ll wait a bit before enabling them and trying again. If the tunnel still doesn’t come up, I’ll post the additional configs.
The tunnel is still up/down on the Cisco end, so there’s no routing issue, just the GRE not coming up. I went ahead and upgraded to the latest stable release. Thanks again for taking the time to look at this and offer your assistance. Here is the sanitized config.
# mar/10/2021 14:55:32 by RouterOS 6.48.1
#
# model = RB4011iGS+
/interface ethernet
set [ find default-name=ether2 ] disabled=yes
set [ find default-name=ether3 ] disabled=yes
set [ find default-name=ether4 ] disabled=yes
set [ find default-name=ether6 ] disabled=yes
set [ find default-name=ether7 ] disabled=yes
set [ find default-name=ether8 ] disabled=yes
set [ find default-name=ether9 ] disabled=yes
set [ find default-name=ether10 ] poe-out=off
set [ find default-name=sfp-sfpplus1 ] disabled=yes
/interface bridge
add admin-mac= auto-mac=no comment=defconf name=bridge
/interface gre
add allow-fast-path=no clamp-tcp-mss=no keepalive=10s,3 local-address=mikrotik_router_public_ip name=Tunnel1 remote-address=cisco_router_public_ip
/interface vlan
add interface=bridge name=vlan54 vlan-id=54
/interface ethernet switch port
set 0 default-vlan-id=0
set 1 default-vlan-id=0
set 2 default-vlan-id=0
set 3 default-vlan-id=0
set 4 default-vlan-id=0
set 5 default-vlan-id=0
set 6 default-vlan-id=0
set 7 default-vlan-id=0
set 8 default-vlan-id=0
set 9 default-vlan-id=0
set 10 default-vlan-id=0
set 11 default-vlan-id=0
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
add name=GRE
/ip dhcp-server
add disabled=no interface=bridge name=defconf
/user group
set full policy=local,telnet,ssh,ftp,reboot,read,write,policy,test,winbox,password,web,sniff,sensitive,api,romon,dude,tikapp
/interface bridge port
add bridge=bridge comment=defconf interface=ether2
add bridge=bridge comment=defconf interface=ether3
add bridge=bridge comment=defconf interface=ether4
add bridge=bridge comment=defconf interface=ether6
add bridge=bridge comment=defconf interface=ether7
add bridge=bridge comment=defconf interface=ether8
add bridge=bridge comment=defconf interface=ether9
add bridge=bridge comment=defconf interface=ether10
add bridge=bridge comment=defconf interface=sfp-sfpplus1
/ip neighbor discovery-settings
set discover-interface-list=all
/ip settings
set rp-filter=strict
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=ether5 list=WAN
add interface=Tunnel1 list=GRE
/ip address
add address=10.10.1.1/30 interface=ether10 network=10.10.1.0
add address=10.5.4.1/24 interface=vlan54 network=10.5.4.0
add address=192.168.13.2/30 interface=Tunnel1 network=192.168.13.0
/ip dhcp-client
add comment=defconf interface=ether1
add disabled=no interface=ether5
/ip firewall filter
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=drop chain=input comment="post v6.45.1 gre workaround" connection-state=invalid protocol=!gre
add action=accept chain=input comment="accept inbound GRE" protocol=gre src-address=cisco_router_public_ip
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
add action=drop chain=forward comment="defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN
/ip firewall service-port
set ftp disabled=yes
set tftp disabled=yes
set irc disabled=yes
set h323 disabled=yes
set sip disabled=yes
set udplite disabled=yes
set dccp disabled=yes
set sctp disabled=yes
/ip route
add distance=1 dst-address=10.0.0.0/8 gateway=10.10.1.2 pref-src=10.10.1.1
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www-ssl disabled=no
set api disabled=yes
set winbox disabled=yes
set api-ssl disabled=yes
/ip ssh
set allow-none-crypto=yes forwarding-enabled=remote
/system clock
set time-zone-name=America/Denver
/system identity
set name=RB4011iGS+
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN
/tool mac-server ping
set enabled=no
You’ll have to do the sniffing, because I cannot see anything in forward chain of your firewall filter that would prevent the GRE keepalive responses from being delivered.
I shut the tunnel interface on both sides, waited about 30 minutes, kicked off the sniffer, and then re-enabled the tunnel interfaces. This same sequence just repeats. Ether5 is my WAN interface.
INTERFACE TIME NUM DIR SRC-MAC DST-MAC VLAN SRC-ADDRESS DST-ADDRESS PROTOCOL SIZE CPU FP
ether5 11.85 1 -> 74:4D:28:XX:XX:XX 00:01:5C:XX:XX:XX mikrotik cisco ip:gre 62 0 no
ether5 14.574 2 <- 00:01:5C:XX:XX:XX 74:4D:28:XX:XX:XX cisco mikrotik ip:gre 62 3 no
Tunnel1 14.574 3 <- mikrotik cisco ip:gre 24 3 no
ether5 16.614 4 -> 74:4D:28:XX:XX:XX 00:01:5C:XX:XX:XX mikrotik cisco ip:gre 181 1 no
ether5 23.85 5 -> 74:4D:28:XX:XX:XX 00:01:5C:XX:XX:XX mikrotik cisco ip:gre 62 0 no
ether5 23.94 6 <- 00:01:5C:XX:XX:XX 74:4D:28:XX:XX:XX cisco mikrotik ip:gre 60 3 no
ether5 24.573 7 <- 00:01:5C:XX:XX:XX 74:4D:28:XX:XX:XX cisco mikrotik ip:gre 62 3 no
Tunnel1 24.573 8 <- mikrotik cisco ip:gre 24 3 no
What we are interested in are the two pairs of packets at 14.574 and at 24.573. The first packet is the keepalive request coming from the Cisco address, whose payload is the keepalive response, which is shown at the next line on Tunnel1. What is missing is that very same keepalive response at ether5.
So try to add, as the very first rule in chain forward of /ip firewall filter, the following one: chain=forward in-interface-list=GRE protocol=gre action=accept
If the issue is caused by handling of GRE in connection tracking, this should help.
What are your reasons to use /ip settings
set rp-filter=strict
?
This setting checks whether a packet comes in via an interface through which a response to it would be routed, and if it doesn’t, the router silently drops it before it even gets to the firewall. Since the route for the GRE keepalive responses goes via ether5 but they come in via Tunnel1, they get dropped this way.
I would agree that the culprit could be the RP filter settings.
I’ve set up GRE tunnels Cisco<–>MikroTik many times and it has always just worked without a lot of fuss. Something has to be weird about this configuration.
if this needs to be turned off in your setup in order to WORK, the setup is messed. RP filter are a kind of anti-spoofing prevention and the GRE KA messages should go via the tunnel itself!
Unfortunately, that's how GRE keepalives have been designed by Cisco. The aim is that the keepaliving would work for each peer even if the other peer doesn't support it. So the setup is not messed, but the implementation of the keepalives is incompatible with rp-filter. It's a tradeoff.
The importance of rp-filter varies depending on use case - for ISP routers interfacing the internet, non-symmetric routing is more a rule than an exception, so strict is usually too strict and even loose may be too strict; at home with a single WAN, strict can give you some protection against attacks from WAN that spoof your LAN addresses, but you can always use firewall rules to implement more targeted source checking.
