Hello ,
I’ve posted a problem about the GRE tunnel between Mikrotik and Cisco router that has a problem which is Keep a live problem
I mean when I configure Keep alive on both sides the tunnel is going down and when I remove the keep alive the tunnel is up and running !!
I really need to know what are the reasons for this problem ?? I mean the possibilities Please I really need your help
Best Regards
I wonder why you open a new topic rather than continuing in the existing one. Your last post there was just “thank you”, so I’ve expected it started to work.
Without the keepalive, the system (doesn’t matter whether Cisco IOS or Mikrotik RouterOS) cannot know whether the transport packets of the tunnel can get through or not, so it always reports the tunnel interface as up even if no transport packets are coming. So when you say “tunnel is up and running”, does it mean that you’ve tested that you can actually use it or it’s just that the devices report the tunnel interfaces as being up? The direction of further investigation depends on this answer.
I’m so sorry Sindy
Actually I meant the tunnel is up and running means it’s ok and I can ping but once I configure the keep alive on both sides the tunnel is going down !!
I can’t figure where is the problem !! how can I know where to search I mean the firewall !! I don’t know
and sorry again for open a new post
Good, so the only issue is the handling of keepalive packets.
I’ll summarize again - the way how GRE keepalive works, it needs no special algorithm on the remote side. The sender of the keepalive packet prepares the response, which is a GRE packet the remote side would send if it was to encapsulate an empty payload, and sends that response as a payload of its own GRE packet. So the whole keepalive request packet looks as follows (simplified):
from:A.A.A.A to:B.B.B.B type:GRE payload:{from:B.B.B.B to:A.A.A.A type:GRE payload:{}}
The recipient handles such a packet exactly the same way it would handle any other GRE packet coming from the peer - it extracts the inner GRE packet and forwards it to its destination, which happens to be the sender of the keepalive request packet.
Now on Mikrotik, a different path through the firewall is used for packets generated by the router itself and for packets received from somewhere else, which the router is just forwarding. So if a payload packet coming in via some in-interface is routed via a GRE tunnel, it is is handled by firewall chain forward and its out-interface is the GRE interface; the GRE packet into which that payload one gets encapsulated is sent by the router itself (so it has no in-interface defined), and is handled by firewall chain output.
So whereas the keepalive request packet is handled by chain output, the keepalive response packet is handled by chain forward.
On Cisco, the philosophy of the firewall is different, and I don’t know it good enough to suggest anything.
But if you enable the keepalive only at Cisco side, you can make the command line window at Mikrotik side as wide as your screen allows and run /tool sniffer quick interface=the-gre-interface-name ip-protocol=gre in it, and it should show you packets with come in via the GRE interface and have a source address of the Mikrotik itself, and a destination address of the Cisco - these are the “prepaid” keepalive responses. If you add the cisco-facing interface name to the interface list in that command (so it will say interface=the-gre-interface-name,cisco-facing-interface-name), you should see the keepalive request to arrive through the cisco-facing one, the keepalive response coming in via the GRE interface, and the same response leaving through the cisco-facing one. If the second item on this list is missing, the decapsulation doesn’t work which is unlikely; if the third one is missing, the firewall at Mikrotik side blocks it, or you use some policy routing which causes the response to be misrouted.
Hey Sindy thank a lot for all of these information
The problem is I have the whole configuration from both sides Mikrotik and Cisco router but the problem there are a lot of configuration on Mikrotik router but I can see the whole configuration related to the gre tunnel between Cisco router and Mikrotik and I think the problem is in the firewall
I did like a simulation on my local machine for a virtual router for both mikrotik and cisco and the tunnel is work perfect but when I try to do it on a real setup using public IPS between Mikrotik and cisco the tunnel is down when I configure keep alive
Have you tried the sniffing as suggested above in the real life case? What did it show? Do both machines have public IP on themselves or is there some NAT between them?
They have public IPS , I didn’t try sniffing because I have to make an appointment with the customer because he is not allowing me to take the full control
I wished I could contact you directly , honestly tomorrow I’m going to copy the configuration on my local machine to see if the tunnel is going to be down it’s a big problem a actually if I have the router myself I could do it by myself or at least try to solve it
I’m not sure what the direct contact to me should change given that you don’t have unlimited access to the routers. What are your expectations?
Can you at least export the configuration to see what the firewall looks like?
Do I read you right that the tunnel runs between public IP addresses of the routers with no encryption?
Forward to what? I’d expect keep-alive response ending up in input chain
Yes just a simple GRE tunnel not IPSEC or anything else I will put the configuration of the mikrotik router hoping I can found a solution
At the device which has received the keepalive request, the “pre-cooked” keepalive response extracted from the request is forwarded from its in-interface, which is the GRE tunnel beeing keepalived itself, to some out-interface, which is the gateway interface facing towards the remote GRE peer.
Of course, at the device which has sent the keepalive request, the keepalive response is handled by input chain. But as the response is an ordinary GRE transport packet coming from the address to which the keepalive request has been previously sent (except that it has an empty payload), it is normally handled by the action=accept connection-state=established,related rule.
I figure the GRE keep-alive requests ends up in , in the rectangle on the right in:
https://wiki.mikrotik.com/images/thumb/6/67/PacketFlowDiagram_v6_a.svg/1275px-PacketFlowDiagram_v6_a.svg.png
Then the CPU creates the GRE keep-alive response , which loops once through all lower blocks to get encapsulated, and then is sent outbound. This without ever traversing iptables forward chain
That would be true if the header of the GRE keepalive request packet contained some distinctive bit that would make the protocol stack distinguish it from ordinary GRE transport packets and handle it in a dedicated branch of an algorithm, resulting in local generation of a response packet.
But the reality is that there is no difference between the GRE keepalive request packet and ordinary GRE transport packets except that the payload of the keepalive request is the keepalive response prepared by the keepalive request sender. The idea behind this is that responding to a keepalive request requires zero additional effort from the recipient of the request, it is enough that it handles it like any other received GRE packet.
I.e. the received GRE transport packet reaches the <decapsulate?> decision block in the top right corner of the diagram you refer to, goes north-west to the /decapsulate/, the decapsulated payload emerges from the (logical interface) to the left, and through several decision block it arrives to point (I), where the routing finds out that the destination address is none of the router’s own ones, so it sends the packet down the =forward> path to point (L).
Here is the Mikrotik configuration and I hope someone can help me with it
Mikrotik Router :
/interface gre add keepalive=5s,4 local-address=80.18.22.29 name=gre-Main-Tu
remote-address=71.22.55.20 add comment=“Main-Tu” disabled=yes keepalive=5s,4 local-address=
80.18.22.29 name=gre-Main-Tu remote-address=71.22.55.20
/ip address add address=10.130.3.2/30 interface=gre-Main-Tu network=10.130.3.0/30
/ip firewall filter add action=accept chain=input comment=Main-Tu src-address=71.22.55.20
add action=accept chain=output comment=Main-Tu dst-address=71.22.55.20
add action=accept chain=input comment=Main-Tu in-interface=gre-Main-Tu
add action=accept chain=input protocol=icmp
add action=drop chain=input dst-port=23 protocol=tcp
add action=drop chain=input dst-port=22 protocol=tcp
add action=drop chain=forward connection-limit=50,32 disabled=yes protocol=
udp
add action=drop chain=forward connection-limit=50,32 disabled=yes protocol=
tcp
add action=drop chain=forward dst-address=80.18.22.0/21
/ip firewall service-port
set ftp disabled=yes
set tftp disabled=yes
set irc disabled=yes
set h323 disabled=yes
set sip disabled=yes
set pptp disabled=yes
set udplite disabled=yes
set dccp disabled=yes
set sctp disabled=yes
/routing bgp aggregate add advertise-filter=connected-in disabled=yes include-igp=yes instance=
BGP-Velconet prefix=80.18.22.0/21
/routing bgp network add network=80.18.22.0/21 synchronize=no
On cisco router :
! interface GigabitEthernet1
ip address 71.22.55.20 255.255.255.240
negotiation auto !
int tu52
ip add 10.130.3.1 255.255.255.252
ip policy route-map MTU
ip mtu 1476
ip flow monitor
ip tcp adjust-mss 1436
keepalive 5 4
tunnel source 71.22.55.20
tunnel destination 80.18.22.29
Your configuration looks just fine from the point of view of GRE keepalives (and awful in terms of firewall rules, but that’s another can of worms, we’ll get to it later), so the only way to move forward is to sniff what is actually going on as I’ve suggested before.
When you disable keepalive at Mikrotik side (/interface gre unset gre-Main-Tu keepalive) and keep it allowed at Cisco side, running /tool sniffer quick ip-protocol=gre interface=gre-Main-Tu should show the keepalive responses extracted from the keepalive requests to emerge from the tunnel every 5 seconds. And if it does, plain /tool sniffer quick ip-protocol=gre should show you also the requests coming via some interface, and the responses being routed back to the address of the Cisco, most likely via the same interface.
So the whole picture should be something like
[me@MyTik] > tool sniffer quick ip-protocol=gre
INTERFACE TIME NUM DIR SRC-MAC DST-MAC VLAN SRC-ADDRESS DST-ADDRESS PROTOCOL SIZE CPU FP
ether1 1.102 1 <- 00:15:5D:FC:E9:01 00:15:5D:FC:E9:06 71.22.55.20 80.18.22.29 ip:gre 62 0 no
gre-Main-Tu 1.102 2 <- 80.18.22.29 71.22.55.20 ip:gre 24 0 no
ether1 1.103 3 -> 00:15:5D:FC:E9:06 00:15:5D:FC:E9:01 80.18.22.29 71.22.55.20 ip:gre 38 0 no
ether1 6.102 1 <- 00:15:5D:FC:E9:01 00:15:5D:FC:E9:06 71.22.55.20 80.18.22.29 ip:gre 62 0 no
gre-Main-Tu 6.102 2 <- 80.18.22.29 71.22.55.20 ip:gre 24 0 no
ether1 6.103 3 -> 00:15:5D:FC:E9:06 00:15:5D:FC:E9:01 80.18.22.29 71.22.55.20 ip:gre 38 0 no
If that is the case, and the Cisco says the tunnel is down, the issue is not at Mikrotik side. If only the first rows of each trinity above (the GRE keepalive request coming in) appears, the keepalive format sent by Cisco is different than expected, which would be a surprise as that encapsulated response is designed by Cisco.
Since the keepalive response is a GRE packet carrying an empty payload, and since the two GRE peers are at public addresses, I can imagine the reason to be some paranoid security device between the Cisco and the Mikrotik, filtering out “weird” GRE packets with no payload.
Regarding the firewall rules: whereas in the Cisco world, packets that match no row on an access-list are dropped, in the Mikrotik world, packets that match no rule in a firewall chain are accepted. Hence the firewall rules at Mikrotik side accept almost all traffic, and if these are really your complete firewall rules, some malware may well be squatting on that Mikrotik by now.
Thanks a lot Sindy as I told you before the problem that I don’t have a FULL control to the Mikrotik with my client so I will have a meeting with him next week to test what you told me and I will let you know I’m so grateful for your help
Best Regards
Hello Sindy
Tomorrow I will have a call with the client tomorrow with a brand new Mikrotik I just want to ask you how many firewalls rules do I have to implement on this Mikrotik to make the GRE Tunnel working perfect ?
I really need your help please
Best Regards
I’m afraid there is a problem with my English. I keep trying to help you and you keep asking for something else. So once more - before analysing what is actually going on, the way suggested above, there is no point in asking me further questions.
The problem may not be in the Mikrotik or the Cisco at all. Something is unusual in your setup as a whole, as the keepalives normally work and they don’t in this particular setup. Only seeing what actually happens can give a clue what is wrong and how to possibly fix it.
Ok I will try it today and I’m so so sorry for keep asking today I will try to do the sniff and give here the results thanks a lot and sorry again
As you’ve posted only the part of Mikrotik configuration you deemed relevant, is it possible that you have the same setting found to be a culprit here?