I have several Mikrotik routers with GRE running on top of IPSec. Both GRE ends are configured and are working properly.
I’d like to try OSPF and I wanted to get it running on my GRE links. I followed a number of instructions (including the official docs), but my OSPF only got as far as ‘ExStart’ state. I did a packet sniff on the GRE link and I can see ‘Hello’ packets from both ends, but I don’t see any DD packets at all...
Firewalls on both Mikrotiks do have ‘Allow’ rule for OSPF protocol on the GRE.
Most tutorials basically say that it should ‘just work’ with a pretty basic configuration… Perhaps, I am missing something obvious… I kindly ask for an advise here!
Here’s my config:
Router 1:
/routing ospf instance add disabled=no name=ospf-172.23.31.0 originate-default=always router-id=172.23.31.1 /routing ospf area add disabled=no instance=ospf-172.23.31.0 name=ospf-area-1 /routing ospf area range add area=ospf-area-1 disabled=no prefix=172.23.31.0/24 /routing ospf interface-template add area=ospf-area-1 disabled=no interfaces=gre-to-20 priority=0 type=ptp /routing/ospf/neighbor/print Flags: V - virtual; D - dynamic 0 D instance=ospf-172.23.31.0 area=ospf-area-1 address=172.23.231.82 router-id=172.23.20.1 state="ExStart" state-changes=2 timeout=37s /interface/gre/print Flags: X - disabled; R - running 0 R ;;; 20.0/24 name="gre-to-20.0" mtu=auto actual-mtu=1280 local-address=172.23.231.81 remote-address=172.23.231.82 keepalive=10s,10 dscp=inherit clamp-tcp-mss=yes dont-fragment=no allow-fast-path=no
Router 2:
/routing ospf instance add disabled=no name=ospf-172.23.20.0 originate-default=always router-id=172.23.20.1 /routing ospf area add disabled=no instance=ospf-172.23.20.0 name=ospf-area-1 /routing ospf area range add area=ospf-area-1 disabled=no prefix=172.23.20.0/24 /routing ospf interface-template add area=ospf-area-1 disabled=no interfaces=gre-to-231 priority=0 type=ptp /routing/ospf/neighbor/print Flags: V - virtual; D - dynamic 0 D instance=ospf-172.23.20.0 area=ospf-area-1 address=172.23.231.81 router-id=172.23.31.1 state="ExStart" state-changes=4 timeout=32s /interface/gre/print Flags: X - disabled; R - running 0 R name="gre-to-231" mtu=auto actual-mtu=1280 local-address=172.23.231.82 remote-address=172.23.231.81 keepalive=10s,10 dscp=inherit clamp-tcp-mss=yes dont-fragment=no allow-fast-path=no
The GRE tunnel uses 172.23.231.80/30 subnet that is being handled by the IPSec. I have manually-added routes and traffic flowing through the GRE tunnel.
I still don't get it. Do you mean my IPSec & GRE config is conceptually broken? It could be, although it worked fine so far...
My Router 1 is configured as an IPSec server with a WAN IP of 1.1.1.1 (address changed for simplicity). It also has a policy template set, so when Router 2 connects - it creates the relevant policy.
This is the Router 1 config:
/ip ipsec peer
add name=Server passive=yes profile=server-profile
/ip/ipsec/policy/print
Flags: T - TEMPLATE; D - DYNAMIC, A - ACTIVE; \* - DEFAULT
Columns: PEER, TUNNEL, SRC-ADDRESS, DST-ADDRESS, PROTOCOL, ACTION, LEVEL, PH2-COUNT
PEER TUNNEL SRC-ADDRESS DST-ADDRESS PROTOCOL ACTION LEVEL PH2-COUNT
5 D Server-231-crt3 yes 172.23.231.81/32 172.23.231.82/32 all encrypt unique 1
Router 2's IPSec config is respectively as follows:
So this gives me 172.23.231.81/30 IP at Router 1 and 172.23.231.82/30 at Router 2.
I then have these assigned to the GRE tunnel interfaces and also have them configured as tunnel endpoints.
For the tunnel config, "just GRE" uses an interface's IP address, though not the tunnel's, and it has to be routable between the 2 routers. For example if you have router 1 as 1.1.1.1 and router 2 as 2.2.2.2, a config for router 1 would be local-address=1.1.1.1 remote-address=2.2.2.2 and assigning /ip address address=172.16.255.1/30 interface=gre1.
With what you have shown so far, it might be that the tunnel is trying to go inside itself at some point.
With the ipsec protection, i need to check. A typical setup is to assign a loopback address on each router, ipsec tunnel these and only these, and establish the gre tunnel between the loopback addresses. A better option us to use vti interfaces but that's not available on Mikrotik.
I can't use "just GRE" because in my real setup there is one Mikrotik "Server" with a static IP address and a number of Mikrotik "Clients" with dynamic IPs (or behind NAT, etc.). So I can't have "just GRE" between each Client and the Server.
With IPSec, a Server accepts all the connections - regardless of the Clients' real addresses - and lets them use the self-defined addresses in a private range (172.23.2xx.xxx in my case). And then I'm setting up a GRE on top of those self-defined addresses.
The reason for me to use GRE and not just IPSec Policies is that I want to be able to set up a bit more complex routing (i.e. restrict some subnets from accessing others). I can probably do all that with just policies, but I feel much more comfortable with the 'classical routes' approach...
Now, I think I tried this before and the GRE tunnel never got to the 'Running' state if the GRE interface address was different from its local-address configured in the tunnel itself. But that was a long time ago, so I will re-check it again... If will be funny if GRE IP is the root cause of my issue...
EDIT: Confirmed, as soon as I make the IP address of the GRE interface different from the 'local-address=...' configured for that interface, the GRE link goes down.
... and the reason I want to have OSPF on my network is because I'm currently busy setting up additional 'Servers'. So when one Server becomes unreachable, I want the traffic to be automatically re-routed via other Servers.
After some more fiddling I discovered that if I add a second IP address to the GRE interface, the OSPF will happily pick them up and will go to 'Full' state. So it looks like some of the OSPF traffic got 'lost' in my original setup.
Now, this setup gets a bit too complex. I have several 'user subnets' (where the actual user data is), then I have 'internal' /30 subnets between each routers pair - handled by IPSec in order to establish GRE links, and now I'm adding 'another internal' /30 subnets to each GRE link that will finally allow OSPF. This doesn't feel right. I wonder how a "proper" network should be designed in cases like mine - a few 'backbone' routers with fixed IPs and a number of client routers connecting using dynamic / NATed addresses...
the public IP, the hubs must have static IP addresses, the spokes can have dynamic
a loopback IP, which will be used to "anchor" the GRE tunnel
a /30 for each tunnel [*]
[*] I tried to use GRE as unnumbered but OSPF doesn't come up. Assigning an IP address works.
Here is the configuration for the hub:
public IP: 10.0.0.01
Loopback: 172.31.254.1/32
GRE to spoke01: 172.30.255.1/30
Given that this is a lab environment, I added a route to 172.31.255.1/32 via ether1 - otherwise the IPSEC policy is never matched. In your environment, you may need it or it may be addressed by the default route.
This is an older routeros that doesn't have a proper lo interface, so I added the loopback as a secondary address on the bridge interface. I added a routing filter rule to redistribute only the routes in 192.168.0.0/16, but not the loopback/tunnel IP.
[admin@hub01] > /routing/route/print
Flags: A - ACTIVE; c - CONNECT, s - STATIC, o - OSPF
Columns: DST-ADDRESS, GATEWAY, AFI, ROUTING-TABLE, DISTANCE, SCOPE, TARGET-SCOPE, IMMEDIATE-GW
DST-ADDRESS GATEWAY AFI ROUTING-TABLE DISTANCE SCOPE TARGET-SCOPE IMMEDIATE-GW
Ac 10.0.0.0/24 ether1 ip main 0 10 5 ether1
Ac 172.30.255.0/30 gre_spoke01 ip main 0 10 5 gre_spoke01
Ac 172.31.254.1/32 lo ip main 0 10 5 lo
As 172.31.255.0/24 ether1 ip main 1 30 10 ether1
Ao 172.31.255.1/32 172.30.255.2%gre_spoke01 ip main 110 20 10 172.30.255.2%gre_spoke01
Ac 192.168.90.0/24 bridge ip main 0 10 5 bridge
Ao 192.168.100.0/24 172.30.255.2%gre_spoke01 ip main 110 20 10 172.30.255.2%gre_spoke01
Ac ::1/128 lo ipv6 main 0 10 5 lo
Ac fe80::/64 ether1 ipv6 main 0 10 5 ether1
Ac fe80::/64 bridge ipv6 main 0 10 5 bridge
Ac fe80::/64 gre_spoke01 ipv6 main 0 10 5 gre_spoke01
Ac fe80::/64 gre_spoke01 ipv6 main 0 10 5 gre_spoke01
A lo link main 0
A ether1 link main 0
A ether4 link main 0
A bridge link main 0
A gre_spoke01 link main 0
Neighbors
[admin@hub01] > /routing/ospf/neigh/print
Flags: V - virtual; D - dynamic
0 D instance=main-ospf area=backbone address=172.30.255.2 router-id=192.168.100.1 state="Full" state-changes=6 adjacency=19m14s timeout=37s
Lastly, the situation you have is the exact situation where vti would be perfect, but unfortunately not available in Mikrotik.
If you have a full-Mikrotik ecosystem, you can consider to use wireguard instead of ipsec/gre. There you have 2 choices: one master interface on the hubs and each spoke is configured with its local subnets, or you have one interface per spoke on each hub. Each have pros and cons.
Wow, this is amazing! Thank you so much!
This seem to work exactly as intended!!!
One question: in your example (and in many other tutorials) there is a loopback interface (basically a bridge) with a dedicated private IP address. If I use the address of a router's main bridge interface instead - would that cause me any troubles? For example, if a remote LAN is 192.168.100.0/24 and the router's bridge address is 192.168.100.1/24 - can I use this address as the GRE endpoint and then be able to access other hosts on the 100.0/24 network in a similar way? I think it should work just fine, but maybe I miss something...
P.S. Wireguard wasn't the best option in my case as we have to use some providers that seem to block it (and IPSec works fine so far).