ip-sec between MikroTik and Cisco ASA not passing traffic

RouterOS General
1

Having an issue with a VPN tunnel between a MikroTik router (1100AHx2) and a Cisco ASA. I have the tunnel established, so I am fairly confident I have that set up but am having issues getting traffic to route across the tunnel at either end.

The IPsec policy has the local and remote networks defined, the proposal is set such that it’s a match to the remote end, the peer has the remote IP as well as the local IP set to what they should be, and the profile used by the the peer looks OK too.

On the MikroTik I have a NAT rule to allow traffic from the local network destined to the remote network set to “accept” and placed at the tope. I have a few filter rules to allow the traffic there as well. I do have a route defined, which looks like:

10.0.0.0/24 gateway: WANBridge - pref. source: WAN IP (of remote end)

This was what was put in place by the guy who previously set it up. It looks wrong to me, because with my own MikroTik device at home I have the remote subnet and then the gateway defined as the LAN interface with the internal IP address of the remote router (the ASA in this case).

On the MikroTik, a traceroute from the LAN interface just times out for every hop. From the WAN interface, it hits the external IP (gateway), then a 10.255.255.101 address (so something upstream of the firewall, probably the ISP’s router which sits in our rack).

My config is:

/ip ipsec> export

mar/02/2020 10:05:35 by RouterOS 6.45.6

software id = U8MT-VIIM

model = 1100AHx2

serial number = XXXXXXX

/ip ipsec profile
add dh-group=modp1024 enc-algorithm=aes-256,aes-192,aes-128,3des name=profile_8

/ip ipsec peer
add address=IP_ADDR/32 exchange-mode=ike2 local-address=IP_ADDR name=peer40 profile=profile_8

/ip ipsec proposal
add enc-algorithms=aes-256-cbc,aes-192-cbc,3des lifetime=8h name=NAME

/ip ipsec identity
add comment=“Site PSK” peer=peer40 secret=SECRET

/ip ipsec policy
add comment=“Site tunnel” dst-address=10.0.0.0/24 peer=peer40 proposal=Site-VPN sa-dst-address=IP_ADDR sa-src-address=IP_ADDR src-address=192.168.16.0/24 tunnel=yes

full disclosure - there are other ip-sec tunnels from this MikroTik, 3 in total including mine. I can see of those 3, that 3 are connected, but only 1 is actively sending/receiving. I’m not sure if that’s relevant/important. Exchange mode for other tunnels is set to ‘main’.

I feel that I’m missing something fundamentally important here, and I’m hoping someone can point me in the right direction.

2

About routes, pure policy-based IPSec doesn’t usually need any specific one for remote network.

It’s a little confusing, but all it needs is any valid route to destination (including default route) and it doesn’t matter where it points to, because it’s not really used anyway. What happens is that before packet from local network to remote network is sent anywhere, it’s snatched by IPSec and encrypted. And this encrypted packet has remote peer as destination and local WAN address (or whatever is used for outgoing connections) as source, so it’s routed like any other regular router’s output.

With most basic config where the only existing route (except connected ones) is default, i.e.:

/ip route
add dst-address=0.0.0.0/0 gateway=<ISP's router>

you don’t need any other route to remote 10.0.0.0/24. You’d need one if you either didn’t have default route or if you’d block route to this subnet, e.g. using (such route can be useful for blocking traffic to unused private subnets):

/ip route
add dst-address=10.0.0.0/8 type=unreachable

With this, router would see 10.0.0.0/24 as unreachable and access to there would not work. You’d then need to add another route like this:

/ip route
add dst-address=10.0.0.0/24 gateway=<gateway>

But the only important fact is that it exists and overrides unreachable 10.0.0.0/8, gateway could be anything, ISP’s router, WAN interface, LAN interface, everything would work. As written above, it’s not really used. And again, if you do have only default route, you don’t need any other. Although it can be useful to add it anyway with pref-src set to router’s local address (which belongs to local subnet defined in IPSec policy):

/ip route
add dst-address=10.0.0.0/24 gateway=<gateway> pref-src=192.168.16.1

It will help when you try to access remote subnet from router itself, because it will select correct source address that will be accepted by tunnel. Also when doing traceroute from remote subnet to local one, router will answer from this address and you won’t have dead hop.

So that’s some theory.

If by “external IP (gateway)” you mean ISP’s router, then there’s some problem with packets’ source address which does not match policy and IPSec ignores those packets. That’s what the accept NAT rule should fix, and according to your description it sounds correct.

3

From the WAN interface, it hits the external IP (gateway), then a 10.255.255.101 address (so something upstream of the firewall, probably the ISP's router which sits in our rack).

If by "external IP (gateway)" you mean ISP's router, then there's some problem with packets' source address which does not match policy and IPSec ignores those packets. That's what the accept NAT rule should fix, and according to your description it sounds correct.

I should clarify that. A traceroute would look like this, from the CLI on the firewall:

/tool traceroute 8.8.8.8

ADDRESS LOSS SENT LAST AVG BEST WORST STD-DEV STATUS

1 0% 3 0.3ms 0.5 0.3 0.8 0.2
2 10.255.255.101 0% 3 8.8ms 8 7.4 8.8 0.6
3 91.151.0.147 0% 3 7.9ms 8 7.9 8.1 0.1
4 195.66.224.125 0% 3 7.5ms 7.5 7.5 7.5 0
5 108.170.246.129 0% 3 7.5ms 7.8 7.5 8.1 0.2
6 108.170.238.117 0% 3 8ms 7.6 7.4 8 0.3
7 8.8.8.8 0% 3 7.4ms 7.5 7.4 7.7 0.1

/tool traceroute 10.0.0.1

ADDRESS LOSS SENT LAST AVG BEST WORST STD-DEV STATUS

1 0% 2 0.3ms 0.5 0.3 0.7 0.2
2 10.255.255.101 0% 2 7.4ms 8.2 7.4 8.9 0.8
3 100% 2 timeout
4 100% 2 timeout
5 100% 2 timeout
6 100% 2 timeout
7 100% 2 timeout

The 10.255.255.101 is the ISP router that sits between our MikroTik and the Virgin Media NTE

4

That’s clearly not going to tunnel. But that’s expected, because router will by default choose its WAN address as source. Try it with (I assume 192.168.16.1 is what router has):

/tool traceroute src-address=192.168.16.1 8.8.8.8
5

Not expecting the trace route to tunnel, it’s more to illustrate the first few hops and what it goes through. specifying the IP at least for the trace to google just looks like it did before. Doing the same for the internal resource on the remote end looks like:

/tool traceroute src-address=192.168.16.1 10.0.0.1
 # ADDRESS                          LOSS SENT    LAST     AVG    BEST   WORST STD-DEV STATUS                                                                                                                                                             
 1                                  100%    3 timeout                                                                                                                                                                                                    
 2                                  100%    3 timeout                                                                                                                                                                                                    
 3                                  100%    2 timeout                                                                                                                                                                                                    
 4                                  100%    2 timeout                                                                                                                                                                                                    
 5                                  100%    2 timeout

However, the tunnel still does not pass any traffic. Could the fact that there’s another ip-sec tunnel be causing any issues?

6

Specifying src-address when tracing to Google won’t of course do anything useful, when you have tunnel between local 192.168.16.0/24 and remote 10.0.0.0/24. When tracing to 10.0.0.1, you can see that something changed with included src-address.

Next step, what exactly means that tunnel does not pass any traffic? There are no responses from remote subnet, but what about IP->IPSec->Installed SAs? If tunnel is established, you will have some etries there. Do their counters in Current Bytes really stay at zero?

7

They do - the one of interest highlighted:

8

If you add this on top as first rule:

/ip firewall mangle
add chain=postrouting src-address=192.168.16.1 dst-address=10.0.0.1 action=log log-prefix=xxx

And then you try ping from router’s address to remote network:

/ping src-address=192.168.16.1 address=10.0.0.1

Do you see every single packet logged as:

firewall,info xxx postrouting: in:(unknown 0) out:<interface>, proto ICMP (type 8, code 0), 192.168.16.1->10.0.0.1, len 56

Or do they become like this after first one:

firewall,info xxx postrouting: in:(unknown 0) out:<interface>, proto ICMP (type 8, code 0), 192.168.16.1->10.0.0.1, NAT (192.168.16.1-><probably WAN address>)->10.0.0.1, len 56

Or you don’t see them logged at all?

9

So this is how they appear, which admittedly doesn’t look correct:

10

Do you have any fasttrack rules or other routing tables than main?

11

You’ll have to forgive my ignorance - where should I check for those 2 things? This is an inherited system that’s quite a mess.

12

if you mean filter rules with an action of ‘fasttrack’, then no - there are no filter rules under ip > firewall that have that as an action

13

Fasttrack would be rule in “/ip firewall filter” with action=fasttrack-connection. And other routing tables in “/ip route”, routes with configured routing-mark option.

And the log does look correct, I wanted to make sure that it’s really excluded from NAT and according to this it is.

14

Just checked, definitely no rules with fasttrack-connection. just a mixure of the typical accept, drop, and a few entries for ‘add src to address list’. As for routing-mark options, none are configured with routing-mark.

15

By looking at installed SA counters my guess is that RouterOS matches packets against policy properly, encapsulates and sends them to remote peer.
Either remote peer is dropping incoming packets or does not send a reply.

16

So they now look like they are getting to the remote end. I’m seeing this in the logs on the Cisco ASA which is where the tunnel goes to:

3	Mar 05 2020	12:43:00		192.168.16.1				Denied ICMP type=8, code=0 from 192.168.16.1 on interface outside

So the packets are arriving on the WAN interface and it’s dropping them. Some progress!

17

Thing is i still don’t see ‘current bytes’ increasing for the particular SA for this tunnel. Not sure if I should at this point, unless it only increases on received bytes

18

Make sure you’re looking at the right line. There can be more than one for same policy.

19

In the past we had IPSec tunnel between MT and ASA, no problems at all.
Not sure is it relevant for you, but I will share my case:
few days ago had issue with IPSec when client connects from NAT’ed ISP network (4G).
Tunnel initiated, but no traffic could pass the tunnel.
The solution was - turn on NAT-T on client side. Have no idea how to have NAT-T if client is Mikrotik.
I have created post, but later solved that case by my self :sunglasses:
http://forum.mikrotik.com/t/ipsec-problems-over-4g/137279/1

20

On the ASA, try a “show crypto ipsec sa peer ”. Do you have SAs established (it looks like the Mikrotik thinks SAs are established from your screenshot)? If so, you should see #pkts encaps and #pkts decaps values > 0. If decaps = 0, then encrypted packets are not being received by the ASA. If decaps > 0 but encaps = 0, then packets from the ASA side toward the Mikrotik are not being encapsulated by the ASA.

adamantium# sh crypto ips sa peer <remote peer IP>
peer address: <remote peer IP>
    Crypto map tag: Public_map, seq num: 10, local addr: <local peer IP>

      access-list vpn.cosentry permit ip 172.16.5.0 255.255.255.0 192.168.1.0 255.255.255.0
      local ident (addr/mask/prot/port): (172.16.5.0/255.255.255.0/0/0)
      remote ident (addr/mask/prot/port): (192.168.1.0/255.255.255.0/0/0)
      current_peer: <remote peer IP>

      #pkts encaps: 967441550, #pkts encrypt: 967441549, #pkts digest: 967441549
      #pkts decaps: 946612210, #pkts decrypt: 946612210, #pkts verify: 946612210
      #pkts compressed: 0, #pkts decompressed: 0
      #pkts not compressed: 967441550, #pkts comp failed: 0, #pkts decomp failed: 0
      #pre-frag successes: 0, #pre-frag failures: 1, #fragments created: 0
      #PMTUs sent: 1, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0
      #send errors: 0, #recv errors: 0

      local crypto endpt.: <local peer IP>, remote crypto endpt.: <remote peer IP>

      path mtu 1500, ipsec overhead 74, media mtu 1500
      current outbound spi: 8F4D2E40

    inbound esp sas:
      spi: 0x00A596D5 (10852053)
         transform: esp-aes-256 esp-sha-hmac none
         in use settings ={L2L, Tunnel, PFS Group 5, }
         slot: 0, conn_id: 2531, crypto-map: Public_map
         sa timing: remaining key lifetime (kB/sec): (2795269/20033)
         IV size: 16 bytes
         replay detection support: Y
    outbound esp sas:
      spi: 0x8F4D2E40 (2404200000)
         transform: esp-aes-256 esp-sha-hmac none
         in use settings ={L2L, Tunnel, PFS Group 5, }
         slot: 0, conn_id: 2531, crypto-map: Public_map
         sa timing: remaining key lifetime (kB/sec): (3330799/20033)
         IV size: 16 bytes
         replay detection support: Y