GRE traffic being dropped somewhere after ESP packet arrived

trunet · Thu Mar 17, 2016 12:36 pm

Hello,

I have a pfsense -> mikrotik using ipsec and gre.

I sanitized the public IPs.

I shoot a ping between the tunnel IPs. On pfsense side(172.16.0.1), I have:

(wan traffic)
10:23:32.367083 IP 1.1.1.1 > 2.2.2.2: ESP(spi=0xcdaad2ef,seq=0xd), length 100
10:23:32.367163 IP 2.2.2.2 > 1.1.1.1: ESP(spi=0x08732688,seq=0xb), length 116
10:23:33.372045 IP 1.1.1.1 > 2.2.2.2: ESP(spi=0xcdaad2ef,seq=0xe), length 100
10:23:33.372099 IP 2.2.2.2 > 1.1.1.1: ESP(spi=0x08732688,seq=0xc), length 116
10:23:34.380881 IP 1.1.1.1 > 2.2.2.2: ESP(spi=0xcdaad2ef,seq=0xf), length 100
10:23:34.380933 IP 2.2.2.2 > 1.1.1.1: ESP(spi=0x08732688,seq=0xd), length 116

(gre traffic)
10:20:37.955303 IP 172.16.0.11 > 172.16.0.1: ICMP echo request, id 5987, seq 0, length 36
10:20:37.955333 IP 172.16.0.1 > 172.16.0.11: ICMP echo reply, id 5987, seq 0, length 36
10:20:38.960379 IP 172.16.0.11 > 172.16.0.1: ICMP echo request, id 5987, seq 1, length 36
10:20:38.960391 IP 172.16.0.1 > 172.16.0.11: ICMP echo reply, id 5987, seq 1, length 36
10:20:39.968975 IP 172.16.0.11 > 172.16.0.1: ICMP echo request, id 5987, seq 2, length 36
10:20:39.968988 IP 172.16.0.1 > 172.16.0.11: ICMP echo reply, id 5987, seq 2, length 36
10:20:40.974048 IP 172.16.0.11 > 172.16.0.1: ICMP echo request, id 5987, seq 3, length 36
10:20:40.974063 IP 172.16.0.1 > 172.16.0.11: ICMP echo reply, id 5987, seq 3, length 36

On mikrotik side(172.16.0.11, and I took some non related traffic out):

60    3.09 casasv-caovh       172.16.0.11                      172.16.0.1                                        icmp           56   0 no 
61   3.091 ether1-gateway     1.1.1.1                                     2.2.2.2                                    ipsec-esp     120   0 no 
63   3.258 ether1-gateway     2.2.2.2                                    1.1.1.1                                     ipsec-esp     136   0 no 
67   4.095 casasv-caovh       172.16.0.11                     172.16.0.1                                        icmp           56   0 no 
68   4.095 ether1-gateway     1.1.1.1                                     2.2.2.2                                    ipsec-esp     120   0 no 
69   4.262 ether1-gateway     2.2.2.2                                    1.1.1.1                                     ipsec-esp     136   0 no

So, do you have any idea where is the GRE reply packet inside mikrotik as the ESP package arrived?

Best regards,
Wagner Sartori Junior

trunet · Thu Mar 17, 2016 12:45 pm

By the way, I'm on latest version, 6.34.3 on a 2011UiAS-2HnD.

(peer)
 1    address=2.2.2.2/32 local-address=1.1.1.1 passive=no port=500 auth-method=pre-shared-key 
      secret="*SANITIZED*" generate-policy=no policy-template-group=default exchange-mode=main 
      send-initial-contact=yes nat-traversal=yes proposal-check=obey hash-algorithm=sha1 enc-algorithm=aes-256 
      dh-group=modp1024 lifetime=8h lifebytes=0 dpd-interval=2m dpd-maximum-failures=5 

(policy)
 2     src-address=1.1.1.1/32 src-port=any dst-address=2.2.2.2/32 dst-port=any protocol=gre 
       action=encrypt level=require ipsec-protocols=esp tunnel=no sa-src-address=1.1.1.1 
       sa-dst-address=2.2.2.2 proposal=default priority=0 

(gre)
 0  R name="casasv-caovh" mtu=auto actual-mtu=1426 local-address=1.1.1.1 remote-address=2.2.2.2 
      dscp=inherit clamp-tcp-mss=yes dont-fragment=no allow-fast-path=yes 

(ip address)
 3   172.16.0.11/24     172.16.0.0      casasv-caovh

trunet · Thu Mar 17, 2016 4:35 pm

Attached the decrypted ESP returning packet showing the ping reply inside. The packet was captured on mikrotik using /tool sniffer/start/stop/save and exported to wireshark. On wireshark I added the encryption and authentication SA keys to decrypt.

returning_esp_packet_decrypted.png

trunet · Thu Mar 17, 2016 5:10 pm

I just figured out that GRE packet from pfsense, has 4 extra bytes and is type 0x883e(WCCP). Still not sure how it works, but is there any possibility of mikrotik drop the GRE packet because it doesn't understand?

trunet · Thu Mar 17, 2016 5:58 pm

I found the problem. pfsense has a bug. ifconfig gre0 with LINK2 flag, enable WCCP v2 that add extra 4 bytes to the GRE that mikrotik don't understand.

Even with "WCCP version" unset on pfsense webui, LINK2 stays enabled.

GRE traffic being dropped somewhere after ESP packet arrived

GRE traffic being dropped somewhere after ESP packet arrived

Re: GRE traffic being dropped somewhere after ESP packet arrived

Re: GRE traffic being dropped somewhere after ESP packet arrived

Re: GRE traffic being dropped somewhere after ESP packet arrived

Re: GRE traffic being dropped somewhere after ESP packet arrived

Who is online