Community discussions

MUM Europe 2020
 
Zoolander06
newbie
Topic Author
Posts: 47
Joined: Thu Jan 03, 2019 5:26 pm

GRE tunnel established, ping ok, but no traffic

Thu Jan 09, 2020 7:02 pm

Hello,

I still have a lot of issues with IPSec, getting my GRE over IPSec tunnels down without any reason, with a log message about a phase 1 timeout.
So I tried to just disable IPSec encryption on one tunnel, and it instantaneously get up.
But, there is a big big "but" !

I can ping any device on the other side of the tunnel, but I cannot establish any TCP connection (SIP phones doesn't register, I can't access the web admin of a device).
This is weird, and I don't understand what happens.

Does anyone had this issue in the past and knows how to fix it ???

Thanks :)

Joris
 
Zacharias
Forum Guru
Forum Guru
Posts: 1386
Joined: Tue Dec 12, 2017 12:58 am
Location: Greece

Re: GRE tunnel established, ping ok, but no traffic

Thu Jan 09, 2020 10:35 pm

Did you setup the GRE/IPsec tunnel using the wiki example ?
It can be anything wrong... Since you have problems with IPsec as well...
Could you export with hide sensitive your IPsec config with policies, proposals, peers etc everything, your NAT rules, your firewall and your routes ?
 
Zoolander06
newbie
Topic Author
Posts: 47
Joined: Thu Jan 03, 2019 5:26 pm

Re: GRE tunnel established, ping ok, but no traffic

Fri Jan 10, 2020 11:57 am

Hello,

The problems I experience with IPSec are not the same.
Actually, I have problems with IPSec, with or without GRE, those are instability problems (tunnel stopping to work without any apparent reason).
The fact is that the same GRE tunnel work great with IPSec (except the instability), and doesn't work at all without IPSec.
I indeed follow the wiki example when I set up my first GRE tunnels, but I always used IPSec encryption, so I never noticed the issues without it.
It's difficult to make anything wrong following the wiki example, since it's basically 3 lines on each side, but I could have done a mistake with everything not in the example, I admit it.

Here is the export of one side :
export hide-sensitive 

# jan/10/2020 10:35:09 by RouterOS 6.46.1

# software id = LH3E-PIED

#

# model = RB2011UiAS-2HnD

# serial number = B9070A875DD7


/interface gre

add allow-fast-path=no dont-fragment=inherit local-address=aaa.bbb.ccc.ddd name=gre-vence-1 remote-address=eee.fff.ggg.hhh

/interface list

add comment=defconf name=WAN

add name=GRE

add comment=defconf include=GRE name=LAN

/interface list member

add comment=defconf interface=bridge list=LAN

add interface=vdsl-orange-ether1 list=WAN

add interface=adsl-sfr-ether2 list=WAN

add interface=gre-vence-1 list=GRE

/ip address

add address=192.168.3.1/24 comment=defconf interface=bridge network=192.168.3.0

add address=172.16.0.17/30 interface=gre-vence-1 network=172.16.0.16

/ip firewall filter

add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked

add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid

add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp

# For testing purpose

add action=accept chain=input dst-port=500,4500 in-interface-list=WAN protocol=udp

#For testing purpose

add action=accept chain=input in-interface-list=WAN protocol=gre

add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN

add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec

add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec

add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked

add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid

add action=drop chain=forward comment="defconf:  drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN

/ip firewall mangle

add action=mark-connection chain=prerouting connection-mark=no-mark in-interface=vdsl-orange-ether1 new-connection-mark=From-WAN1 passthrough=yes

# adsl-sfr-ether2 not ready

add action=mark-connection chain=prerouting connection-mark=no-mark in-interface=adsl-sfr-ether2 new-connection-mark=From-WAN2 passthrough=yes

add action=mark-routing chain=prerouting connection-mark=From-WAN1 dst-address-type=!local new-routing-mark=WAN1 passthrough=yes

add action=mark-routing chain=prerouting connection-mark=From-WAN2 dst-address-type=!local new-routing-mark=WAN2 passthrough=yes

add action=mark-connection chain=prerouting in-interface=bridge new-connection-mark=VoIP-cnx passthrough=yes src-address=192.168.3.250

add action=mark-connection chain=prerouting dst-address=192.168.3.250 in-interface-list=GRE new-connection-mark=VoIP-cnx passthrough=yes

add action=mark-packet chain=prerouting connection-mark=VoIP-cnx new-packet-mark=VoIP-Pkt passthrough=yes

add action=mark-connection chain=input connection-mark=no-mark new-connection-mark=GRE-cnx passthrough=yes src-address-list=gre

add action=mark-connection chain=output connection-mark=no-mark dst-address-list=gre new-connection-mark=GRE-cnx passthrough=yes

add action=mark-packet chain=output connection-mark=GRE-cnx new-packet-mark=GRE-Pkt passthrough=yes

add action=mark-connection chain=prerouting connection-mark=no-mark in-interface-list=LAN new-connection-mark=Data-cnx passthrough=yes

add action=mark-packet chain=prerouting connection-mark=Data-cnx new-packet-mark=Data-Pkt passthrough=yes

/ip firewall nat

#For testing purpose

add action=accept chain=srcnat dst-address=192.168.1.0/24 src-address=192.168.3.0/24

#For testing purpose

add action=accept chain=srcnat dst-address=192.168.0.0/24 src-address=192.168.3.0/24

add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface=vdsl-orange-ether1

# adsl-sfr-ether2 not ready

add action=masquerade chain=srcnat ipsec-policy=out,none out-interface=adsl-sfr-ether2

/ip firewall service-port

set sip disabled=yes

/ip route

add check-gateway=ping distance=1 gateway=vdsl-orange-ether1 routing-mark=WAN1

add check-gateway=ping distance=1 gateway=adsl-sfr-ether2 routing-mark=WAN2

add check-gateway=ping distance=1 gateway=vdsl-orange-ether1

add check-gateway=ping distance=2 gateway=adsl-sfr-ether2

add check-gateway=ping distance=1 dst-address=eee.fff.ggg.hhh/32 gateway=vdsl-orange-ether1

add distance=10 dst-address=eee.fff.ggg.hhh/32 type=blackhole

add check-gateway=ping distance=1 dst-address=192.168.0.0/24 gateway=172.16.0.18

/system clock

set time-zone-name=Europe/Paris

/system ntp client

set enabled=yes server-dns-names=fr.pool.ntp.org
Is there something wrong ?

Joris
 
User avatar
Anumrak
Forum Guru
Forum Guru
Posts: 1176
Joined: Fri Jul 28, 2017 2:53 pm

Re: GRE tunnel established, ping ok, but no traffic

Fri Jan 10, 2020 3:34 pm

Hey. Does your destination address is behind interface through which source NAT rule apply?
 
Zoolander06
newbie
Topic Author
Posts: 47
Joined: Thu Jan 03, 2019 5:26 pm

Re: GRE tunnel established, ping ok, but no traffic

Fri Jan 10, 2020 4:04 pm

Yes it is.

There is a route for my destination address using pppoe interface "vdsl-orange-ether1"
/ip route
add check-gateway=ping distance=1 dst-address=eee.fff.ggg.hhh/32 gateway=vdsl-orange-ether1
There is a src-nat rule for this interface :
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface=vdsl-orange-ether1
Is that not ok ?
 
Zoolander06
newbie
Topic Author
Posts: 47
Joined: Thu Jan 03, 2019 5:26 pm

Re: GRE tunnel established, ping ok, but no traffic

Wed Jan 15, 2020 3:36 pm

Hello,

Fun fact : yesterday, I was able to establish a fully fonctionnal tunnel between a Mikrotik 4011 and a Zyxel USG20.
Mikrotik was connected via LTE router (NAT on the router + NAT on the provider side), and the Zyxel was connected through the provider's router (with NAT).

So I don't understand why I can't do the same with my two RB2011 connected through PPPoE interfaces...
 
himvas
just joined
Posts: 24
Joined: Fri Apr 15, 2016 9:26 am

Re: GRE tunnel established, ping ok, but no traffic

Wed Jan 15, 2020 11:39 pm

Your firewall stops traffic comming from GRE (GRE in WAN list and not NATed).
 
Zoolander06
newbie
Topic Author
Posts: 47
Joined: Thu Jan 03, 2019 5:26 pm

Re: GRE tunnel established, ping ok, but no traffic

Thu Jan 16, 2020 11:00 am

Hi himvas, thanks for answering :)
I don't think the issue is in firewall, my gre interface is in a list named GRE, which is included in the LAN interface list.
Plus, I tried to put my gre interface directly in the LAN list, and also to add some filter rules to accept anything coming and going through this interface...

Joris
 
pe1chl
Forum Guru
Forum Guru
Posts: 6175
Joined: Mon Jun 08, 2015 12:09 pm

Re: GRE tunnel established, ping ok, but no traffic

Thu Jan 16, 2020 12:15 pm

When you have issues like "one type of traffic works and another type doesn't" you need to debug your firewall.
You write:
I don't think the issue is in firewall, my gre interface is in a list named GRE, which is included in the LAN interface list.
but that isn't even possible in RouterOS!
You may have lists with the same name as interfaces but that does not make them the same thing. Lists can only have interfaces as members, not other lists.
So first get that straight.

When it still does not work for TCP connections, start debugging MTU issues. You mention you use PPPoE so check the MTU of the PPPoE interface (will usually be 1480 or 1492) and see how much less than 1500 this is (20 or 8 in these cases) and subtract that number from the MTU that has been automatically set on the GRE interface (by default that will be 1476 so change it to 1456 or 1468).
When that still does not fix it, apply this rule to the mangle list:
/ip firewall mangle
add action=change-mss chain=forward new-mss=clamp-to-pmtu passthrough=yes \
    protocol=tcp tcp-flags=syn
 
User avatar
Anumrak
Forum Guru
Forum Guru
Posts: 1176
Joined: Fri Jul 28, 2017 2:53 pm

Re: GRE tunnel established, ping ok, but no traffic

Thu Jan 16, 2020 12:41 pm

Yes it is.

There is a route for my destination address using pppoe interface "vdsl-orange-ether1"
/ip route
add check-gateway=ping distance=1 dst-address=eee.fff.ggg.hhh/32 gateway=vdsl-orange-ether1
There is a src-nat rule for this interface :
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface=vdsl-orange-ether1
Is that not ok ?
Try to create NAT rule before main masquerade rule with chain-accept and source address of LAN to not source nat them.
 
Zoolander06
newbie
Topic Author
Posts: 47
Joined: Thu Jan 03, 2019 5:26 pm

Re: GRE tunnel established, ping ok, but no traffic

Thu Jan 16, 2020 4:29 pm

Hello,

Sorry, I misunderstood the meaning of "include" in my list declaration :
/interface list add include=GRE name=LAN

But I tried with my gre interface in list LAN (same as my LAN interface) without success...

The mangle rule doesn't do anything...

And the working tunnel I had yesterday doesn't work anymore, I don't understand why...

Well, I will continue to try...
 
Zoolander06
newbie
Topic Author
Posts: 47
Joined: Thu Jan 03, 2019 5:26 pm

Re: GRE tunnel established, ping ok, but no traffic

Thu Jan 16, 2020 4:54 pm

It seems indeed to be tcp related, since UDP works well, and ICMP too.
I tried to low MTU value (I tried 1400 and even 1300), but it is not resolving any issues.
I tried the mangle rule to clamp mss to pmtu, nothing.

If I try a ping with "don't fragment" flag, it works up to the MTU value.

An idea ?

Here is what I've done :
after setting up my tunnel with default MTU, I check the maximum packet size with the tool ping : 1440
from a device on the network, the maximum size packet is 1412, I think that is normal.
I tried to enable or disable the "Clamp TCP MSS" option in GRE tunnel configuration : no change.
I tried with and without the mangle rule : no change.
I tried to enable or disable "Allow FastPath" : without this option, I cannot even ping, which is weird.
With the "Allow FastPath" option enabled, I can ping, I can connecte SIP phones over the tunnel in UDP, but not in TCP, and I can't establish any http connections.
 
Zoolander06
newbie
Topic Author
Posts: 47
Joined: Thu Jan 03, 2019 5:26 pm

Re: GRE tunnel established, ping ok, but no traffic

Thu Jan 16, 2020 6:45 pm

I checked what happens in case of an http connection on wan port with packet sniffer :

I see the initial packet from my laptop going out the router, and I see the ACK going back to my router, with correct informations inside.
But, sometimes after, I see retransmissions of my initial packet, which means that the ACK is received by the router, but not transmitted to my laptop...

I double checked by capturing packets from my laptop : the ACK never arrives.

So it seems that for some reasons, the router doesn't forward the received ACK, which is why TCP doesn't work.

And just to be sure the firewall isn't the cause, I've added rules to accept anything coming from and going through my gre interface, it doesn't change anything, but the counter of the rule for what is coming from doesn't increase, so the problem seems to be before the firewall. It's like routeros doesn't decapsulate packets.

So, what could cause that kind of issue ?

Thanks
 
Zoolander06
newbie
Topic Author
Posts: 47
Joined: Thu Jan 03, 2019 5:26 pm

Re: GRE tunnel established, ping ok, but no traffic

Fri Jan 17, 2020 5:26 pm

Hello folks,

I think I solved my problem !
I had to add a filter rule to accept GRE protocol in the input chain, and I had to add it before the default rule dropping invalid connections.

Without this rule, ICMP and UDP works, but only with the "Allow Fast Track" option enabled, with it, I can disallow fast track, and TCP works.

I have a small question, just to understand exactly what happened : why is this rule
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
dropping GRE connections ?

Thanks a lot :)

Joris
 
pe1chl
Forum Guru
Forum Guru
Posts: 6175
Joined: Mon Jun 08, 2015 12:09 pm

Re: GRE tunnel established, ping ok, but no traffic

Fri Jan 17, 2020 7:16 pm

This is due to a recently introduced bug, that was the result of fixing an (apparent) other bug in the firewall handling of GRE.
It was quickly noticed that this problem was introduced at the time but the promised fix has not yet been delivered it seems.
I did not know that it could impact part of GRE traffic, I would think it affected the basic GRE tunnel traffic so everything sent over GRE.
 
Zoolander06
newbie
Topic Author
Posts: 47
Joined: Thu Jan 03, 2019 5:26 pm

Re: GRE tunnel established, ping ok, but no traffic

Mon Jan 20, 2020 11:40 am

Actually, it does affect all the traffic.
But since I had enabled the "allow fast track" option, I think that UDP and ICMP didn't pass through the firewall... (I'm not sure how fast track works exactly, so I'm speculating here)

Thanks for your help :)
 
Zoolander06
newbie
Topic Author
Posts: 47
Joined: Thu Jan 03, 2019 5:26 pm

Re: GRE tunnel established, ping ok, but no traffic

Tue Jan 28, 2020 12:07 am

Hello,

I still have issues with others GRE tunnels...
For now I succeeded to establish a working tunnel between a RB2011 directly connected with PPPoE and another RB2011 behind a NAT router.
So tonight I try to establish another tunnel between the same PPPoE connected RB2011 and another PPPoE connected RB2011, with exactly the same settings, and it doesn't work !

I made multiple tests, and I have found that almost every packets coming in my first router input from the second router have a raw length of only 90 bytes, but the IPv4 total length is much more than that (for example, 576 bytes).
Wireshark display this :
[Expert Info (Error/Protocol): IPv4 total length exceeds packet length (76 bytes)]

So that's probably why nothing works...

Does anybody here have an idea about it ?

Thanks,

Joris

Who is online

Users browsing this forum: gibi13, hero1c, oskarsk and 190 guests