IPv6 issues via HE tunnel

RouterOS General
1

Hi,

I have a CCR1009-7G-1C-1S+ router with HE IPv6 tunnel. Ping6 works fine, but seemingly nothing else.
Do you have any idea what could be the problem? I could setup successfully HE tunnel in my other flat.
Full router config can be found here: https://gist.github.com/halacs/7194ff37a373fe7d9a5fc7e7ffbfe73e

ICMPv6 is allowed in IPv6 firewall.

I have read that HTTPS issues usually happens in case of wrong MTU settings so this was my first suspect. Previously I had IPv4 MTU issues causing HTTPS connection problems, but I have solved it before.

Might useful info:

  • HE tunnel MTU is 1280.
  • I have a EoIP tunnel between my two flats. IPv6 works fine in flat “B” but in flat “A”.
  • I have PPPoE internet connection in flat “A” where IPv6 problem happens. PPPoE MTU is 1480 and I couldn’t set higher.
2

Did you check that protocol 41 is not blocked in-transit (in- and outbound to HE)?
Have you cross-checked the IPv4 addresses to be static on both ends?

3

If protocol 41 is blocked then no IPv6 ping should work.
IPv4 addresses at both end of the tunnel should be also fine as IPv6 ping works fine. If I disable the tunnel interface no ping anymore.
Please correct me if I’m wrong.

4

Hey. You should advertise your IPv6 /64 prefixes in your LAN. And in IPv6 - ND you should enable advertise mac address and DNS. Also you should write ipv6 dns servers in ip - dns. You don’t have them either.

5

ah, sorry, it seems I attached configuration when I really disabled IPv6 address advertisement. IPv6 advertisement was on on the vlan-local interface where IPv6 address itself is now disabled to avoid full internet outage at the clients.
I have tried to add IPv6 DNS servers and enable MAC and DNS advertisements in ND, but I will double test it. Thanks for the hint! Lets see!

6

Sure! Hope you will find the issue.

7

Well, it didn’t help.
I have set what you suggested but problem still exists: ipv6 connection is slow and not all the https pages are loading.
Downloading google.com on ipv6 is wrong, but on ipv4: curl -6 -v -L google.com is stuck at https handshake but curl -4 -v -L google.com quickly gives back the html code.




$ ping6 google.com
PING google.com(bud02s27-in-x0e.1e100.net (2a00:1450:400d:804::200e)) 56 data bytes
64 bytes from bud02s27-in-x0e.1e100.net (2a00:1450:400d:804::200e): icmp_seq=1 ttl=57 time=14.7 ms
64 bytes from bud02s27-in-x0e.1e100.net (2a00:1450:400d:804::200e): icmp_seq=2 ttl=57 time=20.3 ms
64 bytes from bud02s27-in-x0e.1e100.net (2a00:1450:400d:804::200e): icmp_seq=3 ttl=57 time=16.8 ms
^C
--- google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 14.786/17.346/20.376/2.306 ms

$ curl -6 ipv6-test.com -vL
* Rebuilt URL to: ipv6-test.com/
*   Trying 2001:41d0:701:1100::29c8...
* TCP_NODELAY set
* Connected to ipv6-test.com (2001:41d0:701:1100::29c8) port 80 (#0)
> GET / HTTP/1.1
> Host: ipv6-test.com
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Date: Sat, 14 Dec 2019 07:11:20 GMT
< Server: Apache/2.4.25 (Debian)
< Location: https://ipv6-test.com/
< Content-Length: 0
< Content-Type: text/html; charset=UTF-8
<
* Connection #0 to host ipv6-test.com left intact
* Issue another request to this URL: 'https://ipv6-test.com/'
* Connection 0 seems to be dead!
* Closing connection 0
*   Trying 2001:41d0:701:1100::29c8...
* TCP_NODELAY set
* Connected to ipv6-test.com (2001:41d0:701:1100::29c8) port 443 (#1)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
^C
8

Auch, I have pasted wrong curl call. Curl with google.com is not working even in case of http.

$ time curl -6 -v -L google.com
* Rebuilt URL to: google.com/
*   Trying 2a00:1450:400d:808::200e...
* TCP_NODELAY set
* Connected to google.com (2a00:1450:400d:808::200e) port 80 (#0)
> GET / HTTP/1.1
> Host: google.com
> User-Agent: curl/7.58.0
> Accept: */*
>
^C

real    0m29.183s
user    0m0.005s
sys     0m0.009s
9

When your MTU is not 1500 you should enter the correct MTU in the ND settings or else you will have problems like this.
It is better to make a native IPv6 connection instead of such a tunnel.

10

Thanks for the ideas!

I have changed MTU in ND as showed on the below picture but didn’t help, even after a reboot on the client PC side then on the router side as well.

Yes, native IPv6 would be better but as a Hungarian Telecom subscriber I can get native IPv6 address only via zero configuration. My Windows PC can get IPv6 address directly from the modem but I could not configure my Mikrotik router to do so.

11

You could try to make a packet sniffer trace on the tunnel and see if the TCP SYN packets contain the correct MSS value.

12

I started a packet sniffer on the tunnel interface.

I can see a TCP SYN MSS=1220 packet then a TCP SYN ACK mss=1220 (between my PC and google.com)
This seems to be good if 1280 is the MTU on the tunnel. Right?

What is interesting for me is the TCP Retransmission part.

13

Yes, this is what you want to see.

What is interesting for me is the TCP Retransmission part.

That is not good! The other end apparently does not respect the MSS, or the datapackets are dropped for some other reason.

14

So this is the point when I have to contact the tunnel support. Okay, thanks. I will contact them and come back to share the solution.

15

I got answer from the support:

We don’t have any firewall rules on our side configured for any tunnel, except SMTP and IRC filters as exist by default.
– -H U R R I C A N E - E L E C T R I C-

16

I have no idea. MTU 1280 on the IPv6 interface should be low enough to pass the largest packet through a tunnel even with the limited MTU of 1480 you have.
That is why I suggested that the other side may ignore the MSS and still send full-size (1500 byte) packets which do not fit the 1480 byte internet MTU.
I have lots of experience with different types of tunnels in combination with such limited MTU sizes imposed by PPPoE and it always is a headache, but lowering the MSS normally solves it for TCP.
No idea why it still does not work for you.

17

Thanks a lot pe1chl!

18

Hey again. Did you try to connect to another web sites?

19

Hi,

yes, sure! I pase few other tests below.

I have an Ubuntu server with working ipv6 somewhere else. I have curl-ed it while tcpdump was writing pcap file. Here I also found “TCP Retransmission” lines.
In contrast with the previous sniff what happened locally in my Mikrotik router, SYN ACK MSS is not as expected:
SYN MSS=1220
SYN ACK MSS=1440




gabor@home:~$ time curl -vL6 index.hu
* Rebuilt URL to: index.hu/
*   Trying 2a02:730::1860...
* TCP_NODELAY set
* Connected to index.hu (2a02:730::1860) port 80 (#0)
> GET / HTTP/1.1
> Host: index.hu
> User-Agent: curl/7.58.0
> Accept: */*
>
^C

real    0m40.441s
user    0m0.007s
sys     0m0.007s
130 gabor@home:~$ time curl -vL6 origo.hu
* Rebuilt URL to: origo.hu/
*   Trying 2001:4c48:16:6::2:1c...
* TCP_NODELAY set
* Connected to origo.hu (2001:4c48:16:6::2:1c) port 80 (#0)
> GET / HTTP/1.1
> Host: origo.hu
> User-Agent: curl/7.58.0
> Accept: */*
>
^C

 gabor@home:~$ time curl -vL6 forum.mikrotik.com
* Rebuilt URL to: forum.mikrotik.com/
*   Trying 2a02:610:7501:1000::205...
* TCP_NODELAY set
* Connected to forum.mikrotik.com (2a02:610:7501:1000::205) port 80 (#0)
> GET / HTTP/1.1
> Host: forum.mikrotik.com
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Server: nginx
< Date: Mon, 16 Dec 2019 10:59:57 GMT
< Content-Type: text/html
< Content-Length: 178
< Connection: keep-alive
< Keep-Alive: timeout=5
< Location: https://forum.mikrotik.com/
< Strict-Transport-Security: max-age=31536000
<
* Ignoring the response-body
* Connection #0 to host forum.mikrotik.com left intact
* Issue another request to this URL: 'https://forum.mikrotik.com/'
* Connection 0 seems to be dead!
* Closing connection 0
*   Trying 2a02:610:7501:1000::205...
* TCP_NODELAY set
* Connected to forum.mikrotik.com (2a02:610:7501:1000::205) port 443 (#1)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
^C

real    0m25.049s
user    0m0.028s
sys     0m0.000s

130 gabor@home:~$ ping ipv6.google.com
PING ipv6.google.com(bud02s26-in-x0e.1e100.net (2a00:1450:400d:809::200e)) 56 data bytes
64 bytes from bud02s26-in-x0e.1e100.net (2a00:1450:400d:809::200e): icmp_seq=1 ttl=57 time=15.7 ms
64 bytes from bud02s26-in-x0e.1e100.net (2a00:1450:400d:809::200e): icmp_seq=2 ttl=57 time=15.3 ms
^C
--- ipv6.google.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 15.332/15.530/15.729/0.234 ms

gabor@home:~$ ping forum.mikrotik.com
PING forum.mikrotik.com(forum.mikrotik.com (2a02:610:7501:1000::205)) 56 data bytes
^C
--- forum.mikrotik.com ping statistics ---
7 packets transmitted, 0 received, 100% packet loss, time 6130ms
20

I will try to catch my tcp exchange today on normal web serfing via ipv6, but for now I think it is high delay between ack segments in your exchange. Try to sniff that on ipv4(if your server has ipv4), will you find the difference? Also you have spur retransmissions, which means that you’ve already saw that ack segment from server, but server received your first ack too late - delayed and server had to send it again.

What mtu and tcp mss do you have in your server config?