Community discussions

MikroTik App
 
clan
just joined
Topic Author
Posts: 6
Joined: Sat Feb 20, 2021 1:01 pm

Simple HE 6to4 tunnel can Tx but not Rx

Sat Feb 20, 2021 1:34 pm

I am strugling to properly configure a HE 6to4 tunnel. Traffic seems to go out but it does not look like anything is returned.

RouterOS v6.48.1 (stable). The config has been factory reset and few things have been changed. Typical home setup with cable modem (Sagem Fast 3890V3) in bridge mode. I have a fixed public IPv4 (no NAT44/CGNAT) but no native IPv6.

I have tested with the default MTU of 1480 set on the Hurricane Electric Avanced Tunnel Options page with the same results but for now it is set to 1280.

$MYLOCALIP4 is my public static IPv4 address (part of AS203953)
$HETRANSPORTNET is my assigned by HE part of the tunnel

The setup is completely standard and a copy paste of the HE example config:
/interface 6to4 add comment="Hurricane Electric IPv6 Tunnel Broker" disabled=no local-address=$MYLOCALIP4 mtu=1280 name=sit1 remote-address=216.66.80.90
/ipv6 route add comment="" disabled=no distance=1 dst-address=2000::/3 gateway=2001:470:27:$HETRANSPORTNET::1 scope=30 target-scope=10
/ipv6 address add address=2001:470:27:$HETRANSPORTNET::2/64 advertise=no disabled=no eui-64=no interface=sit1
  • On my local router I can ping6 my end of the tunnel (2001:470:27:$HETRANSPORTNET::2)
  • From an external site I can ping6 the remote end of the tunnel (2001:470:27:$HETRANSPORTNET::1)
  • On my local router I cannot ping6 the remote end of the tunnel (2001:470:27:$HETRANSPORTNET::1)
  • From an external site I cannot ping6 my end of the tunnel (2001:470:27:$HETRANSPORTNET::2)
  • From an external site I can ping4 $MYLOCALIP4

So the basic test of tunnel connectivity utterly fails.

My first idea was to look at the firewall.

I cleared "/ipv6 firewall address-list" and "/ipv6 firewall filter".

Just to be sure I also tried with:
/ipv6 firewall filter print
Flags: X - disabled, I - invalid, D - dynamic
 0    chain=input action=accept log=yes log-prefix=""

 1    chain=output action=accept log=yes log-prefix=""

 2    chain=forward action=accept log=no log-prefix=""
Obviously I need to take care of the IPv4 firewall as well. I do not feel comfortable to completely wipe the IPv4 filter rules so I have kept the default rules. To allow for tunnel traffic I have added rule 1-3 to handle protocol 41.
 /ip firewall filter print
Flags: X - disabled, I - invalid, D - dynamic
 0  D ;;; special dummy rule to show fasttrack counters
      chain=forward action=passthrough

 1 X  chain=input action=accept protocol=ipv6-encap log=yes log-prefix=""

 2    chain=input action=accept protocol=ipv6-encap src-address=216.66.80.90 log=yes log-prefix=""

 3    chain=output action=accept protocol=ipv6-encap log=yes log-prefix=""

 4    ;;; defconf: accept established,related,untracked
      chain=input action=accept connection-state=established,related,untracked

 5    ;;; defconf: drop invalid
      chain=input action=drop connection-state=invalid log=no log-prefix=""

 6    ;;; defconf: accept ICMP
      chain=input action=accept protocol=icmp log=no log-prefix=""

 7    ;;; defconf: accept to local loopback (for CAPsMAN)
      chain=input action=accept dst-address=127.0.0.1

 8    ;;; defconf: drop all not coming from LAN
      chain=input action=drop in-interface-list=!LAN log=no log-prefix=""

 9    ;;; defconf: accept in ipsec policy
      chain=forward action=accept ipsec-policy=in,ipsec

10    ;;; defconf: accept out ipsec policy
      chain=forward action=accept ipsec-policy=out,ipsec

11    ;;; defconf: fasttrack
      chain=forward action=fasttrack-connection connection-state=established,related

12    ;;; defconf: accept established,related, untracked
      chain=forward action=accept connection-state=established,related,untracked

13    ;;; defconf: drop invalid
      chain=forward action=drop connection-state=invalid log=no log-prefix=""

14    ;;; defconf: drop all from WAN not DSTNATed
      chain=forward action=drop connection-state=new connection-nat-state=!dstnat in-interface-list=WAN log=no log-prefix=""
Rule 1 has been for testing and rule 2 & 3 was what I thought should be enough to allow tunnel traffic.

When I ping from my end I see traffic on rule 3 but no return traffic on rule 2 (or 1).

When I look at sit1 I see Tx bytes and packets but no Rx. There are no Tx/Rx drops or errors.

My next guess was that NAT was playing games. I just have the default masquerade setup
/ip firewall nat print
Flags: X - disabled, I - invalid, D - dynamic
 0    ;;; defconf: masquerade
      chain=srcnat action=masquerade out-interface-list=WAN ipsec-policy=out,none
I tried to setup a rule to monitor this
/ip firewall mangle
add action=log chain=prerouting in-interface=ether1 log=yes protocol=ipv6-encap
I am a bit unsure whether this is correct as I see no traffic (bytes/packets) on this rule.

What I see in the log:
output: in:(unknown 0) out:sit1, proto ICMP (type 128, code 0), 2001:470:27:$HETRANSPORTNET::2->2001:470:27:$HETRANSPORTNET::1, len 10
output: in:(unknown 0) out:ether1, proto 41, $MYLOCALIP4->216.66.80.90, len 70
I have tried logging on the drop rules just to be sure but I see nothing dropped on proto 41.

And the connection status when I ping6 from my end to the far end. The rest of the time I see nothing.
/ip firewall connection print detail where protocol=ipv6-encap
Flags: E - expected, S - seen-reply, A - assured, C - confirmed, D - dying, F - fasttrack, s - srcnat, d - dstnat
 0    C     protocol=ipv6-encap src-address=$MYLOCALIP4 dst-address=216.66.80.90 reply-src-address=216.66.80.90 reply-dst-address=$MYLOCALIP4 timeout=9m59s orig-packets=15 orig-bytes=1 050 orig-fasttrack-packets=0
            orig-fasttrack-bytes=0 repl-packets=0 repl-bytes=0 repl-fasttrack-packets=0 repl-fasttrack-bytes=0 orig-rate=560bps repl-rate=0bps
To me it looks like my traffic goes out and looks correct but nothing comes back. Hurricane Electric claims their site is up and from an external location I am able to ping6 to the remote end of the tunnel (2001:470:27:$HETRANSPORTNET::1). The hypothesis is then that the tunnel broker is working.

The next hypothesis is then that something between me and them does not like to get protocol 41 traffic to me.

I then construct a 6to4 address from my public IPv4 address using:
ipv4="$MYLOCALIP4"; printf "2002:%02x%02x:%02x%02x::1\n" `echo $ipv4 | tr "." " "
The constructed IPv6 resolves to the same location as my IPv4 on https://www.ip2location.com/demo/

I then try to ping6 this address from an external site to see if I get *any* incoming protocol 41 traffic (enabling rule 1 in the above firewall config). And I see nothing.

All this leads me to the highly unlikely conclusion that my ISP is mangling protocol 41. I do not feel confident enough in my abilities to make such a bold claim.

Occam's razor is usually correct: I am probably doing something stupid.

This should be a rather simple and common setup. What am I doing wrong? Are there other troubleshooting steps I could perform before looking to wireshark?

When I am doing stupid things I can usually find a lot of people with the same problems in the Internet. But in this case I have only been able to find a couple of suggestions related to connection tracking which I do not think is my problem.

6to4 tunnel & source NAT:
viewtopic.php?t=105348

6in4 tunnel with wrong source addres:
viewtopic.php?t=171496

The one I found which looked pretty much like my issue ended up being over LTE which for sure have a lot of "stuff" in the traffic path.

IPV6 Tunnel (6in4) not receiving any data - transmit works
viewtopic.php?t=110868

Any help would be highly appreciated. I have sunk counless hours into this and got none the wiser.
 
clan
just joined
Topic Author
Posts: 6
Joined: Sat Feb 20, 2021 1:01 pm

Re: Simple HE 6to4 tunnel can Tx but not Rx

Thu Feb 25, 2021 2:40 pm

Update - tried to do a quick snif.
/tool sniffer quick ip-protocol=41
INTERFACE                                   TIME    NUM DIR SRC-MAC           DST-MAC           VLAN   SRC-ADDRESS                         DST-ADDRESS                         PROTOCOL   SIZE CPU FP
ether1                                   140.909      1 ->  6C:3B:6B:29:33:4D 34:6A:C2:3F:1E:EE        $MYLOCALIP4                         216.66.80.90                        ip:ipv6...   84   0 no
ether1                                    141.92      2 ->  6C:3B:6B:29:33:4D 34:6A:C2:3F:1E:EE        $MYLOCALIP4                         216.66.80.90                        ip:ipv6...   84   0 no
ether1                                   142.931      3 ->  6C:3B:6B:29:33:4D 34:6A:C2:3F:1E:EE        $MYLOCALIP4                         216.66.80.90                        ip:ipv6...   84   0 no
....
As I interpret this I get packets out on ether1 but I am getting nothing back.

Would I be reasonable to complain to my ISP at this point?

I know it is hard proving a negative. But have I done enough on my end? Is there anything obvious I am missing?
 
clan
just joined
Topic Author
Posts: 6
Joined: Sat Feb 20, 2021 1:01 pm

Re: Simple HE 6to4 tunnel can Tx but not Rx

Thu Feb 25, 2021 3:46 pm

I saw that Telia a few days ago had a routing loop which could give the same symptoms (https://forums.he.net/index.php?topic=4080.0)

I then considered contacting my ISP with that in mind when I got this surprise:
/tool traceroute 216.66.80.90
 # ADDRESS                          LOSS SENT    LAST     AVG    BEST   WORST STD-DEV STATUS
 1 10.203.48.1                        0%   48  14.2ms      15     7.7    27.7     3.8
 2 10.242.131.42                      0%   48   8.8ms    10.4     6.6    24.8     3.5
 3 10.0.30.45                         0%   48  12.2ms    11.8     6.3    29.5     3.5
 4 87.61.121.170                      0%   48  12.1ms    10.8     6.7    15.1       2
 5 83.88.19.117                       0%   48   9.1ms    12.3     8.1    32.4     4.8
 6 212.237.192.187                    0%   48  15.3ms    21.2     7.6      62    11.7
 7 184.104.194.221                    0%   48    22ms    21.7    15.2    32.4     4.1
 8 216.66.80.90                       0%   48  18.3ms    19.7    13.9    29.9     3.2
So now my first 3 hops are RFC 1918 addresses. To me this indicates that my ISP is not giving me a clean IP anymore. I do have a proper public IP on my interface but some trickery is going on.
 
tdw
Forum Guru
Forum Guru
Posts: 1847
Joined: Sat May 05, 2018 11:55 am

Re: Simple HE 6to4 tunnel can Tx but not Rx

Thu Feb 25, 2021 4:15 pm

I use HE based on the example configuration without any issues, currently on 6.47.9 (long-term). I don't have any specific input firewall rules for the IPv4 encapsulated tunnel packets - the input chain established,related rule permits inbound tunnel traffic once outbound traffic has established a connection-tracking entry.

Having RFC1918 addresses in an outbound traceroute is OK, the ISP has hopefully ensured they never appear in an inbound traceroute as they would be unreachable from the internet. It does look as though your ISP is blocking protocol 41 somewhere, can you successfully reach IPv4 services on your router from the internet?
 
clan
just joined
Topic Author
Posts: 6
Joined: Sat Feb 20, 2021 1:01 pm

Re: Simple HE 6to4 tunnel can Tx but not Rx

Thu Feb 25, 2021 4:46 pm

Thank you so much for that confirmation!

The inbound traceroute does not look happy:
[14:33:57] clan@host:/home/clan$ traceroute $MYLOCALIP4
traceroute to $MYLOCALIP4 ($MYLOCALIP4), 64 hops max, 40 byte packets
 1  ae20-114.ar4tdm2nxf1.dk.ip.tdc.net (193.162.159.66)  33.694 ms  15.117 ms  0.178 ms
 2  ae1-0.khk7nqp8.dk.ip.tdc.net (83.88.12.15)  2.659 ms  2.753 ms  2.813 ms
 3  cpe.ae20-0.khk7nqp8.dk.customer.tdc.net (87.61.121.169)  2.619 ms  2.640 ms  3.076 ms
 4  * * *
....
64  * * *
Technically this is the same network operator but when I try from another location in Germany I get the same result after reaching cpe.ae20-0.khk7nqp8.dk.customer.tdc.net (87.61.121.169)

I can ping myself from the outside and have verified that I see the real src and dst address using:
/tool sniffer quick ip-protocol=icmp
INTERFACE                                                                                        TIME    NUM DIR SRC-MAC           DST-MAC           VLAN   SRC-ADDRESS                         DST-ADDRESS                         PROTOCOL   SIZE CPU FP
ether1                                                                                          0.695      1 <-  34:6A:C2:3F:1E:EE 6C:3B:6B:29:33:4D        $REMOTEIP4                          $MYLOCALIP4                         ip:icmp      98   0 no
ether1                                                                                          0.695      2 ->  6C:3B:6B:29:33:4D 34:6A:C2:3F:1E:EE        $MYLOCALIP4                         $REMOTEIP4                          ip:icmp      98   0 no
I have written to my ISP and is currently waiting to traverse 1st level support.
 
clan
just joined
Topic Author
Posts: 6
Joined: Sat Feb 20, 2021 1:01 pm

Re: Simple HE 6to4 tunnel can Tx but not Rx

Fri Mar 05, 2021 1:39 pm

Update:

Avoided level 1 hell and got usable answers from the ISP:

1) Traceroute has RFC 1918 addresses on the way out and is not mapped nicely on the way in. They confirmed that I am not touched by NAT
2) They verified there was no routing loop.
3) Surprise!!!

They confirmed they have mangled the traffic. No explanation on how, what, why. They have queued my CPE for an update 16/3 which should allow 6to4. Whether this works is yet to be seen. And for now it is only my assumption it will work for bridge mode.

I am following up with my ISP to get information on what other protocols they're mangling. I do not question their right to do so. I am just very unhappy that they are not up-front about it.

This then might not be so relevant for a Mikrotik forum. But I will at least update when I get this working. Maybe it can help others argue with their ISP if they have issues.
 
clan
just joined
Topic Author
Posts: 6
Joined: Sat Feb 20, 2021 1:01 pm

Re: Simple HE 6to4 tunnel can Tx but not Rx  [SOLVED]

Wed Mar 17, 2021 11:58 am

Those bastards!

My ISP did indeed mangle my traffic and stopped protocol 41 (6to4). There is no public information on this.

In Denmark the primary provider of Internet via coax/Cable-TV is YouSee/TDC NET. Then there is a number of resellers on this infrastructure. In my case this is Hiper. They claimed this was a "troublesome answer" - they did not interfere but their subcontractor did. They do however admit to this anywhere on their homepage.

They queued me for a new firmware which now has solved my problem. The Sagem FAST 3890v3 CPE is now running firmware "FAST3890V3_TDC-SIP_sw18.76.10.11i-2" in bridge mode. If any other changes have been made is unknown to me.

After a bit of digging I found that TDC NET actually documents what traffic they mangle. This is however the wholesale provider and not something you find without some inside information:

https://wholesale.tdc.dk/da/produkter/b ... %20aftaler

In "Bilag 1a: Produktspecifikation", we see:
4.5.4 Spærrede porte og protokoller

TCP Port: 135, 139, 445, 548
UDP Port: 135, 137, 138
Protokol: 53, 55, 77, 103
Again this is not brought to attention on their homepage and protocol 41 (6to4) is not even mentioned.

I am still waiting on an answer from my ISP why this is not specified. I contacted another ISP on the same net and they upddated their support page within 3 days. They did however not confirm port 41 so I am not switching for now.

For others in similar situation the main Mikrotik lesson for me was the huge value of:
/tool sniffer quick ip-protocol=41

Who is online

Users browsing this forum: Benzebub, matiss, raiser and 77 guests