Route traffic from ether interface to wireguard

diniboy · August 8, 2022, 4:34pm

Hi,

I have a traditional single network (192.168.1.0/24) home network powered by a hap ac2. wlan1, wlan2 and the ether interfaces are parts of a bridge.

Now I got a set top box for certain vod and live tv from a local provider connecting on ether2 of the router. It’s working fine, however its often laggy at night I assume due to my ISP’s too saturated link to the other provider.

I want to set up cloudflare warp that provides a working wireguard setup to have a better peering. On the router I have a working wgcf interface, I can now ping and fetch with it.

Then comes the questionable part. I want to route all traffic from the ether2 interface to wgcf but leave the rest of traffic using my ISP’s default gateway. On the other hand I also want to be able to access the box on my local network so the remote control app works.

Therefore I created a NAT masquerade rule for the wgcf interface then created a route mark rule where I specified everything that’s not with the destination network 192.168.1.0/24 and it comes from ether2 should be marked. Finally I created a simple default route for this route mark to wgcf.

The setup works fine, however establishing connections take a hella lot of time on the box while the router’s CPU isn’t used much. Once the connection is established after a few seconds its able to get 100 mbit/s downloads which is fine.

What’s the bottleneck? I know the hap ac2 isn’t a super powerful hw, but I thought it should handle this scenario at least quite decently.

If I set up IP based routing to wgcf, the connections are instantly established and everything is fine. However the provider uses many IPs and I don’t want to maintain an IP list.

I also cannot set up wireguard directly on the set top box as its really closed down.

Any ideas? Much appreciated

anav · August 8, 2022, 5:19pm

Your explanation is confusing
Assuming you have only ONE ISP ?
You use the internet to get TV from a website?
The TV box is setup on ether2.
You want to be able to access ether2 from the local network.
You want the ether2 connected tv box to go out the internet from cloudfare wireguard instance…

What lan network is the tv box using (ip address of the box for example)?

a. a diagram would be helpful
b. the mt config full /export and simply use fake numbers for WANIP or Gateway IP info.
c. the paramters that cloudfare gave you to use. (except any keys of course…and the IP: of the cloudfare just use a fake one etc…)

diniboy · August 9, 2022, 11:12am

Thanks for your quick reply! Find the network diagram attached.

I have only one ISP, the other provider I was referring to was the one that provides the content over the internet. There is no VLANs or internal ISP networks involved, just the internet and some remote servers.

What I want:

STB (set top box) remains accessible on 192.168.1.8
STB can access the local DNS server on 192.168.124.2
Rest of traffic is routed over Wireguard (wgcf)

The config contains way too many sensitive and a lot of useless information which I tried to clean up, but it would have been too long to share. Here are the interesting bits from CLI:

[admin@Redacted] > /interface/wireguard/print detail 
Flags: X - disabled; R - running 
 0  R name="wgcf" mtu=1280 listen-port=13231 private-key="redacted" 
      public-key="redacted"

[admin@Redacted] > /interface/wireguard/peers/print detail where interface=wgcf 
Flags: X - disabled 
 3   interface=wgcf public-key="redacted" 
     endpoint-address=engage.cloudflareclient.com endpoint-port=2408 
     current-endpoint-address=162.159.192.1 current-endpoint-port=2408 allowed-address=0.0.0.0/0 
     rx=495.0MiB tx=4607.9KiB last-handshake=12s

Then I am able to ping a random host over this VPN:

[admin@Redacted] > /ping 1.1.1.1 interface=wgcf count=5
  SEQ HOST                                     SIZE TTL TIME       STATUS                               
    0 1.1.1.1                                    56  64 9ms515us  
    1 1.1.1.1                                    56  64 9ms943us  
    2 1.1.1.1                                    56  64 10ms602us 
    3 1.1.1.1                                    56  64 11ms56us  
    4 1.1.1.1                                    56  64 11ms50us  
    sent=5 received=5 packet-loss=0% min-rtt=9ms515us avg-rtt=10ms433us max-rtt=11ms56us

Then I made sure I have NAT set up:

[admin@Redacted] > /ip/firewall/nat/print detail where out-interface
Flags: X - disabled, I - invalid; D - dynamic 
 0    chain=srcnat action=masquerade out-interface=wan1 log=no log-prefix="" 

 3    chain=srcnat action=masquerade out-interface=wgcf log=no log-prefix=""

I added a “box” mark:

[admin@Redacted] > /routing/table/print 
Flags: D - dynamic; X - disabled, I - invalid; U - used 
 0 D   name="main" fib 

 1     name="box" fib

Then created the following mangle:

[admin@Redacted] > /ip/firewall/mangle/print detail where src-address
Flags: X - disabled, I - invalid; D - dynamic 
 3    ;;; mark packets from STB
      chain=prerouting action=mark-routing new-routing-mark=box passthrough=yes 
      src-address=192.168.1.8 dst-address-list=!LAN log=no log-prefix=""

My LAN address list is as follows:

[admin@Redacted] > /ip/firewall/address-list/print 
Columns: LIST, ADDRESS, CREATION-TIME
# LIST  ADDRESS          CREATION-TIME       
;;; LAN
0 LAN   192.168.1.0/24   aug/08/2022 14:32:57
;;; wireguard roadwarrior
1 LAN   192.168.98.0/24  aug/08/2022 14:33:08
;;; DNS
2 LAN   192.168.124.2    aug/08/2022 14:33:20

And finally created a route:

[admin@Redacted] > /ip/route/print detail where routing-table=box
Flags: D - dynamic; X - disabled, I - inactive, A - active; 
c - connect, s - static, r - rip, b - bgp, o - ospf, d - dhcp, v - vpn, m - modem, y - copy; 
H - hw-offloaded; + - ecmp 
 0  As   dst-address=0.0.0.0/0 routing-table=box pref-src="" gateway=wgcf immediate-gw=wgcf distance=1 
         scope=30 target-scope=10 suppress-hw-offload=no

Now the client’s traffic is indeed going thru the warp tunnel as if I set up the same thing to my laptop and check what IP does it display, I can see that its the cloudflare one. However connection establishment is extremely slow, takes a few seconds until it gets fast.

The wireguard config looks like this:

[Interface]
PrivateKey = redacted
Address = 172.16.0.2/32
Address = fd01:5ca1:ab1e:8ba8:6971:e887:d515:dc23/128
DNS = 1.1.1.1
MTU = 1280
[Peer]
PublicKey = redacted
AllowedIPs = 0.0.0.0/0
AllowedIPs = ::/0
Endpoint = engage.cloudflareclient.com:2408

holvoetn · August 9, 2022, 12:38pm

Question … if the connection to the current ISP is already shaky, how would adding a layer of wireguard on top of that same connection improve things ??
Because logically you may connect from your router to Cloudflare, physically you still pass that same ISP.

And parts of config, that’s a no-no for anav

anav · August 9, 2022, 12:47pm

What belgian waffles said, think raspberry jam, whipped cream and a spanish coffee.
The wireguard connection is to securely connect two spots due to security needs primarily, its not going to help you avoid a shitty ISP connection.
Most questions are answered here…
https://forum.mikrotik.com/viewtopic.php?t=182340

Long configs are not an issue just use the code tags above to make it palatable on the forum the black square with white square brackets on the same line as Bold and Underline for example.
( the only things you need to keep hidden, use fake numbers are public WANIP numbers, any keys, and we dont need to know your real wireguard ports or winbox ports.

diniboy · August 9, 2022, 12:51pm

Cloudflare has a local pop in the country and the ISP has a decent connectivity there. I get better latencies if I ping a random IP almost always over warp as well.

[admin@Redacted] > /ping 9.9.9.9 count=10 interface=wgcf
  SEQ HOST                                     SIZE TTL TIME       STATUS                               
    0 9.9.9.9                                    56  59 13ms89us  
    1 9.9.9.9                                    56  59 9ms539us  
    2 9.9.9.9                                    56  59 10ms827us 
    3 9.9.9.9                                    56  59 8ms722us  
    4 9.9.9.9                                    56  59 8ms901us  
    5 9.9.9.9                                    56  59 9ms551us  
    6 9.9.9.9                                    56  59 8ms890us  
    7 9.9.9.9                                    56  59 8ms587us  
    8 9.9.9.9                                    56  59 9ms559us  
    9 9.9.9.9                                    56  59 8ms783us  
    sent=10 received=10 packet-loss=0% min-rtt=8ms587us avg-rtt=9ms644us max-rtt=13ms89us

Without Cloudflare:

[admin@Redacted] > /ping 9.9.9.9 count=10
  SEQ HOST                                     SIZE TTL TIME       STATUS                               
    0 9.9.9.9                                    56  61 10ms332us 
    1 9.9.9.9                                    56  61 10ms729us 
    2 9.9.9.9                                    56  61 9ms894us  
    3 9.9.9.9                                    56  61 10ms29us  
    4 9.9.9.9                                    56  61 10ms551us 
    5 9.9.9.9                                    56  61 10ms280us 
    6 9.9.9.9                                    56  61 10ms328us 
    7 9.9.9.9                                    56  61 10ms40us  
    8 9.9.9.9                                    56  61 10ms377us 
    9 9.9.9.9                                    56  61 11ms21us  
    sent=10 received=10 packet-loss=0% min-rtt=9ms894us avg-rtt=10ms358us max-rtt=11ms21us

And this was measured during daytime, at the evening the difference is even more visible.

I am sorry then, will try to finish cleaning up the config then later.

PS: changing ISP isn’t an option sadly, there are only worse ones here