Hi Guys
I have a simple network layout. Using a RB11AH with v6.13 which behaves weirdly after enabling policy based load balanced routing this weekend.
Here is an image to help explain the layout of the physical network:
On the wireless highsite I have a few clients (about 30) that terminate directly to the RB1100 using l2tp. They also have their default gateway set to the l2tp connection. These clients all get assigned an IP in the 10.100.100.0/24 range with the RB using .1 for itself. There are 3 ADSL Dialups that these clients use for Internet breakout. Then there is also a Static Fiber Connection that the RB's own default route is set to and everything not marked to route via the ADSL connections, will be default route via the fiber. That all works (kinda).
This weekend I configured load balanced policy based routing as follows (Very simple actually).
_These are my mangle rules that mark the connections and then marks the routing in a round-robin like way based on those connection marks._8 ;;; Mark ADSL 1 Connections
chain=prerouting action=mark-connection new-connection-mark=route-via-adsl-line-1 passthrough=yes connection-state=new src-address=10.100.100.0/24 in-interface=!ether6-core-network
per-connection-classifier=src-address:3/0
9 chain=prerouting action=mark-routing new-routing-mark=route-via-ADSL1 passthrough=no connection-state=new src-address=10.100.100.0/24 connection-mark=route-via-adsl-line-1
10 ;;; Mark ADSL 2 Connections
chain=prerouting action=mark-connection new-connection-mark=route-via-adsl-line-2 passthrough=yes connection-state=new src-address=10.100.100.0/24 in-interface=!ether6-core-network
per-connection-classifier=src-address:3/1
11 chain=prerouting action=mark-routing new-routing-mark=route-via-ADSL2 passthrough=no connection-state=new src-address=10.100.100.0/24 connection-mark=route-via-adsl-line-2
12 ;;; Mark ADSL 3 Connections
chain=prerouting action=mark-connection new-connection-mark=route-via-adsl-line-3 passthrough=yes connection-state=new src-address=10.100.100.0/24 in-interface=!ether6-core-network
per-connection-classifier=src-address:3/2
13 chain=prerouting action=mark-routing new-routing-mark=route-via-ADSL3 passthrough=no connection-state=new src-address=10.100.100.0/24 connection-mark=route-via-adsl-line-3_Then, my routing table looks like this. The 3 ADSL pppoe interfaces obviously is matched based on the routing mark._
DST-ADDRESS PREF-SRC GATEWAY DISTANCE
0 A S 0.0.0.0/0 pppoe-ADSL1 1
1 A S 0.0.0.0/0 pppoe-ADSL2 1
2 A S 0.0.0.0/0 pppoe-ADSL3 1
3 A S ;;; My own default route
0.0.0.0/0 41.X(hidden).X(hidden).X(hidden) 1Now you will notice that I did include a Internal Linux server in the drawing. The issue I have is the following two things:
- The Linux server has an OpenVPN connection to the outside world on udp port 1194. It works, and traffic from that server (is not natted, the RB knows how to route to it via the pfsense firewall) routes properly via the default gateway on the RB which is the fiber connection. But if I ssh from the server to another OpenVPN host the connection breaks after a few seconds. I did confirm that the connection is not marked as an ADSL1, 2 or 3 connection so the round-robin routing shouldn't apply to it but I think somehow it does.
- If I do a traceroute from my l2tp clients, I can see every now and then some packets do route via the fiber. Not a lot, but some do. And as far as I know, with the config I have that should not happen at all. Each client coming from a 10.100.100.0/24 IP should have their routing marked to route via the ADSL connections only.
Can someone please point me in the right direction here?
Thank you very much.