An update to all this, that I am still working out bugs and issues an really have been trying to put the final grasp on some of these configuration options.
I'm going to post a small book here an hope that someone can turn on my lightbulb an get me on my way.
When having to deal with Mangle chains I am stumped on the reason for each chain choice can someone please lay this out for me.
The Wiki is rather vague:
chain (name; Default: ) Specifies to which chain rule will be added. If the input does not match the name of an already defined chain, a new chain will be created.
I will put what I feel is my understanding of each an see how wrong I am.
chain=prerouting: header changes; ie connection labels, packet labels, all packets WAN or LAN
chain=input: packets destined for the mikrotik itself.
chain=forward: packets destined for a device behind the mikrotik
chain=output: packets leaving the Mikrotik WAN(s); routing changes etc...
chain=postrouting: Not even sure what to say on this one...
The reason I bring this up is there are so many examples of PCC balancing out there. some use prerouting, some use input, an forward. Then you have some that do a combination of two or even all three, with the same "inbound" rules, just applying a different chain. That to me points to uncertainty of not really knowing exactly which chain you need to apply.
I'm sure someone out there can answer this an provide the description needed for people to choose the right chain for their needs. redundant chains seems like a easy way to raise CPU overhead.
The other issues I am facing is inbound connections destined for devices behind my NAT. I'm experiencing issues where people who share the same WAN2 provider as me are unable to load web content behind my NAT. But at my work for example, I can reach the same content without issue. (not the same provider as my WAN2)
So I did more web crawling. forum reading an TikTube watching an now I'm at a place where nothing from the outside works an my hairpin access doesn't even work anymore.
Connection balancing seems to be working.
So at this point Im going to summarize what I have for connections/network & what services over those connections I want to accomplish.
- WAN1: PPPoE bridged connection w/ server assigned static IP. ~22/2Mb
- WAN2: DHCP assigned NOT static Cable connection; Charter. ~30/4Mb
- LAN: typical household 192.168.1.0/24 NAT'd private network.
- PCC: For increased throughput and connection failover (either direction; I understand if the DSL goes down, my hosting goes down)
- Web hosting: I host a very basic website from my home via the DSL connection on a Synology NAS.
I would like HTTP 80 requests to accept/return connections over the DSL(mangle), but would also like the NAS to be able to perform its own internet requests over both WANs for increased throughput. So I've ruled out src address routing the NAS to only the DSL for that reason.
- Gaming: I'm also a gamer, an I would think that allowing a Friend to connect to me shouldn't be a issue if all of my mangle rules to keep the connection stuck to one WAN or the other should allow this, and that I have the correct ports opened on the firewall. (which was never a issue prior to attempting PCC)
Here is my current config prints relative to the connections/routing
Filter
[admin@RB2011] /ip firewall> filter print
Flags: X - disabled, I - invalid, D - dynamic
0 ;;; Input: Drop Invalid | LAN access | LAN established | LAN related | ping | drop
chain=input action=drop connection-state=invalid
1 chain=input action=accept connection-state=new in-interface=ether1-Network
2 chain=input action=accept connection-state=established
3 chain=input action=accept connection-state=related
4 chain=input action=accept protocol=icmp
5 chain=input action=drop
6 ;;; Forward: Drop Invalid | LAN access | LAN established | LAN related | drop
chain=forward action=drop connection-state=invalid
7 chain=forward action=accept connection-state=new in-interface=ether1-Network
8 chain=forward action=accept connection-state=established
9 chain=forward action=accept connection-state=related
10 chain=forward action=drop
Mangle Notes
address list "LAN" is just the private 192.168.1.0/24 network, I don't have static WAN routes due to PPPoE and DHCP connections, so I can't specify those IPs like used in Manual: PCC page.
Eventhough the PPPoE is a static IP, its server assigned static, so I pick it up dynamically when the PPP connects. So I at least told it to quit processing LAN destined traffic.
/ ip firewall mangle
add chain=prerouting dst-address=10.111.0.0/24 action=accept in-interface=LAN
add chain=prerouting dst-address=10.112.0.0/24 action=accept in-interface=LAN
With policy routing it is possible to force all traffic to the specific gateway, even if traffic is destined to the host (other that gateway) from the connected networks. This way routing loop will be generated and communications with those hosts will be impossible. To avoid this situation we need to allow usage of default routing table for traffic to connected networks.
Ive yet to see a ping/tracert go out the other WAN when destined for that WAN's network.
[admin@RB2011] /ip firewall mangle> print
Flags: X - disabled, I - invalid, D - dynamic
0 ;;; Accept LAN
chain=prerouting action=accept src-address-list=LAN dst-address-list=LAN
1 ;;; Connection Marks
chain=prerouting action=mark-connection new-connection-mark=PPPoE_conn passthrough=yes in-interface=PPPoE-WAN1 connection-mark=no-mark
2 chain=prerouting action=mark-connection new-connection-mark=WAN2_conn passthrough=yes in-interface=ether8-WAN2 connection-mark=no-mark
3 ;;; Connection Splitting
chain=prerouting action=mark-connection new-connection-mark=PPPoE_conn passthrough=yes dst-address-type=!local in-interface=ether1-Network connection-mark=no-mark
per-connection-classifier=both-addresses:2/0
4 chain=prerouting action=mark-connection new-connection-mark=WAN2_conn passthrough=yes dst-address-type=!local in-interface=ether1-Network connection-mark=no-mark
per-connection-classifier=both-addresses:2/1
5 ;;; Packet Routes
chain=prerouting action=mark-routing new-routing-mark=WAN1-Route passthrough=yes in-interface=ether1-Network connection-mark=PPPoE_conn
6 chain=prerouting action=mark-routing new-routing-mark=WAN2-Route passthrough=yes in-interface=ether1-Network connection-mark=WAN2_conn
7 ;;; Outbound Packet Marks
chain=output action=mark-routing new-routing-mark=WAN1-Route passthrough=yes connection-mark=PPPoE_conn
8 chain=output action=mark-routing new-routing-mark=WAN2-Route passthrough=yes connection-mark=WAN2_conn
Routes
[admin@RB2011] /ip route> print detail
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit
0 A S dst-address=0.0.0.0/0 gateway=PPPoE-WAN1 gateway-status=PPPoE-WAN1 reachable check-gateway=ping distance=1 scope=30 target-scope=10 routing-mark=WAN1-Route
1 A S dst-address=0.0.0.0/0 gateway=ether8-WAN2 gateway-status=ether8-WAN2 reachable check-gateway=arp distance=1 scope=30 target-scope=10 routing-mark=WAN2-Route
2 A S dst-address=0.0.0.0/0 gateway=PPPoE-WAN1 gateway-status=PPPoE-WAN1 reachable distance=2 scope=30 target-scope=10
3 S dst-address=0.0.0.0/0 gateway=ether8-WAN2 gateway-status=ether8-WAN2 reachable distance=3 scope=30 target-scope=10
4 ADC dst-address=10.255.1.1/32 pref-src=[WAN1 IP ADDR] gateway=PPPoE-WAN1 gateway-status=PPPoE-WAN1 reachable distance=0 scope=10
5 ADC dst-address=[WAN2 IP ADDR] pref-src=[WAN2 IP SRC] gateway=ether8-WAN2 gateway-status=ether8-WAN2 reachable distance=0 scope=10
6 ADC dst-address=192.168.1.0/24 pref-src=192.168.1.xxx gateway=ether1-Network gateway-status=ether1-Network reachable distance=0 scope=10
NAT - It's my understanding that if Mangle is setup properly, I don't need to use any marking here, just treat it like the only connection available, an worked fine with the DSL only.
chain=dstnat action=dst-nat to-addresses=192.168.1.xxx to-ports=80 protocol=tcp dst-address=[WAN1 IP addr.] in-interface=PPPoE-WAN1 dst-port=80
Hairpin to allow me to hit my site from the inside by name, which worked up until my attempts to get the outside working fully.
chain=srcnat action=masquerade protocol=tcp src-address=192.168.1.0/24 dst-address=192.168.1.xxx out-interface=ether1-Network dst-port=80
An a example hole for Torchlight 2 game hosting. Which again, worked without issue prior to 2nd Connection an PCC.
;;; TL2
chain=dstnat action=dst-nat to-addresses=192.168.1.xxx to-ports=4171-4179 protocol=udp dst-port=4171-4179
So currently HTTP requests are timing out from anywhere, game hosting isn't working, but the connections seem to be balancing.
An I won't find out for sure until I try an post this message if secure website logins are broken or not. (was working prior to latest changes)
Any and all help would be appreciated, thank you.