Page 1 of 1

TLS 1.3 + dual WAN session drops

Posted: Tue Oct 08, 2019 6:13 pm
by NetWorker
Hi all,

we've been using mangle and nth (connection marks+routing marks) with great success over the years. This makes perfect use of our two ISP lines (cable+DSL) and assymetrically distributes the traffic (3 to 1) as they're of different bandwidth.

Now TLS 1.3 comes along and many TLS hosts check the src-address and drop the session when they detect that connections come from two different IP addresses. In other words, upon first access to a particular website, like a bank, the site loads fine but when clicking a hyperlink to load a different part of said website, nth cycles, some part of the website is requested over a "new" connection, it goes out the other ISP line and the user gets kicked out to the login page.

This has been a problem for some time now but steadily growing worse. So far we've been manually wiresharking connections to said websites to identify their addresses and add them to a list that excludes them from nth processing.

However this now needs to be automated. The number of sites presenting the problem has increased to the point where the above action is no longer practical. The easy solution would be to fire off all https connections from a single WAN with failover but almost every site including google drive, youtube, etc now use https so this isn't a practical solution either (we'd saturate that WAN).

I'm thinking of adding individual user addresses (internal DHCP assigned) to lists and have each user use that particular ISP line for all https traffic the duration of the timeout (thinking 2 hours, plenty for normal secure online work).
It's not perfect but it's the simplest solution I came up with that doesn't involve heavy scripting that could interfere with all out other scripting for failover and general admin tasks.

Has anyone else faced this problem? What was your solution? I'm open to suggestions.

Re: TLS 1.3 + dual WAN session drops

Posted: Tue Oct 08, 2019 9:22 pm
by Sob
What about something a little more advanced than nth, like PCC (https://wiki.mikrotik.com/wiki/Manual:PCC)?

Re: TLS 1.3 + dual WAN session drops

Posted: Wed Oct 09, 2019 5:12 am
by NetWorker
Hey Sob, thanks for replying. The one sentence that put me off from using PCC in the first place was that at the very beginning it reads "PCC matcher will allow you to divide traffic into equal streams". Since there's nothing symmetric about our links (different bandwidth) or traffic (requirements are all over the place) I kinda skipped it and didn't look back. I'll give it a spin using either both addresses, dst address and port or a combination thereof.

Are you using some sort assymetric load balancing scheme by any chance?

Re: TLS 1.3 + dual WAN session drops

Posted: Wed Oct 09, 2019 6:15 am
by Sob
Don't worry, it's not exactly true. First, equal streams is more theoretical, maybe for longer term average it would be that way, but that's with any connection-based load balancing. But, which is more interesting for you, you can always have something like ten "equal streams" and send one to WAN1, three to WAN2 and remaining six to WAN3, and cover your asymmetric requirements this way. You're doing the same with nth, don't you?

I don't currently have PCC anywhere myself, but I did use it in the past and it worked. Good thing is that you have several choices for per-connection-classifier option, so you can select one that will work for you.

Re: TLS 1.3 + dual WAN session drops

Posted: Wed Oct 09, 2019 4:40 pm
by NetWorker
Gotcha. Never realized it works exactly the same as nth only with the added benefit of the PCC field. I'll set it up and see if the screams coming out of the accounting the department get any louder lol.

Re: TLS 1.3 + dual WAN session drops

Posted: Wed Oct 09, 2019 10:51 pm
by NetWorker
Update: here's what I ended up with.
;;; mark https ISP1
chain=prerouting action=mark-connection new-connection-mark=ISP1 passthrough=yes connection-state=new
protocol=tcp connection-mark=no-mark dst-port=443 per-connection-classifier=both-addresses:3/0 log=no
log-prefix=""

;;; mark https ISP2
chain=prerouting action=mark-connection new-connection-mark=ISP2 passthrough=yes connection-state=new
protocol=tcp connection-mark=no-mark dst-port=443 per-connection-classifier=both-addresses:1/0 log=no
log-prefix=""
First rule is to mark one third of https connections to be later assigned the ISP1 routing mark , the second is to mark all the remaining traffic for ISP2. I decided to use both addresses rather than src-address only to add randomness and better devide traffic among internet side interfaces but we could still end up with broken sessions if there's any website that has content hosted on different IPs. I'm not sure if this is possible with TLS 1.3. I'd think not since it makes no sense to me to have integrity checks (IP address) on one side and not the other.

Anyway, I added these before the rules that mark the rest of the connections (left in place as is with NTH for time being). We'll see how these work out.

Re: TLS 1.3 + dual WAN session drops

Posted: Thu Oct 10, 2019 5:40 am
by Sob
You can leave out per-connection-classifier=both-addresses:1/0, it matches everything and doesn't do anything useful. The connection-mark=no-mark is enough to make sure that only packets not marked by any previous rule will be marked by this one.

Re: TLS 1.3 + dual WAN session drops

Posted: Thu Oct 10, 2019 7:28 pm
by NetWorker
Lol, right.

I disabled my old rules yesterday and asked everyone that reported issues with encrypted websites to check those they usually work with and so far so good.