Community discussions

MikroTik App
 
pasmon
just joined
Topic Author
Posts: 2
Joined: Wed Mar 16, 2022 12:51 pm

Use of BGP for Kubernetes hosted web services

Thu Mar 17, 2022 12:01 pm

Hi experts,

I have 2 Raspberry PIs running Kubernetes (k3s actually) and they will be hosting various web applications. Network setup is so that ISP is directly connected to my MikroTik Hex router, and everything at home has IP from the Hex's default range of 192.168.88.0/24. Because I'm overly ambitious, I've also configured MetalLB load balancer with BGP to work with Hex, so that my web services get IP addresses automatically from 192.168.100.0/24 range.

Now I can port forward everything coming to WAN interface with port 443 to certain IP address like 192.168.100.1 and it works fine for one service, but I'm missing probably something quite obvious to get this working automatically so that everything coming in to port 443 gets forwarded to 192.168.100.0/24 so that BGP/MetalLB/whatnot can route the connection to the right service - what am I missing?

Thank you in advance!
 
eduplant
Member Candidate
Member Candidate
Posts: 139
Joined: Tue Dec 19, 2017 9:45 am

Re: Use of BGP for Kubernetes hosted web services  [SOLVED]

Thu Mar 17, 2022 11:59 pm

If I'm reading your scenario correctly, this is mostly a NAT problem. Part of the point of having one IP per service in a load balancer scenario is that each service can use the standard ports. If you crush the entire range of IPs and map it back onto one IP via NAT (technically NAPT is the problem here, stateless 1:1 NAT would be fine) then you necessarily lose all of the benefit. I don't think you're missing anything, I think it's just fundamentally not possible.

At least ... not possible at the network layer. If all of these services are actually HTTP(S), then the answer is probably at the application layer. That is to say, maybe stand up an HTTP(S) reverse proxy and port forward 80/443 on the outside of the NAPT to that. The proxy can then use the headers (Host, etc) to then open its own connections to each of the service IPs on the standard ports. Of course if some of these services aren't HTTP(S) and don't have an equivalent application layer proxy, then you're out of luck.

(I'm assuming the downstream applications for these don't know how to use SRV records which could specify both an address and a port. Oh the things that we have lost to NAT and the homogenization of HTTP-everything...)
 
pasmon
just joined
Topic Author
Posts: 2
Joined: Wed Mar 16, 2022 12:51 pm

Re: Use of BGP for Kubernetes hosted web services

Fri Mar 18, 2022 12:35 pm

Thank you for your contribution, much appreciated!

For now, I've changed my setup so that only the Traefik ingress service is getting the IP from MetalLB via BGP, and so I only have to setup that one IP to the router's NAT to get every web service reachable from internet. This works fine for me currently and probably long into the future as I'm not planning to have so much services hosted anyway.
 
elico
Member Candidate
Member Candidate
Posts: 143
Joined: Mon Nov 07, 2016 3:23 am

Re: Use of BGP for Kubernetes hosted web services

Sun Nov 20, 2022 11:18 pm

Hi experts,

I have 2 Raspberry PIs running Kubernetes (k3s actually) and they will be hosting various web applications. Network setup is so that ISP is directly connected to my MikroTik Hex router, and everything at home has IP from the Hex's default range of 192.168.88.0/24. Because I'm overly ambitious, I've also configured MetalLB load balancer with BGP to work with Hex, so that my web services get IP addresses automatically from 192.168.100.0/24 range.

Now I can port forward everything coming to WAN interface with port 443 to certain IP address like 192.168.100.1 and it works fine for one service, but I'm missing probably something quite obvious to get this working automatically so that everything coming in to port 443 gets forwarded to 192.168.100.0/24 so that BGP/MetalLB/whatnot can route the connection to the right service - what am I missing?

Thank you in advance!
Hey,

Can you share some of the BGP config for the k8s?
In your case it might not work since you have only 1 external IP address but I'm interested in the basic setup for metalLB and Mikrotik.

Who is online

Users browsing this forum: No registered users and 14 guests