Community discussions

MikroTik App
 
tangent
Forum Guru
Forum Guru
Topic Author
Posts: 1331
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

How can I bind a container straight to the bridge?

Mon Apr 03, 2023 4:44 am

In support of another project, I was trying to avoid the NAT layer in the standard container setup. I succeeded in adding veth1 straight to the single hardware bridge, avoiding the software "docker" bridge that MT recommends in their docs, but where I failed is in giving it an IP address in the same subnet as the router itself, then disabling the recommended NAT rules. I thought that would put the container's IP as an alias on the bridge, and that would allow external connections straight into the container, but no joy.

The setup isn't complicated:

/interface veth
add address=192.168.88.2/24 gateway=192.168.88.1 name=veth1
/ip address
add address=192.168.88.1/24 interface=bridge1 comment="the router itself"
add address=192.168.88.2/24 interface=bridge1 comment="container veth"
/interface bridge port
add bridge=bridge1 interface=veth1
/container
add interface=veth1 logging=yes start-on-boot=yes

Although this is an RB4011, I'm using it as a glorified switch, without any firewall rules of consequence.

The new .2 IP does ping, but attempting to connect to the containerized service gives a "connection refused" error.

Why doesn't this work? Is there anything I can change to make it work? NAT's rather silly for this purpose, IMHO.
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 3169
Joined: Sun May 01, 2016 7:12 pm
Location: California

Re: How can I bind a container straight to the bridge?  Topic is solved

Mon Apr 03, 2023 4:57 am

AFAIK, the IP and gateway of veth are essentially copied into the container. So if just put the VETH in bridge, no /ip/address for .2 is needed – the bridge will use ARP to figure that out.

e.g.
/ip address
add address=192.168.88.2/24 interface=bridge1 comment="container veth"
 
tangent
Forum Guru
Forum Guru
Topic Author
Posts: 1331
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: How can I bind a container straight to the bridge?

Mon Apr 03, 2023 5:19 am

the bridge will use ARP to figure that out.

That's what I thought, and I'm certain I tried something like that.

Simply removing the IP from the bridge and restarting the container didn't help. I had to reboot the router to get it working. I guess there was stale routing information in there or something. This might explain my initial failures.

If nothing else, having the confirmation that it could indeed be done gave me the push I needed. Thank you!

Getting rid of the NAT layer did help a smidge. My tests are now up to 3.4 Gbit/sec. Woo!
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 3169
Joined: Sun May 01, 2016 7:12 pm
Location: California

Re: How can I bind a container straight to the bridge?

Mon Apr 03, 2023 5:33 am

I'm not 100% it always picks up IP/gateway if changes, at least seen once or twice (e.g. container IP was using previous IP, not a new one set in VETH) – so that's what may have happened, especially if you fiddling with the setting ;).

I don't know if it's helpful I but I built a netserver (and netperf) containers:
https://github.com/tikoci/netserver
The old HP netserver is what's required for flent tests. I hadn't had time to write any docs or do much tests. There is a script that install/remove it however.
It works in my limited testing, but I have it running RB1100 with SSDs so I wasn't too concern by size.
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 3169
Joined: Sun May 01, 2016 7:12 pm
Location: California

Re: How can I bind a container straight to the bridge?

Mon Apr 03, 2023 5:45 am

I had to reboot the router to get it working. I guess there was stale routing information in there or something.
The ARP cache likely...it would have had the router bridge MAC stored I suspect.
 
tangent
Forum Guru
Forum Guru
Topic Author
Posts: 1331
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: How can I bind a container straight to the bridge?

Tue Apr 11, 2023 3:40 am

For the archives, there's a potential problem with doing this: by putting the veth into the bridge, it participates in ROS's logic for deciding what MAC to give the bridge. I don't understand its rules, but I do know that in one test of this here, it ended up giving my bridge a different random MAC on each reboot because the veth kept winning this "election" process. I had this switch configured for static DHCP addressing for management reasons, which meant it stopped getting the IP I assigned it. I had to put it on pure static addressing to work around this weirdness.
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 3169
Joined: Sun May 01, 2016 7:12 pm
Location: California

Re: How can I bind a container straight to the bridge?

Tue Apr 11, 2023 4:31 am

it ended up giving my bridge a different random MAC on each reboot because the veth kept winning this "election" process.
Is your bridge is using auto-mac=yes? If so, you can use something like this is to control the MAC address of the bridge so it wouldn't pick VETH as the bridge MAC:
/interface/bridge/set bridge1 auto-mac=no admin-mac=[/interface/get [find default-name=ether2] mac-address]
But yeah the docs are vague on how the "automatic" bridge MAC works...

If it's something like lowest MAC, the VETH seem to pick a very random MAC addresses in the few I just checked... But auto-mac=no should avoid, I think...
 
tangent
Forum Guru
Forum Guru
Topic Author
Posts: 1331
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: How can I bind a container straight to the bridge?

Tue Apr 11, 2023 10:17 am

Thanks again, Amm0.

I'll stick with static IP on this switch. It is, after all, core infrastructure. I just thought I'd be cute and allow whole network re-IPing by using DHCP as much as possible. I've been made to change my subnet more than once over the years.

What I really want is for RouterOS to leave veth devices out of the logic entirely. In what world is assigning a veth's random MAC to the bridge ever the right answer?

I'm half-tempted to file a support request on this. The current behavior seems pretty clearly wrong.
 
User avatar
bpwl
Forum Guru
Forum Guru
Posts: 2978
Joined: Mon Apr 08, 2019 1:16 am

Re: How can I bind a container straight to the bridge?

Tue Apr 11, 2023 11:27 am

bridges and IP and MAC addresses ...

Even in non-container setups , it behaves like:

- only the bridge has an IP address. Any IP address given to some (slave interface) bridge-port will be an IP address of the bridge
- the bridge takes the MAC address of the interface of the first active interface-bridge-port. Whatever that one is, and not very clear how "first" is defined.
- some interfaces go up/down like wireless interfaces depending on clients connected or not
- some interfaces have no MAC address, like PPTP. (They need to be connected to a bridge for BCP)
- to avoid MAC address changes and losing connection with e.g. WinBox we set the "Admin MAC Address" of the bridge

This would mean that a specific bridge port cannot be chosen based on IP-address of the interface (slave interface has no IP-address).
IP route needs a non-slave interface to define an interface as gateway.
The bridge "Hosts" table contains the slave interface mapping for MAC addresses. Manual entries are possible. (With HW offloading, some are also in the Switch Host table)

WLAN "station pseudobridge" is such an implementation of multiple differentiating IPv4 addresses and one common MAC address. Not very stable.
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 3169
Joined: Sun May 01, 2016 7:12 pm
Location: California

Re: How can I bind a container straight to the bridge?

Tue Apr 11, 2023 5:06 pm

What I really want is for RouterOS to leave veth devices out of the logic entirely. In what world is assigning a veth's random MAC to the bridge ever the right answer?
I tend to agree. MT has suggested/recommend auto-mac=no for a while. But auto-mac=yes selecting a VETH as a bridge's MAC does seems problematic. No doubt it be "safer" if bridge perfered real ethernet interface to virtual ones FIRST in it's auto-mac selection logic.

Note: Re read docs on your question. I would guessed it was lowest MAC like VRRP – that's wrong. My incorrect thought was the "random MAC address" could be above/below Mikrotik's MAC OIDs. Anyway, see more accurate posts above on "port number". And while I say "random", they do seem use LAA one (local admin scope) as the MAC, which is correct.

I'm half-tempted to file a support request on this. The current behavior seems pretty clearly wrong.
It can't hurt. I don't think it make sense to use VETH as bridge MAC either (unless that was the only choice).
 
tangent
Forum Guru
Forum Guru
Topic Author
Posts: 1331
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: How can I bind a container straight to the bridge?

Wed Apr 12, 2023 5:21 pm

I filed the suggestion, and MikroTik says they'll think about changing the behavior.

In other news, I have a more complete set of test results up now. I was surprised at the 29% best-case margin between the hAP ax lite and the ax². I thought it'd be more nearly night-and-day due to the 64-bit CPU and quad cores, giving something like double the speed. Or, since the "lite" does well over half a gigabit, I'd've liked to see the ax² pegging the needle, as when you use iperf3 between desktop CPUs.

Who is online

Users browsing this forum: tangent and 4 guests