I’m setting up a QEMU based KVM on an Ubuntu 22.04.01 LTS server. My network is such that, the uplink comes to a Mikrotik RB2011UiAS-2HnD on the SFP interface, and then goes out on the RJ45 Ethernets ports, which are all configured under a bridge. The SFP is out side the bridge, as it is the uplink. SFP has the real IP on a x.y.z.78/30 network which is used for a point to point connection. I have another set of a.b.c.232/29 IPs routed over to SFP IP. I have assigned a.b.c.233/29 on the bridge interface, it is pingable from outside. Additionally, 192.168.1.1 is a private network configured on the bridge, with a /24 network configured. There is a standard masquerade rule, which was entered by the Quick Setup guide.
The Ubuntu 22.04.01 LTS is working fine, and getting an IP from the 192.168.1.0/24 network, which I have statically assigned in DHCP Server to be 192.168.1.7. Now, I want to assign real IPs to the VMs. Each of the VMs are pulling their DHCP nicely from the Mikrotik. However, they are all being masqueraded as per the standard config:
However, this fails the purpose of real IP, as they are also being natted and show the router’s x.y.z.78/30 when we do a “what is my IP” search on Google.
So, I change this config to only do masquerading on the 192.168.1.0/24 network as follows:
Open a command line window on the 2011, make it as wide as your screen allows, run /tool sniffer quick ip-protocol=icmp ip-address=8.8.8.8 in that window, and start pinging 8.8.8.8 from one of the VMs. The sniffer will show you from what address the VM actually pings and where does the 2011 send the packets, whether any responses from 8.8.8.8 come back and if so, where does the 2011 send them.
Thank you for your kind reply! I did just that, and found this output:
INTERFACE TIME NUM DIR SRC-MAC DST-MAC VLAN SRC-ADDRESS DST-ADDRESS PROTOCOL SIZE CPU FP
ether1 205.915 121 <- 52:54:00:A5:2B:6C DC:2C:6E:DA:72:AA a.b.c.234 8.8.8.8 ip:icmp 74 0 no
bridgeLocal 205.915 122 <- 52:54:00:A5:2B:6C DC:2C:6E:DA:72:AA a.b.c.234 8.8.8.8 ip:icmp 74 0 no
sfp1 205.915 123 -> DC:2C:6E:DA:72:A9 6C:6C:D3:66:E5:58 a.b.c.234 8.8.8.8 ip:icmp 74 0 no
The DST-MAC on the sfp1 line is the same MAC as the remote point to point end. Does it mean that Mikrotik is routing properly, and it’s the upstream’s problem?
Yes. The source address of the packet as it leaves through the SFP is the correct (public) one, so no NAT took place, and the destination MAC address is the one of gateway, so everything is as it should be. You may try to ping some other device with a public address at which you can sniff to see whether already the requests do not make it to the destnation or whether only the responses are misrouted/dropped. In any case, try pinging other destinations than 8.8.8.8 first. Also, try sniffing with ip-address=a.b.c.234 at the Mikrotik while pinging a.b.c.234 from outside, or while trying to open a.b.c.234 from a browser on your mobile phone. The ip-protocol in the sniffing filter would be tcp in the latter case, not icmp.