Hello.
I’m trying to setup redundancy with Hyper-V CHR instances. The desired configuration is
ISP switch (single public IP) → VLAN4000 → My stackable switches → LACP Bonding to server
→ Windows Server with Intel X520-2 (both ports bound in LACP) → vSwitch with SR-IOV enabled VMQ enabled
→ CHR instance (network adapter in tagged mode all VLANs for now) → Inner network multiple VLANs
VLAN setup (at least what I want to setup first)
VLAN 4000 - ISP passthrough through switches
VLAN 20 - Management devices, i.e. switches, APs and so on, IP subnet > 10.20.0.0/24
VLAN 100 - LAN PCs, IP subnet > 10.100.0.0/24
Everything except CHR instance has already been set up and running with another set of VMs. The problem is that I can’t setup VRRP inside CHR itself. Without VRRP eveything seems to work fine. At first, no single configuration worked at all, then I found that I should enable MAC Spoofing in VM settings. After that, I had some success but still with problems. What I’ve tried (ping tests are done from neighbor Windows Server VM on the same host):
Attempt 1:
This one I saw from one of the guides, and it seems desired for me, but I had zero success with such configuration. It uses only one instance of VRRP and adds VLANs on top of it.
Create VRRP interface on top of ether1.
Assign 192.168.123.1/24 to ether1 and 192.168.123.254/32 to vrrp (those are non-existing IPs just for VRRP communication, according to guide).
Create VLAN 20 and VLAN 100 on top of vrrp.
Assign 10.20.0.100/24 and 10.100.0.100/24 respectively (those then supposed to turn into 0.1 as gateways when everything gets working) Result: vrrp shows RM, but both VLANs IP are not pingable at all.
Attempt 2:This one involves adding VRRP instance on top of each VLAN separately.
Create VLAN 20 and VLAN 100 on top of ether1 (ether1 has no address in this attempt).
Assign 10.20.0.254/24 and 10.100.0.254/24 respectively.
Create VRRP-20 on top of VLAN 20 and VRRP-100 on top of VLAN 100.
Assign 10.20.0.100/32 and 10.100.0.100/32 respectively. Result:
This configuration has very odd results for me. While configuration is similar, both configured VLANs work different:
VLAN 100 works fine, both 10.100.0.254 and 10.100.0.100 are pingable, I’ve even managed to make it work together with hardware MikroTik running 6.49.x, so they reply with 10.100.0.100 per their priority as intended.
VLAN 20 works odd. After everything enabled, 10.20.0.254 stops responding to pings at all. 10.20.0.100 accepts pings though, but I can’t for example connect to CHR instance using WinBox with “connection refused” message, while 10.20.0.254 responds with “connection timeout”
Both attempts are done with clear 7.15.3 CHR instance without any configuration except those described above. What is wrong in my actions done in both attempts, and how can I make it work (preferably #1)?
I am with problems in VRRP + VMs too.
It was working very well, until some updates.
I think I will need to packet sniffer and see in wireshark what is going on.
My VM always assume master, as if it is not communicating well with slave to see priority and so on.
Weird.
you should run VRRPs inside each VLAN
if you are working with group-authority on one VRRP (and there only can be one if there are multiple VRRPs) the vrrp group-authority only works correct on the lowest VLAN ID (found that one out last week after about 2h of “wiresharking” and eliminating all nooks and crannies of possible problems
(checkpoint’s “ClusterXL” has a similar “bug” or behaviour when clustering multiple VLANs of configure HA for more VLANs)
I think my problem is more basic tho (or more complex with host configuration), since I only have one CHR instance for testing now and VRRP has no backups. I did what you suggest (vrrp-per-vlan) in attempt 2 and had problems even with this - I did not use group authority (yet, but I wish to). I wanted to get there step-by-step, but even the first steps are failing.
I've also created a ticket to support, and they suggest to try it out without LACP to server. But as I stated in first post, I'm trying to ping vm from neighbor vm on same host, so I doubt it's LACP, maybe SR-IOV is not fully supported.
No such things. I had no success at all until I googled for esxi that mac spoofing should be enabled and found where to fix that for hyper-v. Now it's this. Gonna give it a try on a sr-iov disabled vswitch when I have more time
Ok, after some investigation I’ve found out that:
Attempt 2: was a stupid misconfiguration by me, where I duplicated vlan20-vrrp IP and static IP of another device in subnet. Everything works.
Attempt 1: I managed to get it working by:
Create a bridge. Add ether1 to bridge. Assign unused subnet IP to bridge (ex. 192.168.123.2/24 and 192.168.123.3/24 for another node). Configure filtering.
Create VRRP on top of bridge, assign IP 192.168.123.1/32
Put all VLANs on top of VRRP, assign desired IPs directly to VLAN interfaces. So the final scheme looks like this:
-bridge
—vrrp
------vlan20
------vlan100
-ether1 (slave to bridge)
The problem with this though is that vrrp frames seem to be untagged, so neither bridge with “allow only tagged”, nor switches between nodes that configured to also only allow tagged frames will not pass the traffic. So the new and probably last question - is it possible to make VRRP frames tagged in such configuration?
It sounds like you’ve put in a lot of effort! For Attempt 1, make sure VRRP is properly configured with the right IPs and VLANs. Try verifying that VRRP communication IPs don’t conflict with other network IPs and that VLANs are correctly tagged on the interface.
For Attempt 2, it seems like VLAN 20’s issue could be due to misconfigured VLAN settings or potential IP conflicts. Double-check the VLAN tagging and make sure there’s no overlap or incorrect configuration causing the ping issues.
Also, ensure that MAC Spoofing is consistently enabled across all VMs involved.
Perhaps post at least some config snippets, especially the bridge configuration. The /interface/bridge/vlans stuff is confusing, and easy to get wrong.
Although your original and simple:
ether1
** /interface/vlan interface=ether1
*** /interface/vrrp interface=
should have worked. And may have given up too soon on this approach, since if you just have only one port to hyper-v host, avoiding the bridge likely be better.
I’m more familar with VMWare+CHR, and VRRP works on bridged VLANs but needs “promiscuous mode” ienable in VMWare… but I’m not familiar with Hyper-V, so perhaps there more setting beyond MAC spoofing. i.e. just WAG for example… Hyper-V may only allow UDP+TCP+ICMP packets… while VRRP using a different top-level IP protocol than UDP/TCP/ICMP, proto=112. Or similar, if multicast is not enabled/blocked/etc. I don’t know the config of Hyper-V but those are two main things that hypervisor need to allow for VRRP (beyond a different MAC).
There is a promiscuous mode in Hyper-V, however, it seems that just enabled spoofing and trunk mode on a virtual port is enough. At least this seem to work between two CHR instances on a single Windows 11 hyper-v host - they failover correctly with just one packet lost. Will try it out with separate physical hosts in a while.