We’re setting up a router in a building that will house a few company tenants, each with their own router. (which are managed by the tenants) Our router connects to the Internet, and we have been assigned a few public IP addresses, which we in turn will assign to each of our tenants. We’re going to set up a bridge on the router, so that the tenants are directly connected to the Internet, using the ISP as their gateway. So far, so good. The problem that arises is that we’ll want to make sure that each tenant only ever uses a specific IP address, not to conflict with each other. How can I achieve this?
I’ve seen suggestions to configure interfaces for ARP replies only, but I think this is only relevant when the router (or “switch”, in this case) itself acts as the gateway.
Whether we need to use a DHCP server or have the tenants set addresses statically doesn’t matter.
By using arp=reply-only on the gateway & DHCP server combo you effectively prevent the tenants from using any other address than the one assigned to them by DHCP, as there will be no ARP record for them, so they can send packets with that incorrect IP address as source but the responses will never reach them due to the missing ARP record. But this doesn’t prevent them from sending from these addresses if they don’t care about responses.
On a switch or wherever the gateway and the DHCP server are not colocated, the solution is ingress filtering by IP address. The specific commands depend on the switch model - CRX1xx/CRS2xx swiches and CRS3xx switches use different switch chips with different way of configuring things like this.
Consider distributing the tenant IPs by static DHCP leases (no dynamic pool) based on the MAC addresses of tenant upstream interfaces. This gives you control over the IP/MAC relations. But MAC addresses can be changed easily, so a tenant could in theory use a MAC address registered for another tenant.
If this is not acceptable, the ROS bridge filtering feature (->Bridge->Filters) can be used to drop all packets not going to / coming from the correct MAC address of the tenant belonging to a physical bridge port. (Requires each tenant connected to different physical port of the bridge). This effectively ties a physical port to a predefined MAC address. Be sure not to block Layer2 broadcasts (used for DHCP and ARP).
On some Router models enabling bridge filtering disables Layer2 HW acceleration.
If the bridging of the tenants to the ISP is the only thing the box does, a manageable switch running SWOS might be the better choice than a ROS router box. Manageable switches can filter MAC addresses at wire speed.
Returning to this, with slightly different requirements...
All addresses are statically set, and I need to do this on a CRS 300 series switch, without going through CPU. Will ingress filtering do that? I'm not sure what it refers to, exactly. Is it `/interface ethernet switch rule, with an empty new-dst-ports= to drop appropriate traffic? Would I need to enable L3 hardware offloading, or something?
Yes, as long as your filtering requirements can be expressed with the packet matching rules available in /interface ethernet switch rule, empty new-dst-ports= allows dropping packets without CPU intervention. It happens inside the switch ASIC.
Enabling L3 hardware offloading is not required. This is only for making forwarding decisions based on L3 addresses aka l3hw, which is a different topic.
This comes up from time to time, always with the same proposed solutions, and the same working but imperfect results. I'm going to disagree a bit, although it's probably a bit like yelling at clouds and tilting at windmills.
This sort of thing is properly done as a routed setup. One should simply establish a default towards the modem/terminal device, and /32 towards the clients, and the rest works itself out. Depending on the bandwidth required, fasttrack, fastpath or L3HW are all appropriate.
The problem with these almost-a-bridge-but-really-routed (also referred to as "bump in the wire") is always around ARP and broadcast traffic. Manufacturers have been proposing protocol-aware stacks for solving some of these issues, such as ARP inspection, which must play catch up with any and all protocol dialects and variations. Routed setups seem intimidating to many, but they aren't ultimately complex, and once properly configured, present a natural solution.
I am open to the idea that this is not an ideal solution, but I don't fully understand what you are proposing, given that clients are currently connected directly into this switch.
Very simply, you establish the connection to your ISP in the way they specify. Usually you get a "normal ip" and you get the additional addresses routed through that. This would look something like this:
... and so on. This can be combined with static ARP entries or full proxy-arp if you want your clients to see the ISP gateway instead of your device, etc. IP use can then be enforced simply by turning on RPF, or via firewall rules, etc. Even switch rules work correctly from this point onward, because broadcast traffic is not routed.
Just a unrelated observation: in your switch rules, you have to specify the ports for the last drop rule (as all of your client ports) in order to not drop incoming packets from you ISP.
I'm having a hard time picturing the topology. Which bit is preventing a rogue client A from using the IP address of client B, thereby deceiving other peers into sending traffic destined for client B to itself instead, when the traffic doesn't need to leave the immediate switch?
You're right, of course. sfpplus1 is the uplink in this example, so I would remove the src-address constraint from that rule.
Again, the topology is not switched/bridged anymore, but routed. The configuration for the client ports, as written, now for two of them, A and B:
# A
/ip address add interface=ether2 address=192.168.0.20/32 network=10.0.0.20
# B
/ip address add interface=ether3 address=192.168.0.20/32 network=10.0.0.21
results in the dynamic routes:
10.0.0.20/32 via ether2 on-link
10.0.0.21/32 via ether3 on-link
This specifically means that all traffic for 10.0.0.20 will be sent out on only ether2, and similarly for 10.0.0.21 on ether3.
Again, if you engage RPF, that ensures that only packets with source addresses where the reverse packet would be routed through that interface are accepted, that is: from ether2 all packets with source addresses other than 10.0.0.20 are rejected, similarly for ether3 all packets with source addresses other than 10.0.0.21 are rejected.
The "B pretending to be A and having traffic destined for A sent to them" situation is a specific case of ARP poisoning, which is, again: only possible on L2 topologies, so bridges, switches, etc. Routed setups are by default and by their nature immune. They direct traffic according to their routing table.
This is a really common setup for multi-tenant distribution. I see it used often in server hosting (where different customers have a small number of servers - usually one - colocated). The setup does involve switches.
The switches are either configured for vlans or port isolation (Cisco naming: PVLAN). Usually the routing is done by the L3 functionality of a switch as well, which is also present on the crs300 devices.
A proper router of course provides more functionality, but in these setups people mostly go for cheap and line-rate, and then if someone wants more advanced firewalling, that can be done behind this distribution layer.